2024-09-10T10:25:07.838 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T10:25:07.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:25:07.955 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7899033' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKyCWG8mckh+7ZE/U7l4W3HoRqbibuLZFJXIk4JKxHB++HiJFYpxh0l2jN1kIdIAIPrqXJEQGrX/RE+cPYQcAc= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRAHaqJuB5MT0tioqnxnrsgd0VGjzqWDK3Au1uXnN869i/0H/UM3fwvxhBrw977iC+pt7RhkTcW6fC/Tqg28qA= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFYC/AJqbV068MGKPMhJw1dxHQn5ocus655ZD2tMH1MAxt7TupdIJBrhUlwv+kb7vEp4cupvPxESqNuriiwSsg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T10:25:07.956 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T10:25:07.956 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T10:25:07.957 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T10:25:07.958 INFO:teuthology.task.internal:Checking packages... 2024-09-10T10:25:07.979 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T10:25:07.979 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T10:25:07.979 INFO:teuthology.packaging:ref: None 2024-09-10T10:25:07.979 INFO:teuthology.packaging:tag: None 2024-09-10T10:25:07.980 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:25:07.980 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:25:07.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:25:08.128 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T10:25:08.129 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T10:25:08.139 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T10:25:08.139 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T10:25:08.144 INFO:teuthology.task.internal:Saving configuration 2024-09-10T10:25:08.164 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T10:25:08.170 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T10:25:08.194 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:19:52.133361', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKyCWG8mckh+7ZE/U7l4W3HoRqbibuLZFJXIk4JKxHB++HiJFYpxh0l2jN1kIdIAIPrqXJEQGrX/RE+cPYQcAc='} 2024-09-10T10:25:08.217 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:19:52.135672', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRAHaqJuB5MT0tioqnxnrsgd0VGjzqWDK3Au1uXnN869i/0H/UM3fwvxhBrw977iC+pt7RhkTcW6fC/Tqg28qA='} 2024-09-10T10:25:08.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:19:52.134748', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFYC/AJqbV068MGKPMhJw1dxHQn5ocus655ZD2tMH1MAxt7TupdIJBrhUlwv+kb7vEp4cupvPxESqNuriiwSsg='} 2024-09-10T10:25:08.242 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T10:25:08.248 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T10:25:08.248 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T10:25:08.248 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T10:25:08.248 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T10:25:08.332 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4ce2707be0>, signals=[15]) 2024-09-10T10:25:08.333 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T10:25:08.339 INFO:teuthology.task.internal:Opening connections... 2024-09-10T10:25:08.339 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T10:25:08.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:25:08.415 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T10:25:08.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:25:08.490 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-09-10T10:25:08.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:25:08.566 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T10:25:08.573 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-09-10T10:25:08.592 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-09-10T10:25:08.592 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-09-10T10:25:08.650 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:25:08.651 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:25:08.651 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:25:08.651 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:25:08.651 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:25:08.651 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:25:08.651 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-09-10T10:25:08.673 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-09-10T10:25:08.692 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-09-10T10:25:08.692 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-09-10T10:25:08.751 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-09-10T10:25:08.752 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:25:08.753 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:25:08.753 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:25:08.753 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:25:08.753 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:25:08.753 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:25:08.753 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-09-10T10:25:08.774 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-09-10T10:25:08.793 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-09-10T10:25:08.793 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-09-10T10:25:08.851 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-09-10T10:25:08.851 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-09-10T10:25:08.851 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:25:08.852 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:25:08.853 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-09-10T10:25:08.873 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T10:25:08.884 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T10:25:08.890 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T10:25:08.890 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:25:08.894 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:25:08.896 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:25:08.913 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T10:25:08.919 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T10:25:08.919 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:25:08.952 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:25:08.955 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:25:08.995 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T10:25:09.073 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T10:25:09.089 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T10:25:09.090 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T10:25:09.090 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:25:09.091 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:25:09.091 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:25:09.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:25:09.109 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-09-10T10:25:09.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:25:09.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:25:09.111 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-09-10T10:25:09.111 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-10T10:25:09.167 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:25:09.167 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-503.el9.x86_64 2024-09-10T10:25:09.167 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-09-10T10:25:09.169 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:25:09.169 INFO:teuthology.task.kernel:Running kernel on smithi143: 5.14.0-503.el9.x86_64 2024-09-10T10:25:09.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum install -y kernel 2024-09-10T10:25:09.170 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:25:09.170 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-503.el9.x86_64 2024-09-10T10:25:09.170 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-09-10T10:25:11.478 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-10T10:25:11.730 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T10:25:12.832 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-10T10:25:13.623 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T10:25:14.784 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-10T10:25:14.938 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 20 MB 00:02 2024-09-10T10:25:18.950 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-10T10:25:19.298 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-10T10:25:20.221 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-10T10:25:20.732 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-10T10:25:21.143 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-10T10:25:21.504 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T10:25:21.647 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 19 kB 00:00 2024-09-10T10:25:22.313 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T10:25:22.393 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-10T10:25:26.884 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-10T10:25:27.802 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T10:25:27.832 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T10:25:28.874 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T10:25:28.874 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T10:25:28.960 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-10T10:25:28.971 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-10T10:25:28.971 INFO:teuthology.orchestra.run.smithi143.stdout: Package Architecture Version Repository Size 2024-09-10T10:25:28.971 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout:Install 4 Packages 2024-09-10T10:25:28.972 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:25:28.973 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 79 M 2024-09-10T10:25:28.973 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 126 M 2024-09-10T10:25:28.973 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-10T10:25:29.619 INFO:teuthology.orchestra.run.smithi143.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2024-09-10T10:25:29.851 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T10:25:29.851 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T10:25:29.879 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T10:25:29.880 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T10:25:29.947 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T10:25:29.951 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:25:29.952 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 79 M 2024-09-10T10:25:29.953 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 126 M 2024-09-10T10:25:29.953 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T10:25:29.968 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:25:29.979 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T10:25:29.980 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T10:25:29.980 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-09-10T10:25:29.980 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:25:29.980 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 79 M 2024-09-10T10:25:29.980 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 126 M 2024-09-10T10:25:29.980 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T10:25:30.137 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 267 kB/s | 23 kB 00:00 2024-09-10T10:25:30.166 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 264 kB/s | 23 kB 00:00 2024-09-10T10:25:30.511 INFO:teuthology.orchestra.run.smithi143.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T10:25:30.512 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-10T10:25:30.921 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 40 MB/s | 35 MB 00:00 2024-09-10T10:25:31.153 INFO:teuthology.orchestra.run.smithi143.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-10T10:25:31.238 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-10T10:25:31.238 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:25:31.238 INFO:teuthology.orchestra.run.smithi144.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-10T10:25:31.922 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T10:25:31.928 INFO:teuthology.orchestra.run.smithi143.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-10T10:25:31.928 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:25:31.929 INFO:teuthology.orchestra.run.smithi143.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-10T10:25:32.264 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T10:25:32.264 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T10:25:32.618 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-10T10:25:32.972 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-10T10:25:32.972 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-10T10:25:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T10:25:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T10:25:33.959 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-10T10:25:33.959 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-10T10:25:34.016 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.0 MB/s | 16 MB 00:03 2024-09-10T10:25:35.673 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T10:25:36.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:25:36.434 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-10T10:25:36.474 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:25:37.124 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.1 MB/s | 29 MB 00:06 2024-09-10T10:25:37.141 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:25:37.239 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:25:37.949 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.4 MB/s | 35 MB 00:07 2024-09-10T10:25:37.950 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:25:37.950 INFO:teuthology.orchestra.run.smithi178.stdout:Total 9.9 MB/s | 79 MB 00:07 2024-09-10T10:25:38.631 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T10:25:38.962 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T10:25:38.962 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T10:25:39.254 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:25:39.400 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:25:39.900 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T10:25:39.900 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T10:25:40.038 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:25:40.182 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:25:42.360 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T10:25:43.056 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:25:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:25:44.249 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:25:44.376 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:25:45.010 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:25:45.041 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:25:45.939 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:25:46.080 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:25:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:25:49.882 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:25:50.916 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:25:50.963 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:25:55.857 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:27.228 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:27.268 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:26:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:26:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:26:28.268 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:28.308 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T10:26:28.505 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T10:26:28.506 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:26:28.506 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T10:26:28.673 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:28.673 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:26:28.673 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:26:28.673 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:26:28.886 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:26:29.014 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.014 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:26:29.014 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T10:26:29.015 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T10:26:29.015 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T10:26:29.015 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T10:26:29.015 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T10:26:29.015 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:26:29.074 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.074 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:26:29.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 82.el9 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8274670 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-09-10T10:26:29.118 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:26:29.119 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:26:29.120 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:26:29.120 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-505.el9.x86_64 2024-09-10T10:26:29.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:26:29.524 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-10T10:26:29.801 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-09-10T10:26:29.872 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:26:29.955 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:29.956 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:26:29.956 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T10:26:29.956 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T10:26:29.956 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T10:26:29.956 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T10:26:29.956 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T10:26:29.957 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:26:30.015 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:30.015 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:26:30.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2024-09-10T10:26:30.057 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.06 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 82.el9 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Group : Unspecified 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 8274670 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:26:30.058 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : builder@centos.org 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : CentOS 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:26:30.059 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:26:30.060 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:26:30.060 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2024-09-10T10:26:30.060 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:26:30.060 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:26:30.061 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:26:30.061 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 5.14.0-505.el9.x86_64 2024-09-10T10:26:30.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:26:30.711 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-09-10T10:26:31.108 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:26:31.146 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-09-10T10:26:31.148 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-10T10:26:31.163 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.m4hgHJfeKL 2024-09-10T10:26:31.164 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.m4hgHJfeKL 2024-09-10T10:26:31.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.m4hgHJfeKL 2024-09-10T10:26:31.393 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.m4hgHJfeKL is 6KB 2024-09-10T10:26:31.441 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.m4hgHJfeKL 2024-09-10T10:26:31.456 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:26:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:26:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:26:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:26:31.519 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T10:26:31.521 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T10:26:31.675 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-09-10T10:26:32.085 INFO:teuthology.orchestra.run.smithi143.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:26:32.105 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-09-10T10:26:32.107 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-09-10T10:26:32.122 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.Ufz0ksxQoL 2024-09-10T10:26:32.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ufz0ksxQoL 2024-09-10T10:26:32.197 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.Ufz0ksxQoL 2024-09-10T10:26:32.344 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.Ufz0ksxQoL is 6KB 2024-09-10T10:26:32.394 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.Ufz0ksxQoL 2024-09-10T10:26:32.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:26:32.471 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:26:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:26:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:26:32.472 INFO:teuthology.orchestra.run.smithi143.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T10:26:32.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T10:26:32.622 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2024-09-10T10:26:34.005 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:34.045 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:34.366 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:34.366 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:26:34.366 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:26:34.366 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:26:35.229 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:26:35.230 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T10:26:35.562 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:26:35.655 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:35.655 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:26:35.655 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T10:26:35.655 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T10:26:35.655 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T10:26:35.655 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-10T10:26:35.655 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T10:26:35.656 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:26:35.714 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:26:35.715 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:26:35.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-09-10T10:26:35.746 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 82.el9 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8274670 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:26:35.747 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:26:35.748 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:26:35.749 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:26:35.749 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 5.14.0-505.el9.x86_64 2024-09-10T10:26:35.749 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:26:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-09-10T10:26:37.691 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:26:37.723 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-09-10T10:26:37.725 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-10T10:26:37.741 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.ZC7Aqsbygh 2024-09-10T10:26:37.741 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZC7Aqsbygh 2024-09-10T10:26:37.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.ZC7Aqsbygh 2024-09-10T10:26:37.960 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.ZC7Aqsbygh is 6KB 2024-09-10T10:26:38.008 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.ZC7Aqsbygh 2024-09-10T10:26:38.023 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:26:38.085 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:26:38.086 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:26:38.086 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:26:38.086 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T10:26:38.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T10:26:38.241 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-09-10T10:27:01.694 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T10:27:01.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T10:27:01.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:27:02.626 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T10:27:02.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-10T10:27:02.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:27:08.247 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-10T10:27:08.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T10:27:08.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:28:01.752 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T10:28:02.630 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T10:28:08.255 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T10:28:10.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T10:28:10.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:28:11.231 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-10T10:28:11.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-10T10:28:11.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:28:11.649 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T10:28:11.649 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T10:28:11.650 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-10T10:28:11.664 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-505.el9.x86_64 2024-09-10T10:28:11.665 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T10:28:11.665 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:28:11.665 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:28:12.139 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-09-10T10:28:12.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-10T10:28:12.653 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T10:28:12.653 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-09-10T10:28:12.665 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T10:28:12.665 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T10:28:12.667 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-505.el9.x86_64 2024-09-10T10:28:12.667 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T10:28:12.667 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:28:12.668 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:28:12.738 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-09-10T10:28:12.763 DEBUG:teuthology.parallel:result is None 2024-09-10T10:28:13.668 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T10:28:13.668 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T10:28:13.750 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2024-09-10T10:28:13.782 DEBUG:teuthology.parallel:result is None 2024-09-10T10:28:17.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T10:28:17.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:28:33.116 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-10T10:28:33.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T10:28:33.501 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-10T10:28:33.501 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-09-10T10:28:33.515 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-505.el9.x86_64 2024-09-10T10:28:33.516 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T10:28:33.516 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:28:33.516 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:28:34.517 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-10T10:28:34.517 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T10:28:34.629 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-09-10T10:28:34.658 DEBUG:teuthology.parallel:result is None 2024-09-10T10:28:34.658 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T10:28:34.667 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T10:28:34.667 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:28:34.671 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:28:34.674 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:28:34.691 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T10:28:34.702 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T10:28:34.717 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T10:28:34.717 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:28:34.732 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:28:34.735 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:28:34.803 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T10:28:34.814 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T10:28:34.814 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:28:34.817 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:28:34.819 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:28:34.847 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:28:34.848 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:28:34.853 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:28:34.858 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:28:34.860 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:28:34.865 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:28:34.867 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T10:28:34.876 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T10:28:34.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:28:34.910 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:28:34.913 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:28:34.939 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T10:28:34.951 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T10:28:34.951 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:28:34.978 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:28:34.981 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:28:34.996 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:28:35.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:28:35.124 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:28:35.207 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:28:35.248 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:28:35.248 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:28:35.317 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:28:35.355 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:28:35.414 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:28:35.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:28:35.543 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:28:35.543 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:28:35.611 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:28:35.650 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:28:35.731 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:28:35.809 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:28:35.876 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:28:35.876 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:28:35.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-09-10T10:28:35.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-09-10T10:28:35.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-09-10T10:28:35.984 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:28:36.002 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:28:36.077 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:28:36.579 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T10:28:36.587 INFO:teuthology.task.internal:Starting timer... 2024-09-10T10:28:36.587 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T10:28:36.626 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T10:28:36.634 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T10:28:36.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-09-10T10:28:36.716 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-09-10T10:28:36.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-09-10T10:28:36.786 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-09-10T10:28:36.787 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-09-10T10:28:36.864 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-09-10T10:28:36.866 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T10:28:36.866 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-09-10T10:28:36.892 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-09-10T10:28:36.892 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-09-10T10:28:36.916 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-09-10T10:28:36.916 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-09-10T10:28:36.935 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-09-10T10:28:36.936 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi143.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-09-10T10:28:36.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:28:36.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:36.966 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:28:36.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:36.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:28:37.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:37.022 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T10:28:37.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:28:37.055 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:28:37.091 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:28:37.166 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T10:28:37.175 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T10:28:37.179 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T10:28:37.291 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T10:28:37.302 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T10:28:37.303 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi144.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T10:32:23.190 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-09-10T10:32:23.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-10T10:32:23.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:32:23.290 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-09-10T10:32:23.372 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-09-10T10:32:23.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T10:32:23.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:32:23.446 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-10T10:32:23.524 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T10:32:23.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T10:32:23.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:32:23.603 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-10T10:32:23.682 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T10:32:23.682 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T10:32:23.693 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T10:32:23.694 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:32:23.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:32:23.697 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:32:23.697 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:32:23.699 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:32:23.700 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:32:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:32:23.728 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:32:23.739 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:32:23.743 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:32:23.753 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:32:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-09-10T10:32:23.768 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-09-10T10:32:23.769 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:32:23.776 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-09-10T10:32:23.779 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-09-10T10:32:23.790 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:32:23.793 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:32:23.794 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-09-10T10:32:23.804 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:32:23.806 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-09-10T10:32:23.807 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:32:23.821 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:32:23.835 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:32:23.864 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:32:23.866 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:32:23.867 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:32:23.868 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-09-10T10:32:23.868 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.868 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.868 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.868 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.870 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:32:23.870 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-09-10T10:32:23.870 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.870 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.870 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.870 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.894 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:32:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:32:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-10T10:32:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:32:23.899 INFO:teuthology.run_tasks:Running task install... 2024-09-10T10:32:23.908 DEBUG:teuthology.task.install:project ceph 2024-09-10T10:32:23.908 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T10:32:23.908 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T10:32:23.908 INFO:teuthology.task.install:Using flavor: default 2024-09-10T10:32:23.914 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T10:32:23.914 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T10:32:23.914 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:32:23.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:32:23.917 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:32:23.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:32:23.918 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:32:23.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:32:24.072 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:32:24.072 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:32:24.084 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:32:24.085 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:32:24.093 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:32:24.093 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:32:24.186 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:32:24.187 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:32:24.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:32:24.199 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:32:24.199 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:32:24.199 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:32:24.208 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:32:24.208 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:32:24.209 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:32:24.214 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:32:24.214 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:32:24.225 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:32:24.225 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:32:24.236 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:32:24.236 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:32:24.280 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:32:24.291 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:32:24.302 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:32:24.356 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:32:24.366 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:32:24.379 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:32:24.420 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-09-10T10:32:24.422 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-09-10T10:32:24.430 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-09-10T10:32:24.432 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-09-10T10:32:24.443 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-09-10T10:32:24.445 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-09-10T10:32:24.646 INFO:teuthology.orchestra.run.smithi143.stdout:45 files removed 2024-09-10T10:32:24.673 INFO:teuthology.orchestra.run.smithi144.stdout:45 files removed 2024-09-10T10:32:24.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:32:24.688 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-09-10T10:32:24.700 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:32:24.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:32:25.258 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 325 kB/s | 84 kB 00:00 2024-09-10T10:32:25.267 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 297 kB/s | 84 kB 00:00 2024-09-10T10:32:25.277 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 264 kB/s | 84 kB 00:00 2024-09-10T10:32:25.525 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 53 kB/s | 12 kB 00:00 2024-09-10T10:32:25.551 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T10:32:25.568 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-10T10:32:25.809 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-10T10:32:25.826 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T10:32:25.844 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T10:32:26.285 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-10T10:32:26.294 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.2 MB 00:00 2024-09-10T10:32:27.026 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.2 MB 00:01 2024-09-10T10:32:29.420 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-10T10:32:30.711 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 20 MB 00:03 2024-09-10T10:32:32.468 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 20 MB 00:04 2024-09-10T10:32:35.012 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-10T10:32:36.312 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-10T10:32:36.396 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 111 kB/s | 19 kB 00:00 2024-09-10T10:32:37.085 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.5 MB 00:00 2024-09-10T10:32:37.276 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T10:32:37.762 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 19 kB 00:00 2024-09-10T10:32:38.836 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-10T10:32:39.061 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 19 kB 00:00 2024-09-10T10:32:40.123 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-10T10:32:42.731 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-10T10:32:44.317 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T10:32:45.078 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:32:45.079 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:32:45.091 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:32:45.092 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:32:45.093 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:32:45.094 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:32:45.097 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:32:45.099 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:32:45.100 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:32:45.162 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T10:32:45.186 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-09-10T10:32:45.186 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T10:32:45.186 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-09-10T10:32:45.186 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T10:32:45.186 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:32:45.187 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:32:45.188 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-09-10T10:32:45.189 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:32:45.190 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:32:45.191 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:32:45.192 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:32:45.193 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:32:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:32:45.195 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:32:45.196 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:32:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:32:45.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:32:45.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:32:45.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:32:45.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:32:45.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:32:45.198 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:32:45.199 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout:====================================================================================== 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout:Install 148 Packages 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 8 Packages 2024-09-10T10:32:45.200 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:32:45.201 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 226 M 2024-09-10T10:32:45.201 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T10:32:45.446 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T10:32:45.916 INFO:teuthology.orchestra.run.smithi144.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-10T10:32:46.166 INFO:teuthology.orchestra.run.smithi144.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 3.5 MB/s | 897 kB 00:00 2024-09-10T10:32:46.266 INFO:teuthology.orchestra.run.smithi144.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-10T10:32:46.291 INFO:teuthology.orchestra.run.smithi144.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.1 MB/s | 145 kB 00:00 2024-09-10T10:32:46.384 INFO:teuthology.orchestra.run.smithi144.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 20 MB/s | 2.4 MB 00:00 2024-09-10T10:32:46.450 INFO:teuthology.orchestra.run.smithi144.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-10T10:32:46.668 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:32:46.668 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:32:46.676 INFO:teuthology.orchestra.run.smithi144.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-10T10:32:46.679 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:32:46.680 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:32:46.681 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:32:46.682 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:32:46.683 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:32:46.685 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:32:46.686 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:32:46.748 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-09-10T10:32:46.769 INFO:teuthology.orchestra.run.smithi143.stdout:====================================================================================== 2024-09-10T10:32:46.769 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-09-10T10:32:46.769 INFO:teuthology.orchestra.run.smithi143.stdout:====================================================================================== 2024-09-10T10:32:46.769 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-09-10T10:32:46.769 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:32:46.770 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:32:46.771 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:32:46.772 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:32:46.773 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:32:46.774 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:32:46.775 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:32:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:32:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:32:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:32:46.779 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:32:46.780 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:32:46.781 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout:====================================================================================== 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout:Install 148 Packages 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 8 Packages 2024-09-10T10:32:46.782 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:32:46.783 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 226 M 2024-09-10T10:32:46.783 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-09-10T10:32:46.943 INFO:teuthology.orchestra.run.smithi144.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 18 MB/s | 21 MB 00:01 2024-09-10T10:32:46.993 INFO:teuthology.orchestra.run.smithi144.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 505 kB/s | 25 kB 00:00 2024-09-10T10:32:47.293 INFO:teuthology.orchestra.run.smithi144.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 17 MB/s | 11 MB 00:00 2024-09-10T10:32:47.352 INFO:teuthology.orchestra.run.smithi144.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.8 MB/s | 228 kB 00:00 2024-09-10T10:32:47.419 INFO:teuthology.orchestra.run.smithi144.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 405 kB/s | 27 kB 00:00 2024-09-10T10:32:47.486 INFO:teuthology.orchestra.run.smithi144.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 477 kB/s | 32 kB 00:00 2024-09-10T10:32:47.593 INFO:teuthology.orchestra.run.smithi143.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2024-09-10T10:32:47.595 INFO:teuthology.orchestra.run.smithi144.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-10T10:32:47.653 INFO:teuthology.orchestra.run.smithi144.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.8 MB/s | 164 kB 00:00 2024-09-10T10:32:47.729 INFO:teuthology.orchestra.run.smithi144.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.7 MB/s | 127 kB 00:00 2024-09-10T10:32:47.816 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:32:47.817 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:32:47.830 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:32:47.831 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:32:47.832 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:32:47.833 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:32:47.835 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:32:47.837 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:32:47.838 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:32:47.902 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:32:47.925 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:32:47.926 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:32:47.927 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:32:47.928 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:32:47.929 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:32:47.930 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:32:47.931 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:32:47.932 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:32:47.933 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:32:47.934 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:32:47.935 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:32:47.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:32:47.937 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout:====================================================================================== 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout:Install 148 Packages 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 8 Packages 2024-09-10T10:32:47.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:32:47.939 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 226 M 2024-09-10T10:32:47.939 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T10:32:47.945 INFO:teuthology.orchestra.run.smithi144.stdout:(17/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 17 MB 00:01 2024-09-10T10:32:47.987 INFO:teuthology.orchestra.run.smithi144.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 1.9 MB/s | 506 kB 00:00 2024-09-10T10:32:48.018 INFO:teuthology.orchestra.run.smithi143.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-10T10:32:48.037 INFO:teuthology.orchestra.run.smithi144.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 907 kB/s | 45 kB 00:00 2024-09-10T10:32:48.121 INFO:teuthology.orchestra.run.smithi144.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.6 MB/s | 136 kB 00:00 2024-09-10T10:32:48.152 INFO:teuthology.orchestra.run.smithi143.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.1 MB/s | 145 kB 00:00 2024-09-10T10:32:48.205 INFO:teuthology.orchestra.run.smithi144.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.9 MB/s | 161 kB 00:00 2024-09-10T10:32:48.305 INFO:teuthology.orchestra.run.smithi144.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 3.1 MB/s | 320 kB 00:00 2024-09-10T10:32:48.398 INFO:teuthology.orchestra.run.smithi144.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.2 MB/s | 301 kB 00:00 2024-09-10T10:32:48.482 INFO:teuthology.orchestra.run.smithi144.stdout:(24/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 10 MB/s | 5.3 MB 00:00 2024-09-10T10:32:48.506 INFO:teuthology.orchestra.run.smithi144.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 915 kB/s | 99 kB 00:00 2024-09-10T10:32:48.522 INFO:teuthology.orchestra.run.smithi178.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-09-10T10:32:48.540 INFO:teuthology.orchestra.run.smithi144.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T10:32:48.632 INFO:teuthology.orchestra.run.smithi144.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-10T10:32:48.716 INFO:teuthology.orchestra.run.smithi144.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 383 kB/s | 32 kB 00:00 2024-09-10T10:32:48.877 INFO:teuthology.orchestra.run.smithi143.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 3.6 MB/s | 5.4 MB 00:01 2024-09-10T10:32:48.947 INFO:teuthology.orchestra.run.smithi178.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-10T10:32:48.961 INFO:teuthology.orchestra.run.smithi143.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 3.0 MB/s | 2.4 MB 00:00 2024-09-10T10:32:49.039 INFO:teuthology.orchestra.run.smithi178.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.5 MB/s | 145 kB 00:00 2024-09-10T10:32:49.182 INFO:teuthology.orchestra.run.smithi144.stdout:(29/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 22 MB/s | 49 MB 00:02 2024-09-10T10:32:49.236 INFO:teuthology.orchestra.run.smithi143.stdout:(6/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 21 MB 00:01 2024-09-10T10:32:49.241 INFO:teuthology.orchestra.run.smithi144.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 4.2 MB/s | 3.1 MB 00:00 2024-09-10T10:32:49.266 INFO:teuthology.orchestra.run.smithi144.stdout:(31/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 266 kB/s | 146 kB 00:00 2024-09-10T10:32:49.278 INFO:teuthology.orchestra.run.smithi143.stdout:(7/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 3.8 MB/s | 1.5 MB 00:00 2024-09-10T10:32:49.408 INFO:teuthology.orchestra.run.smithi144.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.7 MB/s | 249 kB 00:00 2024-09-10T10:32:49.458 INFO:teuthology.orchestra.run.smithi144.stdout:(33/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 13 MB/s | 3.6 MB 00:00 2024-09-10T10:32:49.492 INFO:teuthology.orchestra.run.smithi144.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 590 kB/s | 49 kB 00:00 2024-09-10T10:32:49.551 INFO:teuthology.orchestra.run.smithi144.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 179 kB/s | 16 kB 00:00 2024-09-10T10:32:49.584 INFO:teuthology.orchestra.run.smithi144.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.0 MB/s | 285 kB 00:00 2024-09-10T10:32:49.597 INFO:teuthology.orchestra.run.smithi178.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 4.3 MB/s | 2.4 MB 00:00 2024-09-10T10:32:49.651 INFO:teuthology.orchestra.run.smithi144.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 7.5 MB/s | 766 kB 00:00 2024-09-10T10:32:49.685 INFO:teuthology.orchestra.run.smithi144.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 182 kB/s | 18 kB 00:00 2024-09-10T10:32:49.785 INFO:teuthology.orchestra.run.smithi144.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-10T10:32:49.877 INFO:teuthology.orchestra.run.smithi144.stdout:(40/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-10T10:32:49.919 INFO:teuthology.orchestra.run.smithi144.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-10T10:32:49.953 INFO:teuthology.orchestra.run.smithi144.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-10T10:32:49.973 INFO:teuthology.orchestra.run.smithi178.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-09-10T10:32:49.995 INFO:teuthology.orchestra.run.smithi144.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T10:32:50.045 INFO:teuthology.orchestra.run.smithi144.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-10T10:32:50.070 INFO:teuthology.orchestra.run.smithi144.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-10T10:32:50.096 INFO:teuthology.orchestra.run.smithi144.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-10T10:32:50.121 INFO:teuthology.orchestra.run.smithi144.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-10T10:32:50.146 INFO:teuthology.orchestra.run.smithi144.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T10:32:50.153 INFO:teuthology.orchestra.run.smithi143.stdout:(8/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86_ 18 MB/s | 17 MB 00:00 2024-09-10T10:32:50.212 INFO:teuthology.orchestra.run.smithi143.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 433 kB/s | 25 kB 00:00 2024-09-10T10:32:50.255 INFO:teuthology.orchestra.run.smithi144.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 987 kB/s | 107 kB 00:00 2024-09-10T10:32:50.297 INFO:teuthology.orchestra.run.smithi144.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 524 kB/s | 22 kB 00:00 2024-09-10T10:32:50.322 INFO:teuthology.orchestra.run.smithi144.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-10T10:32:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-10T10:32:50.381 INFO:teuthology.orchestra.run.smithi144.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-10T10:32:50.400 INFO:teuthology.orchestra.run.smithi178.stdout:(6/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-10T10:32:50.406 INFO:teuthology.orchestra.run.smithi144.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-10T10:32:50.423 INFO:teuthology.orchestra.run.smithi144.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 433 kB/s | 18 kB 00:00 2024-09-10T10:32:50.448 INFO:teuthology.orchestra.run.smithi144.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-10T10:32:50.482 INFO:teuthology.orchestra.run.smithi144.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T10:32:50.499 INFO:teuthology.orchestra.run.smithi144.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-10T10:32:50.566 INFO:teuthology.orchestra.run.smithi144.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-10T10:32:50.598 INFO:teuthology.orchestra.run.smithi178.stdout:(7/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 9.1 MB/s | 21 MB 00:02 2024-09-10T10:32:50.599 INFO:teuthology.orchestra.run.smithi144.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 738 kB/s | 24 kB 00:00 2024-09-10T10:32:50.641 INFO:teuthology.orchestra.run.smithi144.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T10:32:50.653 INFO:teuthology.orchestra.run.smithi143.stdout:(10/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-09-10T10:32:50.692 INFO:teuthology.orchestra.run.smithi144.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T10:32:50.780 INFO:teuthology.orchestra.run.smithi144.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 2.0 MB/s | 581 kB 00:00 2024-09-10T10:32:50.801 INFO:teuthology.orchestra.run.smithi144.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T10:32:50.812 INFO:teuthology.orchestra.run.smithi143.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.4 MB/s | 228 kB 00:00 2024-09-10T10:32:50.826 INFO:teuthology.orchestra.run.smithi144.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 846 kB/s | 38 kB 00:00 2024-09-10T10:32:50.912 INFO:teuthology.orchestra.run.smithi143.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 270 kB/s | 27 kB 00:00 2024-09-10T10:32:51.013 INFO:teuthology.orchestra.run.smithi143.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 317 kB/s | 32 kB 00:00 2024-09-10T10:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout:(8/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 17 MB/s | 11 MB 00:00 2024-09-10T10:32:51.290 INFO:teuthology.orchestra.run.smithi178.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 336 kB/s | 25 kB 00:00 2024-09-10T10:32:51.318 INFO:teuthology.orchestra.run.smithi144.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-09-10T10:32:51.338 INFO:teuthology.orchestra.run.smithi143.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 2.3 MB/s | 751 kB 00:00 2024-09-10T10:32:51.391 INFO:teuthology.orchestra.run.smithi178.stdout:(10/156): ceph-base-19.1.1-305.g1cf819b7.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-09-10T10:32:51.513 INFO:teuthology.orchestra.run.smithi143.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 936 kB/s | 164 kB 00:00 2024-09-10T10:32:51.622 INFO:teuthology.orchestra.run.smithi143.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.1 MB/s | 127 kB 00:00 2024-09-10T10:32:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 857 kB/s | 228 kB 00:00 2024-09-10T10:32:51.751 INFO:teuthology.orchestra.run.smithi144.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-10T10:32:51.806 INFO:teuthology.orchestra.run.smithi143.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 2.7 MB/s | 506 kB 00:00 2024-09-10T10:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout:(12/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 12 MB/s | 17 MB 00:01 2024-09-10T10:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 141 kB/s | 27 kB 00:00 2024-09-10T10:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 634 kB/s | 32 kB 00:00 2024-09-10T10:32:51.925 INFO:teuthology.orchestra.run.smithi178.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 3.2 MB/s | 164 kB 00:00 2024-09-10T10:32:52.010 INFO:teuthology.orchestra.run.smithi144.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-09-10T10:32:52.017 INFO:teuthology.orchestra.run.smithi178.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.3 MB/s | 127 kB 00:00 2024-09-10T10:32:52.110 INFO:teuthology.orchestra.run.smithi178.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 5.4 MB/s | 506 kB 00:00 2024-09-10T10:32:52.127 INFO:teuthology.orchestra.run.smithi144.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.7 MB/s | 4.9 MB 00:01 2024-09-10T10:32:52.152 INFO:teuthology.orchestra.run.smithi144.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 967 kB/s | 137 kB 00:00 2024-09-10T10:32:52.228 INFO:teuthology.orchestra.run.smithi144.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-09-10T10:32:52.287 INFO:teuthology.orchestra.run.smithi144.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T10:32:52.426 INFO:teuthology.orchestra.run.smithi178.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-10T10:32:52.460 INFO:teuthology.orchestra.run.smithi178.stdout:(19/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 1.2 MB/s | 751 kB 00:00 2024-09-10T10:32:52.485 INFO:teuthology.orchestra.run.smithi178.stdout:(20/156): python3-ceph-argparse-19.1.1-305.g1cf 770 kB/s | 45 kB 00:00 2024-09-10T10:32:52.503 INFO:teuthology.orchestra.run.smithi144.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-10T10:32:52.535 INFO:teuthology.orchestra.run.smithi178.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.8 MB/s | 136 kB 00:00 2024-09-10T10:32:52.545 INFO:teuthology.orchestra.run.smithi144.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T10:32:52.560 INFO:teuthology.orchestra.run.smithi178.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T10:32:52.619 INFO:teuthology.orchestra.run.smithi178.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T10:32:52.629 INFO:teuthology.orchestra.run.smithi144.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-10T10:32:52.664 INFO:teuthology.orchestra.run.smithi143.stdout:(18/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 20 MB/s | 49 MB 00:02 2024-09-10T10:32:52.686 INFO:teuthology.orchestra.run.smithi178.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.5 MB/s | 99 kB 00:00 2024-09-10T10:32:52.720 INFO:teuthology.orchestra.run.smithi178.stdout:(25/156): python3-rados-19.1.1-305.g1cf819b7.el 1.7 MB/s | 320 kB 00:00 2024-09-10T10:32:52.729 INFO:teuthology.orchestra.run.smithi144.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-10T10:32:52.745 INFO:teuthology.orchestra.run.smithi178.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T10:32:52.771 INFO:teuthology.orchestra.run.smithi144.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-10T10:32:52.781 INFO:teuthology.orchestra.run.smithi143.stdout:(19/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 3.0 MB/s | 11 MB 00:03 2024-09-10T10:32:52.804 INFO:teuthology.orchestra.run.smithi178.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-10T10:32:52.806 INFO:teuthology.orchestra.run.smithi143.stdout:(20/156): python3-ceph-argparse-19.1.1-305.g1cf 318 kB/s | 45 kB 00:00 2024-09-10T10:32:52.813 INFO:teuthology.orchestra.run.smithi144.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 462 kB/s | 19 kB 00:00 2024-09-10T10:32:52.832 INFO:teuthology.orchestra.run.smithi143.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T10:32:52.862 INFO:teuthology.orchestra.run.smithi178.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 547 kB/s | 32 kB 00:00 2024-09-10T10:32:52.882 INFO:teuthology.orchestra.run.smithi143.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T10:32:52.888 INFO:teuthology.orchestra.run.smithi144.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T10:32:52.921 INFO:teuthology.orchestra.run.smithi178.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.4 MB/s | 146 kB 00:00 2024-09-10T10:32:52.930 INFO:teuthology.orchestra.run.smithi144.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-09-10T10:32:52.949 INFO:teuthology.orchestra.run.smithi143.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 4.4 MB/s | 301 kB 00:00 2024-09-10T10:32:52.982 INFO:teuthology.orchestra.run.smithi143.stdout:(24/156): python3-rados-19.1.1-305.g1cf819b7.el 2.1 MB/s | 320 kB 00:00 2024-09-10T10:32:53.033 INFO:teuthology.orchestra.run.smithi143.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.2 MB/s | 99 kB 00:00 2024-09-10T10:32:53.067 INFO:teuthology.orchestra.run.smithi143.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-10T10:32:53.138 INFO:teuthology.orchestra.run.smithi178.stdout:(30/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 16 MB/s | 3.6 MB 00:00 2024-09-10T10:32:53.158 INFO:teuthology.orchestra.run.smithi143.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-10T10:32:53.251 INFO:teuthology.orchestra.run.smithi143.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-10T10:32:53.276 INFO:teuthology.orchestra.run.smithi143.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 273 kB/s | 32 kB 00:00 2024-09-10T10:32:53.301 INFO:teuthology.orchestra.run.smithi143.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T10:32:53.418 INFO:teuthology.orchestra.run.smithi143.stdout:(31/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 5.3 MB 00:01 2024-09-10T10:32:53.502 INFO:teuthology.orchestra.run.smithi143.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.9 MB/s | 249 kB 00:00 2024-09-10T10:32:53.581 INFO:teuthology.orchestra.run.smithi178.stdout:(31/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 21 MB/s | 49 MB 00:02 2024-09-10T10:32:53.903 INFO:teuthology.orchestra.run.smithi144.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.8 MB/s | 5.8 MB 00:01 2024-09-10T10:32:53.903 INFO:teuthology.orchestra.run.smithi144.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-10T10:32:53.903 INFO:teuthology.orchestra.run.smithi144.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 23 MB/s | 7.4 MB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 328 kB/s | 49 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 330 kB/s | 16 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:(37/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 2.4 MB/s | 285 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 435 kB/s | 18 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi178.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 4.9 MB/s | 249 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi178.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 742 kB/s | 49 kB 00:00 2024-09-10T10:32:53.904 INFO:teuthology.orchestra.run.smithi178.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-10T10:32:53.905 INFO:teuthology.orchestra.run.smithi178.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 141 kB/s | 16 kB 00:00 2024-09-10T10:32:53.905 INFO:teuthology.orchestra.run.smithi178.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T10:32:53.905 INFO:teuthology.orchestra.run.smithi178.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 9.9 MB/s | 766 kB 00:00 2024-09-10T10:32:53.914 INFO:teuthology.orchestra.run.smithi144.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T10:32:53.916 INFO:teuthology.orchestra.run.smithi178.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 243 kB/s | 18 kB 00:00 2024-09-10T10:32:53.957 INFO:teuthology.orchestra.run.smithi144.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T10:32:53.962 INFO:teuthology.orchestra.run.smithi143.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 416 kB/s | 80 kB 00:00 2024-09-10T10:32:54.021 INFO:teuthology.orchestra.run.smithi143.stdout:(40/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 4.8 MB/s | 3.6 MB 00:00 2024-09-10T10:32:54.046 INFO:teuthology.orchestra.run.smithi143.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-10T10:32:54.049 INFO:teuthology.orchestra.run.smithi178.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-09-10T10:32:54.082 INFO:teuthology.orchestra.run.smithi144.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-09-10T10:32:54.124 INFO:teuthology.orchestra.run.smithi178.stdout:(40/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 2.2 MB/s | 3.1 MB 00:01 2024-09-10T10:32:54.132 INFO:teuthology.orchestra.run.smithi144.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-09-10T10:32:54.147 INFO:teuthology.orchestra.run.smithi143.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-10T10:32:54.166 INFO:teuthology.orchestra.run.smithi178.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-10T10:32:54.191 INFO:teuthology.orchestra.run.smithi144.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-10T10:32:54.197 INFO:teuthology.orchestra.run.smithi143.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-10T10:32:54.222 INFO:teuthology.orchestra.run.smithi143.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-10T10:32:54.233 INFO:teuthology.orchestra.run.smithi144.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T10:32:54.233 INFO:teuthology.orchestra.run.smithi178.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-10T10:32:54.256 INFO:teuthology.orchestra.run.smithi143.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-10T10:32:54.275 INFO:teuthology.orchestra.run.smithi144.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-09-10T10:32:54.281 INFO:teuthology.orchestra.run.smithi143.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T10:32:54.292 INFO:teuthology.orchestra.run.smithi144.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-09-10T10:32:54.300 INFO:teuthology.orchestra.run.smithi178.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-10T10:32:54.315 INFO:teuthology.orchestra.run.smithi143.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-10T10:32:54.325 INFO:teuthology.orchestra.run.smithi144.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-10T10:32:54.325 INFO:teuthology.orchestra.run.smithi178.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-10T10:32:54.340 INFO:teuthology.orchestra.run.smithi143.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T10:32:54.351 INFO:teuthology.orchestra.run.smithi178.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-10T10:32:54.359 INFO:teuthology.orchestra.run.smithi144.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-10T10:32:54.374 INFO:teuthology.orchestra.run.smithi143.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-09-10T10:32:54.376 INFO:teuthology.orchestra.run.smithi178.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 135 kB 00:00 2024-09-10T10:32:54.399 INFO:teuthology.orchestra.run.smithi143.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T10:32:54.401 INFO:teuthology.orchestra.run.smithi178.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-10T10:32:54.424 INFO:teuthology.orchestra.run.smithi143.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-10T10:32:54.427 INFO:teuthology.orchestra.run.smithi178.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T10:32:54.442 INFO:teuthology.orchestra.run.smithi144.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T10:32:54.458 INFO:teuthology.orchestra.run.smithi143.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-10T10:32:54.476 INFO:teuthology.orchestra.run.smithi144.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-09-10T10:32:54.483 INFO:teuthology.orchestra.run.smithi143.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-10T10:32:54.533 INFO:teuthology.orchestra.run.smithi143.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-10T10:32:54.543 INFO:teuthology.orchestra.run.smithi144.stdout:(96/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-10T10:32:54.560 INFO:teuthology.orchestra.run.smithi144.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-10T10:32:54.584 INFO:teuthology.orchestra.run.smithi143.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T10:32:54.593 INFO:teuthology.orchestra.run.smithi144.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-09-10T10:32:54.594 INFO:teuthology.orchestra.run.smithi178.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-10T10:32:54.636 INFO:teuthology.orchestra.run.smithi178.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-10T10:32:54.642 INFO:teuthology.orchestra.run.smithi143.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T10:32:54.661 INFO:teuthology.orchestra.run.smithi144.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-10T10:32:54.686 INFO:teuthology.orchestra.run.smithi144.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.0 MB/s | 96 kB 00:00 2024-09-10T10:32:54.711 INFO:teuthology.orchestra.run.smithi178.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-09-10T10:32:54.714 INFO:teuthology.orchestra.run.smithi143.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 633 kB/s | 45 kB 00:00 2024-09-10T10:32:54.753 INFO:teuthology.orchestra.run.smithi144.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-09-10T10:32:54.777 INFO:teuthology.orchestra.run.smithi143.stdout:(58/156): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-09-10T10:32:54.778 INFO:teuthology.orchestra.run.smithi144.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-09-10T10:32:54.810 INFO:teuthology.orchestra.run.smithi143.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.8 MB/s | 662 kB 00:00 2024-09-10T10:32:54.819 INFO:teuthology.orchestra.run.smithi178.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-09-10T10:32:54.835 INFO:teuthology.orchestra.run.smithi143.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-10T10:32:54.836 INFO:teuthology.orchestra.run.smithi178.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-10T10:32:54.861 INFO:teuthology.orchestra.run.smithi143.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-10T10:32:54.870 INFO:teuthology.orchestra.run.smithi178.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-09-10T10:32:54.886 INFO:teuthology.orchestra.run.smithi143.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T10:32:54.887 INFO:teuthology.orchestra.run.smithi178.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-10T10:32:54.921 INFO:teuthology.orchestra.run.smithi178.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T10:32:54.944 INFO:teuthology.orchestra.run.smithi143.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-10T10:32:54.970 INFO:teuthology.orchestra.run.smithi143.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T10:32:54.971 INFO:teuthology.orchestra.run.smithi178.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-10T10:32:54.995 INFO:teuthology.orchestra.run.smithi143.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-10T10:32:54.996 INFO:teuthology.orchestra.run.smithi178.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-10T10:32:55.045 INFO:teuthology.orchestra.run.smithi144.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-10T10:32:55.104 INFO:teuthology.orchestra.run.smithi144.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T10:32:55.137 INFO:teuthology.orchestra.run.smithi144.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T10:32:55.162 INFO:teuthology.orchestra.run.smithi143.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-10T10:32:55.163 INFO:teuthology.orchestra.run.smithi144.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-10T10:32:55.188 INFO:teuthology.orchestra.run.smithi178.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.0 MB/s | 581 kB 00:00 2024-09-10T10:32:55.222 INFO:teuthology.orchestra.run.smithi178.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-10T10:32:55.255 INFO:teuthology.orchestra.run.smithi178.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-10T10:32:55.272 INFO:teuthology.orchestra.run.smithi144.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T10:32:55.280 INFO:teuthology.orchestra.run.smithi178.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-09-10T10:32:55.297 INFO:teuthology.orchestra.run.smithi144.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T10:32:55.312 INFO:teuthology.orchestra.run.smithi143.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-10T10:32:55.322 INFO:teuthology.orchestra.run.smithi144.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T10:32:55.355 INFO:teuthology.orchestra.run.smithi144.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-10T10:32:55.364 INFO:teuthology.orchestra.run.smithi178.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T10:32:55.381 INFO:teuthology.orchestra.run.smithi144.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T10:32:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-10T10:32:55.415 INFO:teuthology.orchestra.run.smithi178.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 763 kB/s | 38 kB 00:00 2024-09-10T10:32:55.423 INFO:teuthology.orchestra.run.smithi144.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T10:32:55.455 INFO:teuthology.orchestra.run.smithi143.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-09-10T10:32:55.468 INFO:teuthology.orchestra.run.smithi144.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-10T10:32:55.488 INFO:teuthology.orchestra.run.smithi143.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-10T10:32:55.498 INFO:teuthology.orchestra.run.smithi144.stdout:(114/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-09-10T10:32:55.514 INFO:teuthology.orchestra.run.smithi143.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T10:32:55.524 INFO:teuthology.orchestra.run.smithi144.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-09-10T10:32:55.557 INFO:teuthology.orchestra.run.smithi144.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 537 kB/s | 14 kB 00:00 2024-09-10T10:32:55.583 INFO:teuthology.orchestra.run.smithi144.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-09-10T10:32:55.597 INFO:teuthology.orchestra.run.smithi143.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-10T10:32:55.608 INFO:teuthology.orchestra.run.smithi144.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.5 MB/s | 172 kB 00:00 2024-09-10T10:32:55.642 INFO:teuthology.orchestra.run.smithi144.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-10T10:32:55.667 INFO:teuthology.orchestra.run.smithi144.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-10T10:32:55.681 INFO:teuthology.orchestra.run.smithi143.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-09-10T10:32:55.692 INFO:teuthology.orchestra.run.smithi144.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T10:32:55.706 INFO:teuthology.orchestra.run.smithi143.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 805 kB/s | 87 kB 00:00 2024-09-10T10:32:55.718 INFO:teuthology.orchestra.run.smithi144.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-10T10:32:55.743 INFO:teuthology.orchestra.run.smithi144.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T10:32:55.760 INFO:teuthology.orchestra.run.smithi144.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T10:32:55.777 INFO:teuthology.orchestra.run.smithi144.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-10T10:32:55.794 INFO:teuthology.orchestra.run.smithi144.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T10:32:55.807 INFO:teuthology.orchestra.run.smithi143.stdout:(74/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-09-10T10:32:55.811 INFO:teuthology.orchestra.run.smithi144.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-10T10:32:55.815 INFO:teuthology.orchestra.run.smithi178.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:00 2024-09-10T10:32:55.832 INFO:teuthology.orchestra.run.smithi143.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 488 kB/s | 61 kB 00:00 2024-09-10T10:32:55.870 INFO:teuthology.orchestra.run.smithi144.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-09-10T10:32:55.895 INFO:teuthology.orchestra.run.smithi144.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T10:32:55.916 INFO:teuthology.orchestra.run.smithi143.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-09-10T10:32:55.941 INFO:teuthology.orchestra.run.smithi143.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T10:32:55.945 INFO:teuthology.orchestra.run.smithi144.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-09-10T10:32:55.966 INFO:teuthology.orchestra.run.smithi143.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-10T10:32:55.970 INFO:teuthology.orchestra.run.smithi144.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-10T10:32:55.991 INFO:teuthology.orchestra.run.smithi143.stdout:(79/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-10T10:32:55.996 INFO:teuthology.orchestra.run.smithi144.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T10:32:56.016 INFO:teuthology.orchestra.run.smithi143.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 253 kB/s | 19 kB 00:00 2024-09-10T10:32:56.021 INFO:teuthology.orchestra.run.smithi144.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-10T10:32:56.042 INFO:teuthology.orchestra.run.smithi143.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T10:32:56.055 INFO:teuthology.orchestra.run.smithi144.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.7 MB/s | 90 kB 00:00 2024-09-10T10:32:56.067 INFO:teuthology.orchestra.run.smithi143.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-09-10T10:32:56.080 INFO:teuthology.orchestra.run.smithi144.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-10T10:32:56.101 INFO:teuthology.orchestra.run.smithi143.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-10T10:32:56.105 INFO:teuthology.orchestra.run.smithi144.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T10:32:56.131 INFO:teuthology.orchestra.run.smithi144.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T10:32:56.156 INFO:teuthology.orchestra.run.smithi144.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T10:32:56.176 INFO:teuthology.orchestra.run.smithi143.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-10T10:32:56.189 INFO:teuthology.orchestra.run.smithi144.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-10T10:32:56.201 INFO:teuthology.orchestra.run.smithi143.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 769 kB/s | 77 kB 00:00 2024-09-10T10:32:56.215 INFO:teuthology.orchestra.run.smithi144.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-10T10:32:56.227 INFO:teuthology.orchestra.run.smithi143.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-10T10:32:56.240 INFO:teuthology.orchestra.run.smithi144.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T10:32:56.252 INFO:teuthology.orchestra.run.smithi143.stdout:(87/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-09-10T10:32:56.265 INFO:teuthology.orchestra.run.smithi144.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-10T10:32:56.285 INFO:teuthology.orchestra.run.smithi143.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-10T10:32:56.292 INFO:teuthology.orchestra.run.smithi178.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-09-10T10:32:56.292 INFO:teuthology.orchestra.run.smithi144.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T10:32:56.316 INFO:teuthology.orchestra.run.smithi144.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-09-10T10:32:56.319 INFO:teuthology.orchestra.run.smithi143.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-10T10:32:56.341 INFO:teuthology.orchestra.run.smithi144.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-10T10:32:56.344 INFO:teuthology.orchestra.run.smithi143.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-10T10:32:56.366 INFO:teuthology.orchestra.run.smithi144.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T10:32:56.370 INFO:teuthology.orchestra.run.smithi143.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T10:32:56.382 INFO:teuthology.orchestra.run.smithi178.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-09-10T10:32:56.434 INFO:teuthology.orchestra.run.smithi144.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T10:32:56.437 INFO:teuthology.orchestra.run.smithi143.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-09-10T10:32:56.440 INFO:teuthology.orchestra.run.smithi178.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T10:32:56.462 INFO:teuthology.orchestra.run.smithi143.stdout:(93/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-10T10:32:56.496 INFO:teuthology.orchestra.run.smithi143.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-10T10:32:56.546 INFO:teuthology.orchestra.run.smithi143.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-10T10:32:56.605 INFO:teuthology.orchestra.run.smithi143.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-09-10T10:32:56.647 INFO:teuthology.orchestra.run.smithi143.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-09-10T10:32:56.666 INFO:teuthology.orchestra.run.smithi178.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 2.4 MB/s | 910 kB 00:00 2024-09-10T10:32:56.672 INFO:teuthology.orchestra.run.smithi143.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-10T10:32:56.675 INFO:teuthology.orchestra.run.smithi144.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-10T10:32:56.734 INFO:teuthology.orchestra.run.smithi144.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T10:32:56.758 INFO:teuthology.orchestra.run.smithi178.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-10T10:32:56.808 INFO:teuthology.orchestra.run.smithi178.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-10T10:32:56.839 INFO:teuthology.orchestra.run.smithi143.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-10T10:32:56.864 INFO:teuthology.orchestra.run.smithi143.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 58 kB 00:00 2024-09-10T10:32:56.868 INFO:teuthology.orchestra.run.smithi144.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-10T10:32:56.890 INFO:teuthology.orchestra.run.smithi143.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 440 kB/s | 96 kB 00:00 2024-09-10T10:32:56.910 INFO:teuthology.orchestra.run.smithi144.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 983 kB/s | 41 kB 00:00 2024-09-10T10:32:56.915 INFO:teuthology.orchestra.run.smithi143.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-09-10T10:32:56.940 INFO:teuthology.orchestra.run.smithi143.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 592 kB/s | 45 kB 00:00 2024-09-10T10:32:56.952 INFO:teuthology.orchestra.run.smithi144.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 8.3 MB/s | 1.8 MB 00:00 2024-09-10T10:32:56.985 INFO:teuthology.orchestra.run.smithi144.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-10T10:32:56.999 INFO:teuthology.orchestra.run.smithi143.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T10:32:57.024 INFO:teuthology.orchestra.run.smithi143.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T10:32:57.036 INFO:teuthology.orchestra.run.smithi144.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-10T10:32:57.041 INFO:teuthology.orchestra.run.smithi143.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-10T10:32:57.042 INFO:teuthology.orchestra.run.smithi178.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-09-10T10:32:57.092 INFO:teuthology.orchestra.run.smithi178.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T10:32:57.103 INFO:teuthology.orchestra.run.smithi144.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-10T10:32:57.150 INFO:teuthology.orchestra.run.smithi143.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T10:32:57.166 INFO:teuthology.orchestra.run.smithi143.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-10T10:32:57.183 INFO:teuthology.orchestra.run.smithi143.stdout:(109/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-10T10:32:57.200 INFO:teuthology.orchestra.run.smithi143.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-10T10:32:57.209 INFO:teuthology.orchestra.run.smithi178.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-10T10:32:57.234 INFO:teuthology.orchestra.run.smithi143.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.6 MB/s | 151 kB 00:00 2024-09-10T10:32:57.268 INFO:teuthology.orchestra.run.smithi143.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 803 kB/s | 67 kB 00:00 2024-09-10T10:32:57.301 INFO:teuthology.orchestra.run.smithi143.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-10T10:32:57.327 INFO:teuthology.orchestra.run.smithi143.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T10:32:57.352 INFO:teuthology.orchestra.run.smithi143.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T10:32:57.359 INFO:teuthology.orchestra.run.smithi178.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 6.3 MB/s | 5.8 MB 00:00 2024-09-10T10:32:57.377 INFO:teuthology.orchestra.run.smithi143.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-10T10:32:57.385 INFO:teuthology.orchestra.run.smithi178.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-10T10:32:57.403 INFO:teuthology.orchestra.run.smithi143.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T10:32:57.410 INFO:teuthology.orchestra.run.smithi178.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-10T10:32:57.428 INFO:teuthology.orchestra.run.smithi143.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-09-10T10:32:57.435 INFO:teuthology.orchestra.run.smithi178.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T10:32:57.461 INFO:teuthology.orchestra.run.smithi178.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T10:32:57.461 INFO:teuthology.orchestra.run.smithi143.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-10T10:32:57.486 INFO:teuthology.orchestra.run.smithi178.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-10T10:32:57.486 INFO:teuthology.orchestra.run.smithi143.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-10T10:32:57.512 INFO:teuthology.orchestra.run.smithi143.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T10:32:57.538 INFO:teuthology.orchestra.run.smithi143.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T10:32:57.563 INFO:teuthology.orchestra.run.smithi143.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 422 kB/s | 11 kB 00:00 2024-09-10T10:32:57.588 INFO:teuthology.orchestra.run.smithi143.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-10T10:32:57.603 INFO:teuthology.orchestra.run.smithi178.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-10T10:32:57.622 INFO:teuthology.orchestra.run.smithi143.stdout:(125/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-09-10T10:32:57.647 INFO:teuthology.orchestra.run.smithi143.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-09-10T10:32:57.681 INFO:teuthology.orchestra.run.smithi143.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 749 kB/s | 20 kB 00:00 2024-09-10T10:32:57.714 INFO:teuthology.orchestra.run.smithi143.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-09-10T10:32:57.740 INFO:teuthology.orchestra.run.smithi143.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T10:32:57.765 INFO:teuthology.orchestra.run.smithi143.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T10:32:57.790 INFO:teuthology.orchestra.run.smithi143.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T10:32:57.795 INFO:teuthology.orchestra.run.smithi178.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-09-10T10:32:57.841 INFO:teuthology.orchestra.run.smithi143.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-09-10T10:32:57.866 INFO:teuthology.orchestra.run.smithi143.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 652 kB/s | 16 kB 00:00 2024-09-10T10:32:57.895 INFO:teuthology.orchestra.run.smithi178.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-09-10T10:32:57.899 INFO:teuthology.orchestra.run.smithi143.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-10T10:32:57.920 INFO:teuthology.orchestra.run.smithi178.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 617 kB/s | 77 kB 00:00 2024-09-10T10:32:57.925 INFO:teuthology.orchestra.run.smithi143.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T10:32:57.937 INFO:teuthology.orchestra.run.smithi178.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T10:32:57.950 INFO:teuthology.orchestra.run.smithi143.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T10:32:57.976 INFO:teuthology.orchestra.run.smithi143.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-09-10T10:32:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T10:32:58.001 INFO:teuthology.orchestra.run.smithi143.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-10T10:32:58.026 INFO:teuthology.orchestra.run.smithi143.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T10:32:58.052 INFO:teuthology.orchestra.run.smithi143.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T10:32:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.6 MB/s | 5.1 MB 00:00 2024-09-10T10:32:58.077 INFO:teuthology.orchestra.run.smithi143.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T10:32:58.102 INFO:teuthology.orchestra.run.smithi143.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-10T10:32:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-09-10T10:32:58.127 INFO:teuthology.orchestra.run.smithi143.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T10:32:58.138 INFO:teuthology.orchestra.run.smithi178.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-10T10:32:58.152 INFO:teuthology.orchestra.run.smithi143.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T10:32:58.163 INFO:teuthology.orchestra.run.smithi178.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-09-10T10:32:58.178 INFO:teuthology.orchestra.run.smithi143.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-10T10:32:58.197 INFO:teuthology.orchestra.run.smithi178.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 54 kB 00:00 2024-09-10T10:32:58.203 INFO:teuthology.orchestra.run.smithi143.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T10:32:58.214 INFO:teuthology.orchestra.run.smithi178.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-10T10:32:58.236 INFO:teuthology.orchestra.run.smithi143.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T10:32:58.239 INFO:teuthology.orchestra.run.smithi178.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T10:32:58.265 INFO:teuthology.orchestra.run.smithi178.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T10:32:58.287 INFO:teuthology.orchestra.run.smithi143.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T10:32:58.289 INFO:teuthology.orchestra.run.smithi178.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-10T10:32:58.315 INFO:teuthology.orchestra.run.smithi178.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-10T10:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T10:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-10T10:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T10:32:58.449 INFO:teuthology.orchestra.run.smithi178.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T10:32:58.454 INFO:teuthology.orchestra.run.smithi143.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 14 MB/s | 2.3 MB 00:00 2024-09-10T10:32:58.475 INFO:teuthology.orchestra.run.smithi178.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-10T10:32:58.538 INFO:teuthology.orchestra.run.smithi143.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-09-10T10:32:58.588 INFO:teuthology.orchestra.run.smithi143.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 810 kB/s | 41 kB 00:00 2024-09-10T10:32:58.647 INFO:teuthology.orchestra.run.smithi143.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-10T10:32:58.722 INFO:teuthology.orchestra.run.smithi143.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-10T10:32:58.725 INFO:teuthology.orchestra.run.smithi178.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-09-10T10:32:58.784 INFO:teuthology.orchestra.run.smithi178.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T10:32:58.809 INFO:teuthology.orchestra.run.smithi178.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T10:32:58.831 INFO:teuthology.orchestra.run.smithi143.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-10T10:32:58.834 INFO:teuthology.orchestra.run.smithi178.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-10T10:32:58.889 INFO:teuthology.orchestra.run.smithi143.stdout:(155/156): librados2-19.1.1-305.g1cf819b7.el9.x 5.0 MB/s | 3.4 MB 00:00 2024-09-10T10:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-10T10:32:58.948 INFO:teuthology.orchestra.run.smithi143.stdout:(156/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 4.4 MB/s | 3.2 MB 00:00 2024-09-10T10:32:58.953 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:32:58.953 INFO:teuthology.orchestra.run.smithi143.stdout:Total 19 MB/s | 226 MB 00:12 2024-09-10T10:32:58.960 INFO:teuthology.orchestra.run.smithi178.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T10:32:58.985 INFO:teuthology.orchestra.run.smithi178.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T10:32:59.010 INFO:teuthology.orchestra.run.smithi178.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T10:32:59.036 INFO:teuthology.orchestra.run.smithi178.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T10:32:59.078 INFO:teuthology.orchestra.run.smithi178.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T10:32:59.111 INFO:teuthology.orchestra.run.smithi178.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-10T10:32:59.137 INFO:teuthology.orchestra.run.smithi178.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T10:32:59.162 INFO:teuthology.orchestra.run.smithi178.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T10:32:59.187 INFO:teuthology.orchestra.run.smithi178.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-10T10:32:59.212 INFO:teuthology.orchestra.run.smithi178.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-10T10:32:59.246 INFO:teuthology.orchestra.run.smithi178.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-10T10:32:59.271 INFO:teuthology.orchestra.run.smithi178.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-10T10:32:59.305 INFO:teuthology.orchestra.run.smithi178.stdout:(120/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-10T10:32:59.330 INFO:teuthology.orchestra.run.smithi178.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T10:32:59.372 INFO:teuthology.orchestra.run.smithi178.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.1 MB/s | 144 kB 00:00 2024-09-10T10:32:59.397 INFO:teuthology.orchestra.run.smithi178.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 434 kB/s | 11 kB 00:00 2024-09-10T10:32:59.423 INFO:teuthology.orchestra.run.smithi178.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 703 kB/s | 18 kB 00:00 2024-09-10T10:32:59.448 INFO:teuthology.orchestra.run.smithi178.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-10T10:32:59.473 INFO:teuthology.orchestra.run.smithi178.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-09-10T10:32:59.499 INFO:teuthology.orchestra.run.smithi178.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-10T10:32:59.566 INFO:teuthology.orchestra.run.smithi178.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-10T10:32:59.591 INFO:teuthology.orchestra.run.smithi178.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T10:32:59.625 INFO:teuthology.orchestra.run.smithi178.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-09-10T10:32:59.636 INFO:teuthology.orchestra.run.smithi144.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.0 MB/s | 16 MB 00:05 2024-09-10T10:32:59.642 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:32:59.642 INFO:teuthology.orchestra.run.smithi144.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-10T10:32:59.650 INFO:teuthology.orchestra.run.smithi178.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-10T10:32:59.675 INFO:teuthology.orchestra.run.smithi178.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T10:32:59.701 INFO:teuthology.orchestra.run.smithi178.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-10T10:32:59.726 INFO:teuthology.orchestra.run.smithi178.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-10T10:32:59.751 INFO:teuthology.orchestra.run.smithi178.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T10:32:59.776 INFO:teuthology.orchestra.run.smithi178.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T10:32:59.802 INFO:teuthology.orchestra.run.smithi178.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-10T10:32:59.827 INFO:teuthology.orchestra.run.smithi178.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T10:32:59.852 INFO:teuthology.orchestra.run.smithi178.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T10:32:59.877 INFO:teuthology.orchestra.run.smithi178.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T10:32:59.903 INFO:teuthology.orchestra.run.smithi178.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T10:32:59.928 INFO:teuthology.orchestra.run.smithi178.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T10:32:59.962 INFO:teuthology.orchestra.run.smithi178.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T10:32:59.988 INFO:teuthology.orchestra.run.smithi178.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-10T10:33:00.012 INFO:teuthology.orchestra.run.smithi178.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T10:33:00.046 INFO:teuthology.orchestra.run.smithi178.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T10:33:00.096 INFO:teuthology.orchestra.run.smithi178.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T10:33:00.153 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-09-10T10:33:00.304 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-09-10T10:33:00.304 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-09-10T10:33:00.313 INFO:teuthology.orchestra.run.smithi178.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-10T10:33:00.564 INFO:teuthology.orchestra.run.smithi178.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 9.3 MB/s | 2.3 MB 00:00 2024-09-10T10:33:00.630 INFO:teuthology.orchestra.run.smithi178.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-10T10:33:00.672 INFO:teuthology.orchestra.run.smithi178.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 981 kB/s | 41 kB 00:00 2024-09-10T10:33:00.723 INFO:teuthology.orchestra.run.smithi178.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-10T10:33:00.790 INFO:teuthology.orchestra.run.smithi178.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-10T10:33:00.865 INFO:teuthology.orchestra.run.smithi178.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-10T10:33:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T10:33:00.940 INFO:teuthology.orchestra.run.smithi178.stdout:(155/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 3.7 MB/s | 3.2 MB 00:00 2024-09-10T10:33:01.021 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T10:33:01.021 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T10:33:02.545 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-09-10T10:33:02.545 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-09-10T10:33:03.132 INFO:teuthology.orchestra.run.smithi178.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.3 MB/s | 16 MB 00:04 2024-09-10T10:33:03.137 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:33:03.137 INFO:teuthology.orchestra.run.smithi178.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-10T10:33:03.355 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T10:33:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T10:33:04.376 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T10:33:04.526 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T10:33:04.526 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T10:33:06.746 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T10:33:06.746 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T10:33:08.940 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-09-10T10:33:09.069 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:33:09.107 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:33:09.161 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:33:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:33:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:33:09.273 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:33:09.609 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:33:09.721 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:09.843 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:09.880 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:09.914 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T10:33:09.972 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:10.020 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:33:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:33:10.168 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:33:10.201 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:33:10.241 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:33:10.252 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:33:10.282 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:33:10.288 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:10.316 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:33:10.365 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:33:10.390 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:10.414 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:33:10.480 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:33:10.524 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:33:10.544 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:33:10.692 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:33:10.698 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:33:10.817 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:10.898 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:33:10.941 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:10.970 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:11.023 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:33:11.069 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:11.100 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:33:11.138 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:11.216 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:11.257 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:33:11.304 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:33:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:11.435 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:11.439 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:11.459 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:33:11.522 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:33:11.551 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:33:11.559 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:33:11.573 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:33:11.649 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:33:11.688 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:33:11.723 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:33:11.786 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:33:11.787 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:33:11.831 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:33:11.864 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:33:11.907 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:33:11.940 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:33:11.956 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:33:11.976 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:33:12.045 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:33:12.077 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:33:12.106 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:33:12.138 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:33:12.181 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:33:12.188 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:12.259 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:33:12.259 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:12.292 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:33:12.339 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:33:12.372 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:33:12.442 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:33:12.487 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:33:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:12.569 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:33:12.595 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:33:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:33:12.710 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:33:12.724 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:33:12.766 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:33:12.773 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:33:12.839 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:33:12.883 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:33:12.916 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:33:12.959 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:33:12.965 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T10:33:12.992 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:33:13.028 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:33:13.077 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:33:13.098 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:33:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:33:13.150 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:33:13.182 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:33:13.182 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:33:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:33:13.228 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:33:13.253 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:33:13.295 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:33:13.302 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:33:13.306 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:33:13.345 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:33:13.376 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:33:13.387 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:33:13.415 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:33:13.425 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:33:13.458 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:33:13.495 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:33:13.506 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:33:13.537 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:33:13.556 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:33:13.630 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:33:13.691 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:33:13.698 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:33:13.749 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:33:13.761 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:33:13.804 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:33:13.812 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:33:13.876 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:33:13.908 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:33:13.953 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:33:13.956 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:14.008 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:33:14.044 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:33:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:14.087 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:33:14.117 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:33:14.129 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:33:14.263 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:33:14.298 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:33:14.298 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:33:14.337 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:33:14.338 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:33:14.357 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:33:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:14.437 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:33:14.441 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:33:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:33:14.471 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:14.481 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:33:14.493 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:33:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:33:14.530 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:33:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:33:14.563 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:33:14.588 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:33:14.615 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:33:14.673 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:33:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:33:14.798 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:33:14.834 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:33:14.889 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:33:14.961 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:33:14.973 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:33:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:33:14.994 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:33:15.030 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:33:15.071 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:33:15.076 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:33:15.107 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:33:15.121 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:33:15.150 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:33:15.172 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:33:15.189 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:33:15.214 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:33:15.228 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:33:15.259 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:33:15.264 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:15.293 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:33:15.326 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:33:15.348 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:33:15.353 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:15.367 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:33:15.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:33:15.420 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:33:15.432 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:33:15.514 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:33:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:33:15.591 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:33:15.610 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:33:15.623 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:33:15.694 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:33:15.718 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:33:15.724 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:33:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:33:15.773 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:33:15.787 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:33:15.791 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:33:15.815 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:33:15.816 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:33:15.850 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:33:15.909 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:33:15.915 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:33:15.960 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:33:15.992 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:33:16.036 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:33:16.040 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:33:16.077 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:33:16.090 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:33:16.113 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:33:16.117 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:33:16.154 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:33:16.179 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:33:16.191 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:33:16.222 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:33:16.233 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:33:16.252 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:33:16.266 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:33:16.287 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:33:16.309 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:33:16.320 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:33:16.365 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:33:16.383 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:33:16.400 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:33:16.419 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:33:16.472 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:33:16.485 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:33:16.495 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:33:16.508 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:33:16.538 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:33:16.542 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:33:16.570 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:33:16.581 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:33:16.584 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:33:16.631 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:33:16.671 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:33:16.689 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:33:16.704 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:33:16.720 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:33:16.739 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:33:16.783 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:33:16.836 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:33:16.870 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:33:16.892 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:33:16.929 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:33:17.121 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:33:17.160 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:33:17.237 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:33:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:33:17.427 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:33:17.476 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:33:17.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:33:17.516 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:33:17.526 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:33:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:33:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:33:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:33:17.639 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:33:17.673 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:33:17.718 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:33:17.802 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:33:17.804 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:33:17.854 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:33:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:33:17.911 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:33:17.921 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:33:17.993 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:33:18.024 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:33:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:33:18.038 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:33:18.062 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:33:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:33:18.108 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:33:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:33:18.153 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:33:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:33:18.246 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:33:18.380 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:33:18.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:33:18.456 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:33:18.558 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:33:18.599 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:33:18.638 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:33:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:33:18.697 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:33:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:33:18.803 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:33:18.901 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:33:19.021 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:33:19.054 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:33:19.090 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:33:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:33:19.172 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:33:19.203 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:33:19.233 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:33:19.268 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:33:19.274 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:33:19.301 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:33:19.342 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:33:19.368 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:33:19.377 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:33:19.396 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:33:19.430 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:33:19.472 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:33:19.524 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:33:19.557 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:33:19.659 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:33:19.710 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:33:19.741 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:33:19.785 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:33:19.876 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:33:19.906 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:33:19.943 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:33:20.136 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:33:20.227 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:33:20.255 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:33:20.288 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:33:20.423 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:33:20.434 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:33:20.446 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:33:20.517 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:33:20.556 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:33:20.601 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:33:20.651 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:33:20.715 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:33:20.741 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:33:20.749 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:33:20.772 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:33:20.808 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:33:20.848 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:33:20.903 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:33:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:33:21.035 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:33:21.085 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:33:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:33:21.286 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:33:21.299 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:33:21.320 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:33:21.391 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:33:21.400 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:33:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:33:22.064 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:33:22.180 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:33:22.897 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:22.930 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:23.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:33:23.476 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:23.517 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:33:23.528 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:33:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:33:23.589 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:33:23.602 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:33:23.602 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:33:23.602 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:33:23.602 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:23.643 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:33:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:33:23.952 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:33:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:33:24.074 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:33:24.110 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:33:24.373 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:24.414 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:33:24.507 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:33:24.508 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:33:24.508 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:33:24.508 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:24.549 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:33:24.591 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:33:24.630 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:33:24.701 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:33:24.732 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:33:27.070 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:27.105 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:27.654 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:33:27.691 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:33:27.783 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:33:27.783 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:33:27.784 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:33:27.784 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:27.826 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-09-10T10:33:35.601 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:35.782 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:33:35.798 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:33:35.798 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:35.798 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:33:35.798 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:36.512 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:33:36.512 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-09-10T10:33:36.512 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-09-10T10:33:36.513 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-09-10T10:33:36.513 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-09-10T10:33:36.513 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-09-10T10:33:36.513 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-09-10T10:33:36.513 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-09-10T10:33:36.513 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:36.685 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:33:36.702 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:33:36.702 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:36.702 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:33:36.702 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:37.285 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:33:37.311 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:33:37.311 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:37.312 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:33:37.312 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:33:37.312 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:33:37.312 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:37.544 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:33:37.567 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:33:37.568 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:37.568 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:33:37.568 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:33:37.568 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:33:37.568 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:38.028 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:33:38.052 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:33:38.052 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:38.052 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:33:38.052 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:33:38.052 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:33:38.052 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:38.149 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:33:38.189 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:33:38.198 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:33:38.215 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:33:38.216 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:38.216 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:33:38.216 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:33:38.216 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:33:38.216 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:38.262 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:33:38.447 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:33:38.464 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:33:38.487 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:33:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:33:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:33:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:33:38.488 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:38.496 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:33:38.573 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:33:38.669 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:33:38.799 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:33:38.847 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:33:38.878 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:33:38.906 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:33:38.923 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:33:38.947 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:33:38.947 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:38.947 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:33:38.947 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:33:38.947 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:33:38.947 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:39.053 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:33:39.102 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:33:39.157 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:33:39.335 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:33:39.341 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:33:39.363 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:33:39.395 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:33:39.399 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:33:39.458 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:33:39.468 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-09-10T10:33:39.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:39.486 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:33:39.556 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:33:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:33:39.678 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:33:39.678 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:39.678 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:33:39.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:39.696 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:33:39.742 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:33:39.781 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:33:39.818 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:33:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:33:40.317 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:33:40.348 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:33:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:33:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:33:40.529 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:33:40.571 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:33:41.173 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:33:41.200 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:33:41.200 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:41.200 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:33:41.200 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:33:41.200 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:33:41.200 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:41.423 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:33:41.447 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:33:41.447 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:41.447 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:33:41.447 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:33:41.447 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:33:41.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:41.479 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:33:41.517 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:33:41.536 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:33:41.564 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:33:41.665 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:33:41.834 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:33:41.866 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:33:41.890 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:33:41.890 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:41.891 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:33:41.891 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:33:41.891 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:33:41.891 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:41.891 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:33:41.914 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:33:41.914 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:41.914 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:33:41.914 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:33:41.914 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:33:41.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:41.942 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:33:41.953 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:33:42.029 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:33:42.053 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:33:42.076 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:33:42.111 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:33:42.132 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:33:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:33:42.366 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:33:42.444 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:33:42.475 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:33:42.509 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:33:42.531 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:33:42.610 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:33:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:33:42.717 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:33:42.748 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:33:42.771 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:33:42.776 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:33:42.803 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:33:42.826 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:33:42.827 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:42.827 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:33:42.827 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:33:42.827 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:33:42.827 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:42.878 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:33:42.890 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:33:42.988 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:33:43.033 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:33:43.051 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:33:43.069 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:33:43.094 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:33:43.094 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:43.094 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:33:43.094 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:33:43.094 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:33:43.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:43.145 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:33:43.168 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:33:43.168 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:43.168 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:33:43.168 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:43.196 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:33:43.212 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:33:43.225 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:33:43.235 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:33:43.236 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:43.236 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:33:43.236 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:33:43.236 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:33:43.236 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:33:43.326 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:33:43.365 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:33:43.546 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:33:43.570 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:33:43.570 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:43.570 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:33:43.570 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:33:43.570 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:33:43.570 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:43.989 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:33:44.022 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:33:44.046 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:33:44.046 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:44.046 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:33:44.046 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:33:44.046 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:33:44.046 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:44.098 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:33:44.121 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:33:44.121 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:44.121 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:33:44.121 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:44.165 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:33:44.189 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:33:44.189 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:44.189 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:33:44.189 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:33:44.189 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:33:44.189 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:44.402 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:33:44.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:33:44.500 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:33:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:33:44.523 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:44.523 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:33:44.523 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:33:44.523 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:33:44.523 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:33:45.414 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:33:45.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:33:45.543 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:33:45.712 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:33:45.744 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:33:45.768 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:33:45.768 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:45.768 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:33:45.768 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:33:45.769 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:33:45.769 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:45.819 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:33:45.831 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:33:45.930 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:33:45.988 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:33:46.938 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:33:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:33:46.987 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:33:46.987 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:46.987 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:33:46.987 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:33:46.987 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:33:46.987 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:47.031 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:33:47.054 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:33:47.054 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:47.054 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:33:47.055 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:33:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:33:47.130 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:47.130 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:33:47.130 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:33:47.130 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:33:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:47.440 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:33:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:33:47.464 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:33:47.464 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:33:47.464 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:33:47.464 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:33:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:33:48.159 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:33:48.200 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:33:48.249 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:33:48.290 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:33:48.326 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:33:48.512 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:33:48.564 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:33:48.594 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:33:48.624 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:33:48.671 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:33:49.095 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:33:49.145 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:33:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:33:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:33:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:33:49.275 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:49.319 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:49.341 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:49.426 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:33:49.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:33:49.513 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:33:49.552 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:33:49.598 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:33:50.162 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:33:50.222 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:50.258 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:33:50.271 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:50.292 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:50.304 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:33:50.394 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:33:50.428 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:33:50.453 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:33:50.488 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:33:50.488 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:33:50.572 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:33:50.572 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:33:50.656 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:33:50.681 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:33:50.686 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:50.687 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:50.874 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:50.875 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:33:50.950 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:33:51.015 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:33:51.122 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:33:51.210 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:33:51.259 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:33:51.361 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:33:51.396 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:33:51.421 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:33:51.448 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:33:51.448 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:33:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:33:51.532 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:33:51.616 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:33:51.641 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:33:51.646 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:51.647 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:51.826 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:51.826 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:33:51.919 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:33:51.984 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:33:52.053 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:33:52.094 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:33:52.134 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:33:52.175 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:33:52.210 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:33:52.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:33:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:33:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:33:52.500 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:33:52.546 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:33:53.102 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:53.144 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:53.165 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:33:54.005 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:33:54.091 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:33:54.139 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:33:54.231 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:33:54.260 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:33:54.286 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:33:54.320 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:33:54.320 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:33:54.421 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:33:54.421 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:33:54.513 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:33:54.538 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:33:54.544 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:54.544 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:54.731 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:33:54.732 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:33:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:33:54.890 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:33:56.953 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:33:56.953 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:33:56.953 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:33:56.953 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:33:56.953 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:33:56.953 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:33:56.954 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:33:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:33:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:33:56.957 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:33:56.958 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:33:56.959 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:33:56.960 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:33:56.961 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:33:56.962 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:33:56.963 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:33:56.964 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:33:56.965 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:33:56.966 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:33:56.966 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:33:56.966 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:33:57.847 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:33:57.848 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:33:57.849 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:33:57.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:33:57.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:33:57.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:33:57.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:33:57.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:33:57.866 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:33:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:33:57.868 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:33:57.869 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:33:57.870 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:33:57.871 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:33:59.640 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:33:59.640 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:33:59.641 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.642 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.643 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:33:59.645 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:33:59.646 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:33:59.647 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:33:59.648 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:33:59.649 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:33:59.650 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:33:59.651 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:33:59.652 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:33:59.653 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:33:59.653 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-09-10T10:34:00.124 DEBUG:teuthology.parallel:result is None 2024-09-10T10:34:00.567 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:00.567 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:34:00.567 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-09-10T10:34:00.567 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:00.567 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:00.567 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.568 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.569 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:34:00.570 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:34:00.571 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:34:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:34:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:34:00.574 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:34:00.575 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:34:00.576 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.577 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:34:00.578 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:34:00.579 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:34:00.580 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:34:00.580 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T10:34:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:34:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:34:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:34:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:34:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:34:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:34:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:34:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:34:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:34:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:34:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:34:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:34:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:34:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:34:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:34:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:34:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:34:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:34:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:34:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:34:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:34:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:34:00.991 DEBUG:teuthology.parallel:result is None 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.396 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.397 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.398 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.398 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.398 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.398 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.399 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:34:03.400 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.401 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:03.402 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:34:03.403 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:34:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:34:03.405 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:34:03.406 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.407 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:34:03.408 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:34:03.409 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:34:03.410 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:34:03.410 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:34:03.410 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T10:34:03.840 DEBUG:teuthology.parallel:result is None 2024-09-10T10:34:03.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:34:03.994 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:34:04.021 INFO:teuthology.orchestra.run.smithi143.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:34:04.022 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:34:04.022 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:34:04.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:34:04.161 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:34:04.188 INFO:teuthology.orchestra.run.smithi144.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:34:04.188 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:34:04.188 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:34:04.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:34:04.327 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:34:04.349 INFO:teuthology.orchestra.run.smithi178.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:34:04.349 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:34:04.349 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:34:04.351 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T10:34:04.351 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:34:04.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:34:04.384 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:34:04.385 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:34:04.417 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:34:04.418 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:34:04.445 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T10:34:04.445 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:34:04.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:34:04.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:34:04.542 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:34:04.542 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:34:04.572 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:34:04.640 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:34:04.640 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:34:04.666 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:34:04.731 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T10:34:04.731 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:34:04.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:34:04.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:34:04.826 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:34:04.826 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:34:04.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:34:04.922 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:34:04.922 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:34:04.946 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:34:05.011 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T10:34:05.011 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:34:05.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:34:05.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:34:05.106 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:34:05.107 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:34:05.135 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:34:05.202 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:34:05.202 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:34:05.226 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:34:05.291 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T10:34:05.390 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2}} 2024-09-10T10:34:05.390 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:34:05.390 INFO:tasks.cephadm:Cluster fsid is 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:34:05.390 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T10:34:05.391 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.143', 'mon.b': '172.21.15.144', 'mon.c': '172.21.15.178'} 2024-09-10T10:34:05.391 INFO:tasks.cephadm:First mon is mon.a on smithi143 2024-09-10T10:34:05.391 INFO:tasks.cephadm:First mgr is x 2024-09-10T10:34:05.391 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T10:34:05.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname $(hostname -s) 2024-09-10T10:34:05.419 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-09-10T10:34:05.447 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-09-10T10:34:05.472 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T10:34:05.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:34:05.619 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T10:34:05.754 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:34:05.756 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T10:34:05.756 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T10:34:05.756 DEBUG:teuthology.orchestra.run.smithi143:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.149 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 10:34 /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.149 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.548 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 10:34 /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.549 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.931 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 10:34 /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.932 DEBUG:teuthology.orchestra.run.smithi143:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.952 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:06.972 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T10:34:07.009 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T10:34:07.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T10:34:07.011 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T10:34:07.014 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T10:34:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:34:08.690 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:34:08.698 INFO:teuthology.orchestra.run.smithi143.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:34:35.813 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T10:34:35.813 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:34:35.814 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T10:34:35.814 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-09-10T10:34:35.814 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T10:34:35.814 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-10T10:34:35.814 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T10:34:59.020 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T10:34:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:34:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T10:34:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-09-10T10:34:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T10:34:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-10T10:34:59.021 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T10:34:59.055 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:34:59.055 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:34:59.055 INFO:teuthology.orchestra.run.smithi143.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T10:34:59.055 INFO:teuthology.orchestra.run.smithi143.stdout: "repo_digests": [ 2024-09-10T10:34:59.055 INFO:teuthology.orchestra.run.smithi143.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T10:34:59.056 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:34:59.056 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:34:59.082 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph 2024-09-10T10:34:59.119 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-09-10T10:34:59.146 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-09-10T10:34:59.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /etc/ceph 2024-09-10T10:34:59.205 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-09-10T10:34:59.230 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-09-10T10:34:59.258 INFO:tasks.cephadm:Writing seed config... 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T10:34:59.260 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T10:34:59.261 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T10:34:59.262 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T10:34:59.263 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T10:34:59.264 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:34:59.264 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T10:34:59.279 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 34212b78-6f60-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T10:34:59.280 DEBUG:teuthology.orchestra.run.smithi143:mon.a> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T10:34:59.322 DEBUG:teuthology.orchestra.run.smithi143:mgr.x> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T10:34:59.365 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T10:34:59.365 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.143 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:34:59.566 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:34:59.566 INFO:teuthology.orchestra.run.smithi143.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', '34212b78-6f60-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.143', '--skip-admin-label'] 2024-09-10T10:34:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T10:34:59.567 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying podman|docker is present... 2024-09-10T10:34:59.590 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 5.2.2 2024-09-10T10:34:59.590 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying lvm2 is present... 2024-09-10T10:34:59.590 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying time synchronization is in place... 2024-09-10T10:34:59.597 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T10:34:59.598 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T10:34:59.604 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T10:34:59.604 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-10T10:34:59.610 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout enabled 2024-09-10T10:34:59.617 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout active 2024-09-10T10:34:59.617 INFO:teuthology.orchestra.run.smithi143.stdout:Unit chronyd.service is enabled and running 2024-09-10T10:34:59.617 INFO:teuthology.orchestra.run.smithi143.stdout:Repeating the final host check... 2024-09-10T10:34:59.639 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 5.2.2 2024-09-10T10:34:59.639 INFO:teuthology.orchestra.run.smithi143.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T10:34:59.639 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl is present 2024-09-10T10:34:59.639 INFO:teuthology.orchestra.run.smithi143.stdout:lvcreate is present 2024-09-10T10:34:59.645 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T10:34:59.645 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T10:34:59.652 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T10:34:59.652 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-10T10:34:59.658 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout enabled 2024-09-10T10:34:59.664 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout active 2024-09-10T10:34:59.664 INFO:teuthology.orchestra.run.smithi143.stdout:Unit chronyd.service is enabled and running 2024-09-10T10:34:59.665 INFO:teuthology.orchestra.run.smithi143.stdout:Host looks OK 2024-09-10T10:34:59.665 INFO:teuthology.orchestra.run.smithi143.stdout:Cluster fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:34:59.665 INFO:teuthology.orchestra.run.smithi143.stdout:Acquiring lock 139682547504128 on /run/cephadm/34212b78-6f60-11ef-bcea-c7b262605968.lock 2024-09-10T10:34:59.665 INFO:teuthology.orchestra.run.smithi143.stdout:Lock 139682547504128 acquired on /run/cephadm/34212b78-6f60-11ef-bcea-c7b262605968.lock 2024-09-10T10:34:59.665 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying IP 172.21.15.143 port 3300 ... 2024-09-10T10:34:59.665 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying IP 172.21.15.143 port 6789 ... 2024-09-10T10:34:59.666 INFO:teuthology.orchestra.run.smithi143.stdout:Base mon IP(s) is [172.21.15.143:3300, 172.21.15.143:6789], mon addrv is [v2:172.21.15.143:3300,v1:172.21.15.143:6789] 2024-09-10T10:34:59.669 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.143 metric 100 2024-09-10T10:34:59.669 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.143 metric 100 2024-09-10T10:34:59.671 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T10:34:59.672 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T10:34:59.672 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T10:34:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T10:34:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T10:34:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T10:34:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T10:34:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7783/64 scope link noprefixroute 2024-09-10T10:34:59.674 INFO:teuthology.orchestra.run.smithi143.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T10:34:59.675 INFO:teuthology.orchestra.run.smithi143.stdout:Mon IP `172.21.15.143` is in CIDR network `172.21.0.0/20` 2024-09-10T10:34:59.675 INFO:teuthology.orchestra.run.smithi143.stdout:Mon IP `172.21.15.143` is in CIDR network `172.21.0.0/20` 2024-09-10T10:34:59.675 INFO:teuthology.orchestra.run.smithi143.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T10:34:59.676 INFO:teuthology.orchestra.run.smithi143.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T10:34:59.676 INFO:teuthology.orchestra.run.smithi143.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:35:00.074 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T10:35:00.074 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:35:00.074 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T10:35:00.074 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T10:35:00.075 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T10:35:00.075 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T10:35:00.075 INFO:teuthology.orchestra.run.smithi143.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi143.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi143.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:35:01.012 INFO:teuthology.orchestra.run.smithi143.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T10:35:01.917 INFO:teuthology.orchestra.run.smithi143.stdout:stat: stdout 167 167 2024-09-10T10:35:01.917 INFO:teuthology.orchestra.run.smithi143.stdout:Creating initial keys... 2024-09-10T10:35:02.797 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph-authtool: stdout AQDWIOBmZ0x0DxAA1UzIz3qbbZJTMAxndsp/Aw== 2024-09-10T10:35:03.811 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph-authtool: stdout AQDXIOBmZiDCBBAAQm1uMAaStoSEqzXi7RmkiQ== 2024-09-10T10:35:04.741 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph-authtool: stdout AQDYIOBmABSeCBAAj4K9WWmUGcbYl3vBPXTyRQ== 2024-09-10T10:35:04.741 INFO:teuthology.orchestra.run.smithi143.stdout:Creating initial monmap... 2024-09-10T10:35:05.613 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool for a [v2:172.21.15.143:3300,v1:172.21.15.143:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = quincy 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: set fsid to 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:35:05.614 INFO:teuthology.orchestra.run.smithi143.stdout:Creating mon... 2024-09-10T10:35:06.554 INFO:teuthology.orchestra.run.smithi143.stdout:create mon.a on 2024-09-10T10:35:06.754 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T10:35:06.892 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T10:35:07.064 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-34212b78-6f60-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-34212b78-6f60-11ef-bcea-c7b262605968.target. 2024-09-10T10:35:07.064 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-34212b78-6f60-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-34212b78-6f60-11ef-bcea-c7b262605968.target. 2024-09-10T10:35:07.297 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a 2024-09-10T10:35:07.297 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to reset failed state of unit ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T10:35:07.453 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-34212b78-6f60-11ef-bcea-c7b262605968.target.wants/ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-34212b78-6f60-11ef-bcea-c7b262605968@.service. 2024-09-10T10:35:07.964 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:07 smithi143 podman[30659]: 2024-09-10 10:35:07.922924659 +0000 UTC m=+0.332033310 container start 137fe15d940f982b30eead7857d3e91e5a20cb2503e8d861e20f4ee389be0935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:35:08.032 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:35:08.032 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-10T10:35:08.039 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:35:08.039 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-10T10:35:08.039 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-10T10:35:08.039 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T10:35:08.040 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mon to start... 2024-09-10T10:35:08.040 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mon... 2024-09-10T10:35:08.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:07 smithi143 ceph-mon[30678]: mkfs 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:08.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:07 smithi143 ceph-mon[30678]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T10:35:08.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 bash[30659]: 137fe15d940f982b30eead7857d3e91e5a20cb2503e8d861e20f4ee389be0935 2024-09-10T10:35:08.275 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 systemd[1]: Started Ceph mon.a for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:35:09.151 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T10:35:09.151 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: monmap epoch 1 2024-09-10T10:35:09.151 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:09.151 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: last_changed 2024-09-10T10:35:05.055889+0000 2024-09-10T10:35:09.151 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: min_mon_release 19 (squid) 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: election_strategy: 1 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: fsmap 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: mgrmap e1: no daemons active 2024-09-10T10:35:09.152 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:08 smithi143 ceph-mon[30678]: from='client.? 172.21.15.143:0/2517956465' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T10:35:09.154 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T10:35:09.160 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout id: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:09.160 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T10:35:09.160 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:09.160 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout services: 2024-09-10T10:35:09.160 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.660109s) 2024-09-10T10:35:09.160 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T10:35:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T10:35:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout data: 2024-09-10T10:35:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T10:35:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T10:35:09.161 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.518+0000 7f67ac8d2640 1 Processor -- start 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.518+0000 7f67ac8d2640 1 -- start start 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.518+0000 7f67ac8d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a8106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.518+0000 7f67ac8d2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8106de0 con 0x7f67a8106410 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.519+0000 7f67a659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a8106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.519+0000 7f67a659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a8106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48988/0 (socket says 172.21.15.143:48988) 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.519+0000 7f67a659c640 1 -- 172.21.15.143:0/2432417838 learned_addr learned my addr 172.21.15.143:0/2432417838 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.519+0000 7f67a659c640 1 -- 172.21.15.143:0/2432417838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a8107610 con 0x7f67a8106410 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.521+0000 7f67a659c640 1 --2- 172.21.15.143:0/2432417838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a8106810 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6798009820 tx=0x7f679802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acf533df7877922 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.521+0000 7f67a559a640 1 -- 172.21.15.143:0/2432417838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6798037070 con 0x7f67a8106410 2024-09-10T10:35:09.162 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.521+0000 7f67a559a640 1 -- 172.21.15.143:0/2432417838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6798032440 con 0x7f67a8106410 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.521+0000 7f67a559a640 1 -- 172.21.15.143:0/2432417838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6798032740 con 0x7f67a8106410 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2432417838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 msgr2=0x7f67a8106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 --2- 172.21.15.143:0/2432417838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a8106810 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6798009820 tx=0x7f679802a270 comp rx=0 tx=0).stop 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2432417838 shutdown_connections 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 --2- 172.21.15.143:0/2432417838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a8106810 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2432417838 >> 172.21.15.143:0/2432417838 conn(0x7f67a8101bc0 msgr2=0x7f67a8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2432417838 shutdown_connections 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.522+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2432417838 wait complete. 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67ac8d2640 1 Processor -- start 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67ac8d2640 1 -- start start 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67ac8d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:09.163 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67ac8d2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8107bb0 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67a659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67a659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49004/0 (socket says 172.21.15.143:49004) 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.523+0000 7f67a659c640 1 -- 172.21.15.143:0/2517956465 learned_addr learned my addr 172.21.15.143:0/2517956465 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f67a659c640 1 -- 172.21.15.143:0/2517956465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a819b1f0 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f67a659c640 1 --2- 172.21.15.143:0/2517956465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a819acb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f67980090b0 tx=0x7f67980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6798041070 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f679803c7a0 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a819b480 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6798037040 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a819e170 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6798044070 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.524+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f679803d3f0 con 0x7f67a8106410 2024-09-10T10:35:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.525+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a8108200 con 0x7f67a8106410 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.529+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6798046e20 con 0x7f67a8106410 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.617+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f67a819e730 con 0x7f67a8106410 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.618+0000 7f679f7fe640 1 -- 172.21.15.143:0/2517956465 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6798046450 con 0x7f67a8106410 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 msgr2=0x7f67a819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 --2- 172.21.15.143:0/2517956465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a819acb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f67980090b0 tx=0x7f67980046c0 comp rx=0 tx=0).stop 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 shutdown_connections 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 --2- 172.21.15.143:0/2517956465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67a8106410 0x7f67a819acb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 >> 172.21.15.143:0/2517956465 conn(0x7f67a8101bc0 msgr2=0x7f67a80751f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 shutdown_connections 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:08.619+0000 7f67ac8d2640 1 -- 172.21.15.143:0/2517956465 wait complete. 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:mon is available 2024-09-10T10:35:09.165 INFO:teuthology.orchestra.run.smithi143.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T10:35:10.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:10.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T10:35:10.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout fsid = 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.143:3300,v1:172.21.15.143:6789] 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T10:35:10.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.571+0000 7fec6dde9640 1 Processor -- start 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.571+0000 7fec6dde9640 1 -- start start 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.572+0000 7fec6dde9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec681067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.572+0000 7fec6dde9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec68106dc0 con 0x7fec681063f0 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.572+0000 7fec677fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec681067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.572+0000 7fec677fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec681067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49010/0 (socket says 172.21.15.143:49010) 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.572+0000 7fec677fe640 1 -- 172.21.15.143:0/3986925524 learned_addr learned my addr 172.21.15.143:0/3986925524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.572+0000 7fec677fe640 1 -- 172.21.15.143:0/3986925524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec681075f0 con 0x7fec681063f0 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec677fe640 1 --2- 172.21.15.143:0/3986925524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec681067f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fec58009820 tx=0x7fec5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2f7cd0669b2329 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:10.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec66ffd640 1 -- 172.21.15.143:0/3986925524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec58037070 con 0x7fec681063f0 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec66ffd640 1 -- 172.21.15.143:0/3986925524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58032440 con 0x7fec681063f0 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec66ffd640 1 -- 172.21.15.143:0/3986925524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec58032740 con 0x7fec681063f0 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 -- 172.21.15.143:0/3986925524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 msgr2=0x7fec681067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 --2- 172.21.15.143:0/3986925524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec681067f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fec58009820 tx=0x7fec5802a270 comp rx=0 tx=0).stop 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 -- 172.21.15.143:0/3986925524 shutdown_connections 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 --2- 172.21.15.143:0/3986925524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec681067f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 -- 172.21.15.143:0/3986925524 >> 172.21.15.143:0/3986925524 conn(0x7fec68101ba0 msgr2=0x7fec68103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 -- 172.21.15.143:0/3986925524 shutdown_connections 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.573+0000 7fec6dde9640 1 -- 172.21.15.143:0/3986925524 wait complete. 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec6dde9640 1 Processor -- start 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec6dde9640 1 -- start start 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec6dde9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec68199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:10.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec6dde9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec68107b90 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec677fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec68199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec677fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec68199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49012/0 (socket says 172.21.15.143:49012) 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.574+0000 7fec677fe640 1 -- 172.21.15.143:0/4000344822 learned_addr learned my addr 172.21.15.143:0/4000344822 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec677fe640 1 -- 172.21.15.143:0/4000344822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6819a0d0 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec677fe640 1 --2- 172.21.15.143:0/4000344822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec68199b90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fec580090b0 tx=0x7fec580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec58039600 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec580397a0 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6819a360 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec58039ac0 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6807afb0 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec5804a060 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.575+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec58043730 con 0x7fec681063f0 2024-09-10T10:35:10.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.576+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec681081e0 con 0x7fec681063f0 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.578+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fec5803e070 con 0x7fec681063f0 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.635+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fec6807b5b0 con 0x7fec681063f0 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.638+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fec580439d0 con 0x7fec681063f0 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.638+0000 7fec657fa640 1 -- 172.21.15.143:0/4000344822 <== mon.0 v2:172.21.15.143:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fec5804c8e0 con 0x7fec681063f0 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 msgr2=0x7fec68199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 --2- 172.21.15.143:0/4000344822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec68199b90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fec580090b0 tx=0x7fec580046c0 comp rx=0 tx=0).stop 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 shutdown_connections 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 --2- 172.21.15.143:0/4000344822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec681063f0 0x7fec68199b90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 >> 172.21.15.143:0/4000344822 conn(0x7fec68101ba0 msgr2=0x7fec6818de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 shutdown_connections 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:09.639+0000 7fec6dde9640 1 -- 172.21.15.143:0/4000344822 wait complete. 2024-09-10T10:35:10.148 INFO:teuthology.orchestra.run.smithi143.stdout:Generating new minimal ceph.conf... 2024-09-10T10:35:10.258 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:09 smithi143 ceph-mon[30678]: from='client.? 172.21.15.143:0/4000344822' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T10:35:10.258 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:09 smithi143 ceph-mon[30678]: from='client.? 172.21.15.143:0/4000344822' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T10:35:11.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc7131c8640 1 Processor -- start 2024-09-10T10:35:11.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc7131c8640 1 -- start start 2024-09-10T10:35:11.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc7131c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:11.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc7131c8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c07c8d0 con 0x7fc70c07dea0 2024-09-10T10:35:11.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc710f64640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:11.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc710f64640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49024/0 (socket says 172.21.15.143:49024) 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc710f64640 1 -- 172.21.15.143:0/1853134400 learned_addr learned my addr 172.21.15.143:0/1853134400 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.543+0000 7fc710f64640 1 -- 172.21.15.143:0/1853134400 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c07ca50 con 0x7fc70c07dea0 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.544+0000 7fc710f64640 1 --2- 172.21.15.143:0/1853134400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c07c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc700009820 tx=0x7fc70002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a49b67391ac96f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.544+0000 7fc70b7fe640 1 -- 172.21.15.143:0/1853134400 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc700037070 con 0x7fc70c07dea0 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.544+0000 7fc70b7fe640 1 -- 172.21.15.143:0/1853134400 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc7000328d0 con 0x7fc70c07dea0 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.544+0000 7fc70b7fe640 1 -- 172.21.15.143:0/1853134400 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7000307d0 con 0x7fc70c07dea0 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 -- 172.21.15.143:0/1853134400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 msgr2=0x7fc70c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 --2- 172.21.15.143:0/1853134400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c07c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc700009820 tx=0x7fc70002a270 comp rx=0 tx=0).stop 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 -- 172.21.15.143:0/1853134400 shutdown_connections 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 --2- 172.21.15.143:0/1853134400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c07c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 -- 172.21.15.143:0/1853134400 >> 172.21.15.143:0/1853134400 conn(0x7fc70c102c50 msgr2=0x7fc70c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 -- 172.21.15.143:0/1853134400 shutdown_connections 2024-09-10T10:35:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 -- 172.21.15.143:0/1853134400 wait complete. 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 Processor -- start 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.545+0000 7fc7131c8640 1 -- start start 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc7131c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc7131c8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c10dab0 con 0x7fc70c07dea0 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc710f64640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc710f64640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49034/0 (socket says 172.21.15.143:49034) 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc710f64640 1 -- 172.21.15.143:0/1666282578 learned_addr learned my addr 172.21.15.143:0/1666282578 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc710f64640 1 -- 172.21.15.143:0/1666282578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c19dbb0 con 0x7fc70c07dea0 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc710f64640 1 --2- 172.21.15.143:0/1666282578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c19d670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc7000097f0 tx=0x7fc700030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc700037040 con 0x7fc70c07dea0 2024-09-10T10:35:11.146 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc700032bb0 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc70003a390 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c19de40 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.546+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a0b30 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.547+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc70003a5d0 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.547+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc700044ba0 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.547+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c07c380 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.550+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc700044400 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.607+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc70c1a10b0 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.607+0000 7fc709ffb640 1 -- 172.21.15.143:0/1666282578 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fc700032d60 con 0x7fc70c07dea0 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 msgr2=0x7fc70c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 --2- 172.21.15.143:0/1666282578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c19d670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc7000097f0 tx=0x7fc700030d00 comp rx=0 tx=0).stop 2024-09-10T10:35:11.147 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 shutdown_connections 2024-09-10T10:35:11.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 --2- 172.21.15.143:0/1666282578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc70c07dea0 0x7fc70c19d670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:11.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 >> 172.21.15.143:0/1666282578 conn(0x7fc70c102c50 msgr2=0x7fc70c1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:11.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 shutdown_connections 2024-09-10T10:35:11.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:10.608+0000 7fc7131c8640 1 -- 172.21.15.143:0/1666282578 wait complete. 2024-09-10T10:35:11.148 INFO:teuthology.orchestra.run.smithi143.stdout:Restarting the monitor... 2024-09-10T10:35:11.160 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:10 smithi143 ceph-mon[30678]: from='client.? 172.21.15.143:0/1666282578' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:35:11.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:11 smithi143 systemd[1]: Stopping Ceph mon.a for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:35:11.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a[30674]: 2024-09-10T10:35:11.390+0000 7f509c80c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:35:11.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a[30674]: 2024-09-10T10:35:11.390+0000 7f509c80c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T10:35:11.849 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:11 smithi143 podman[30933]: 2024-09-10 10:35:11.564244587 +0000 UTC m=+0.261423816 container died 137fe15d940f982b30eead7857d3e91e5a20cb2503e8d861e20f4ee389be0935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T10:35:12.199 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:11 smithi143 podman[30933]: 2024-09-10 10:35:11.956762537 +0000 UTC m=+0.653941770 container cleanup 137fe15d940f982b30eead7857d3e91e5a20cb2503e8d861e20f4ee389be0935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T10:35:12.200 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:11 smithi143 bash[30933]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a 2024-09-10T10:35:12.534 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 podman[30945]: 2024-09-10 10:35:12.199244251 +0000 UTC m=+0.630302750 container remove 137fe15d940f982b30eead7857d3e91e5a20cb2503e8d861e20f4ee389be0935 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:35:12.534 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T10:35:12.534 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 systemd[1]: Stopped Ceph mon.a for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:35:12.534 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 systemd[1]: Starting Ceph mon.a for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:35:12.794 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 podman[31038]: 2024-09-10 10:35:12.533278996 +0000 UTC m=+0.101346554 container create bd3317ce38ed9d1b3c7c110f0a5efffff7b36da64ba0b0809d790759dd7c683b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 podman[31038]: 2024-09-10 10:35:12.447889101 +0000 UTC m=+0.015956675 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 podman[31038]: 2024-09-10 10:35:12.735418906 +0000 UTC m=+0.303486464 container init bd3317ce38ed9d1b3c7c110f0a5efffff7b36da64ba0b0809d790759dd7c683b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 podman[31038]: 2024-09-10 10:35:12.738472606 +0000 UTC m=+0.306540165 container start bd3317ce38ed9d1b3c7c110f0a5efffff7b36da64ba0b0809d790759dd7c683b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: pidfile_write: ignore empty --pid-file 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: load: jerasure load: lrc 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: RocksDB version: 7.9.2 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Git sha 0 2024-09-10T10:35:12.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: DB SUMMARY 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: DB Session ID: 7JK6UEC7G5XOZSXPOCOH 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: CURRENT file: CURRENT 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T10:35:12.796 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104892 ; 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.error_if_exists: 0 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.create_if_missing: 0 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.paranoid_checks: 1 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.env: 0x562bbd58e940 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.info_log: 0x562bbe2df6e0 2024-09-10T10:35:12.797 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.statistics: (nil) 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.use_fsync: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_log_file_size: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.allow_fallocate: 1 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.use_direct_reads: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T10:35:12.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.db_log_dir: 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.wal_dir: 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.write_buffer_manager: 0x562bbe2eedc0 2024-09-10T10:35:12.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.unordered_write: 0 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.row_cache: None 2024-09-10T10:35:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.wal_filter: None 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.two_write_queues: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.wal_compression: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.atomic_flush: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.log_readahead_size: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T10:35:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_background_jobs: 2 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_background_compactions: -1 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_subcompactions: 1 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_open_files: -1 2024-09-10T10:35:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_background_flushes: -1 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Compression algorithms supported: 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kZSTD supported: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kXpressCompression supported: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kBZip2Compression supported: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kLZ4Compression supported: 1 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kZlibCompression supported: 1 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: kSnappyCompression supported: 1 2024-09-10T10:35:12.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.merge_operator: 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_filter: None 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562bbe2df320) 2024-09-10T10:35:12.804 INFO:journalctl@ceph.mon.a.smithi143.stdout: cache_index_and_filter_blocks: 1 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: pin_top_level_index_and_filter: 1 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: index_type: 0 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: data_block_index_type: 0 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: index_shortening: 1 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: checksum: 4 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: no_block_cache: 0 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache: 0x562bbe2d31f0 2024-09-10T10:35:12.805 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache_name: BinnedLRUCache 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache_options: 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: capacity : 536870912 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: num_shard_bits : 4 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: strict_capacity_limit : 0 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: high_pri_pool_ratio: 0.000 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_cache_compressed: (nil) 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: persistent_cache: (nil) 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_size: 4096 2024-09-10T10:35:12.806 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_size_deviation: 10 2024-09-10T10:35:12.807 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_restart_interval: 16 2024-09-10T10:35:12.807 INFO:journalctl@ceph.mon.a.smithi143.stdout: index_block_restart_interval: 1 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: metadata_block_size: 4096 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: partition_filters: 0 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: use_delta_encoding: 1 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: filter_policy: bloomfilter 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: whole_key_filtering: 1 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: verify_compression: 0 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: read_amp_bytes_per_bit: 0 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: format_version: 5 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: enable_index_compression: 1 2024-09-10T10:35:12.808 INFO:journalctl@ceph.mon.a.smithi143.stdout: block_align: 0 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout: max_auto_readahead_size: 262144 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout: prepopulate_block_cache: 0 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout: initial_auto_readahead_size: 8192 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression: NoCompression 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T10:35:12.809 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.num_levels: 7 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T10:35:12.810 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T10:35:12.811 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T10:35:12.812 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T10:35:12.813 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T10:35:12.814 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.inplace_update_support: 0 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T10:35:12.815 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.bloom_locality: 0 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.max_successive_merges: 0 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T10:35:12.816 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.ttl: 2592000 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.enable_blob_files: false 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.min_blob_size: 0 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T10:35:12.817 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4a2b5cfa-0f39-49ee-93be-89b94c81c53d 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964512764365, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T10:35:12.818 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964512765397, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100036, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97921, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14187, "raw_average_key_size": 49, "raw_value_size": 90210, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725964512, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4a2b5cfa-0f39-49ee-93be-89b94c81c53d", "db_session_id": "7JK6UEC7G5XOZSXPOCOH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964512765493, "job": 1, "event": "recovery_finished"} 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562bbe300e00 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: DB pointer 0x562bbe3fc000 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** DB Stats ** 2024-09-10T10:35:12.819 INFO:journalctl@ceph.mon.a.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T10:35:12.820 INFO:journalctl@ceph.mon.a.smithi143.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** Compaction Stats [default] ** 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T10:35:12.821 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: Cumulative compaction: 0.00 GB write, 30.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: Interval compaction: 0.00 GB write, 30.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: Block cache BinnedLRUCache@0x562bbe2d31f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: starting mon.a rank 0 at public addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] at bind addrs [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???) e1 preinit fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:12.822 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).mds e1 new map 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).mds e1 print_map 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: e1 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: btime 2024-09-10T10:35:07:959271+0000 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: legacy client fscid: -1 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout: No filesystems configured 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T10:35:12.823 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 bash[31038]: bd3317ce38ed9d1b3c7c110f0a5efffff7b36da64ba0b0809d790759dd7c683b 2024-09-10T10:35:12.824 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 systemd[1]: Started Ceph mon.a for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:35:12.824 INFO:teuthology.orchestra.run.smithi143.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T10:35:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T10:35:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: monmap epoch 1 2024-09-10T10:35:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:13.049 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: last_changed 2024-09-10T10:35:05.055889+0000 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: min_mon_release 19 (squid) 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: election_strategy: 1 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: fsmap 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T10:35:13.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:12 smithi143 ceph-mon[31057]: mgrmap e1: no daemons active 2024-09-10T10:35:13.798 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.226+0000 7fcb90abd640 1 Processor -- start 2024-09-10T10:35:13.798 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.226+0000 7fcb90abd640 1 -- start start 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.227+0000 7fcb90abd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.227+0000 7fcb90abd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c106de0 con 0x7fcb8c106410 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.227+0000 7fcb8a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.227+0000 7fcb8a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49048/0 (socket says 172.21.15.143:49048) 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.227+0000 7fcb8a59c640 1 -- 172.21.15.143:0/466000282 learned_addr learned my addr 172.21.15.143:0/466000282 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.227+0000 7fcb8a59c640 1 -- 172.21.15.143:0/466000282 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c107610 con 0x7fcb8c106410 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.229+0000 7fcb8a59c640 1 --2- 172.21.15.143:0/466000282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c106810 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009820 tx=0x7fcb7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3683543cf6448f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.229+0000 7fcb89d9b640 1 -- 172.21.15.143:0/466000282 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c037070 con 0x7fcb8c106410 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.229+0000 7fcb89d9b640 1 -- 172.21.15.143:0/466000282 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcb7c0328d0 con 0x7fcb8c106410 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.229+0000 7fcb89d9b640 1 -- 172.21.15.143:0/466000282 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c0307d0 con 0x7fcb8c106410 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 -- 172.21.15.143:0/466000282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 msgr2=0x7fcb8c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 --2- 172.21.15.143:0/466000282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c106810 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009820 tx=0x7fcb7c02a270 comp rx=0 tx=0).stop 2024-09-10T10:35:13.799 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 -- 172.21.15.143:0/466000282 shutdown_connections 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 --2- 172.21.15.143:0/466000282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c106810 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 -- 172.21.15.143:0/466000282 >> 172.21.15.143:0/466000282 conn(0x7fcb8c101bc0 msgr2=0x7fcb8c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 -- 172.21.15.143:0/466000282 shutdown_connections 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.230+0000 7fcb90abd640 1 -- 172.21.15.143:0/466000282 wait complete. 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb90abd640 1 Processor -- start 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb90abd640 1 -- start start 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb90abd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb90abd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c10a9c0 con 0x7fcb8c106410 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb8a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb8a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49052/0 (socket says 172.21.15.143:49052) 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb8a59c640 1 -- 172.21.15.143:0/3677298821 learned_addr learned my addr 172.21.15.143:0/3677298821 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:13.800 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.231+0000 7fcb8a59c640 1 -- 172.21.15.143:0/3677298821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c19b9d0 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb8a59c640 1 --2- 172.21.15.143:0/3677298821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c19b490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009940 tx=0x7fcb7c032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c042070 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c19bc60 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcb7c030c80 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c037040 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c19e950 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb7c045070 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.232+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcb7c047a40 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.233+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c10ab40 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.235+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcb7c03c700 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.292+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fcb8c106890 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.296+0000 7fcb837fe640 1 -- 172.21.15.143:0/3677298821 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fcb7c03c8e0 con 0x7fcb8c106410 2024-09-10T10:35:13.801 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 msgr2=0x7fcb8c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 --2- 172.21.15.143:0/3677298821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c19b490 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009940 tx=0x7fcb7c032e10 comp rx=0 tx=0).stop 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 shutdown_connections 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 --2- 172.21.15.143:0/3677298821 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb8c106410 0x7fcb8c19b490 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 >> 172.21.15.143:0/3677298821 conn(0x7fcb8c101bc0 msgr2=0x7fcb8c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 shutdown_connections 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:13.297+0000 7fcb90abd640 1 -- 172.21.15.143:0/3677298821 wait complete. 2024-09-10T10:35:13.802 INFO:teuthology.orchestra.run.smithi143.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T10:35:13.817 INFO:teuthology.orchestra.run.smithi143.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:35:13.817 INFO:teuthology.orchestra.run.smithi143.stdout:Creating mgr... 2024-09-10T10:35:13.818 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T10:35:13.818 INFO:teuthology.orchestra.run.smithi143.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T10:35:14.067 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x 2024-09-10T10:35:14.067 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Failed to reset failed state of unit ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T10:35:14.215 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-34212b78-6f60-11ef-bcea-c7b262605968.target.wants/ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-34212b78-6f60-11ef-bcea-c7b262605968@.service. 2024-09-10T10:35:14.448 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:14 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3677298821' entity='client.admin' 2024-09-10T10:35:14.755 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:14 smithi143 podman[31270]: 2024-09-10 10:35:14.682247861 +0000 UTC m=+0.348159281 container start 9b44e769c0df4075eb1d02bba7bbb7ed5eafcf92bf45be5946c29cb4ce31f8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T10:35:14.755 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:14 smithi143 bash[31270]: 9b44e769c0df4075eb1d02bba7bbb7ed5eafcf92bf45be5946c29cb4ce31f8d5 2024-09-10T10:35:14.770 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:35:14.770 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-10T10:35:14.777 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:35:14.777 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-10T10:35:14.777 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-10T10:35:14.777 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T10:35:14.786 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:35:14.786 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-10T10:35:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:35:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-10T10:35:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-10T10:35:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T10:35:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr to start... 2024-09-10T10:35:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr... 2024-09-10T10:35:15.007 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:14 smithi143 systemd[1]: Started Ceph mgr.x for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:35:15.008 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:14.805+0000 7f7103970040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:35:15.008 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:14.864+0000 7f7103970040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:35:15.273 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.016+0000 7f7103970040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:35:15.273 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.084+0000 7f7103970040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:35:15.273 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.238+0000 7f7103970040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:35:15.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1816712794' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T10:35:15.551 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.389+0000 7f7103970040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:35:15.822 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:15.832 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:35:15.832 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:35:15.832 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T10:35:15.832 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T10:35:15.832 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T10:35:15.832 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 0 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T10:35:15.833 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T10:35:15.834 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T10:35:07:959271+0000", 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T10:35:15.835 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T10:35:07.959912+0000", 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:35:15.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.203+0000 7fbf38f1e640 1 Processor -- start 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.203+0000 7fbf38f1e640 1 -- start start 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.203+0000 7fbf38f1e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3410a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.203+0000 7fbf38f1e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbf340757b0 con 0x7fbf34107fc0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.203+0000 7fbf3259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3410a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.204+0000 7fbf3259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3410a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49086/0 (socket says 172.21.15.143:49086) 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.204+0000 7fbf3259c640 1 -- 172.21.15.143:0/2798191444 learned_addr learned my addr 172.21.15.143:0/2798191444 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.204+0000 7fbf3259c640 1 -- 172.21.15.143:0/2798191444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3410a8f0 con 0x7fbf34107fc0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.204+0000 7fbf3259c640 1 --2- 172.21.15.143:0/2798191444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3410a3b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009820 tx=0x7fbf2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31d5a464bb404a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf31d9b640 1 -- 172.21.15.143:0/2798191444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf24037070 con 0x7fbf34107fc0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf31d9b640 1 -- 172.21.15.143:0/2798191444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbf240328d0 con 0x7fbf34107fc0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf31d9b640 1 -- 172.21.15.143:0/2798191444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf240307d0 con 0x7fbf34107fc0 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 -- 172.21.15.143:0/2798191444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 msgr2=0x7fbf3410a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 --2- 172.21.15.143:0/2798191444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3410a3b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009820 tx=0x7fbf2402a270 comp rx=0 tx=0).stop 2024-09-10T10:35:15.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 -- 172.21.15.143:0/2798191444 shutdown_connections 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 --2- 172.21.15.143:0/2798191444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3410a3b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 -- 172.21.15.143:0/2798191444 >> 172.21.15.143:0/2798191444 conn(0x7fbf34101bc0 msgr2=0x7fbf34103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 -- 172.21.15.143:0/2798191444 shutdown_connections 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.205+0000 7fbf38f1e640 1 -- 172.21.15.143:0/2798191444 wait complete. 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf38f1e640 1 Processor -- start 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf38f1e640 1 -- start start 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf38f1e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf38f1e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3410b8d0 con 0x7fbf34107fc0 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf3259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf3259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49102/0 (socket says 172.21.15.143:49102) 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.206+0000 7fbf3259c640 1 -- 172.21.15.143:0/1816712794 learned_addr learned my addr 172.21.15.143:0/1816712794 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf3259c640 1 -- 172.21.15.143:0/1816712794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3419b990 con 0x7fbf34107fc0 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf3259c640 1 --2- 172.21.15.143:0/1816712794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3419b450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf24032e10 tx=0x7fbf24032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf24042070 con 0x7fbf34107fc0 2024-09-10T10:35:15.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbf24030c80 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf24037040 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3419bc20 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3419e910 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf24045070 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.207+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf24047a40 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.208+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf34108420 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.210+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbf2403c700 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.271+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbf3419eae0 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.272+0000 7fbf2b7fe640 1 -- 172.21.15.143:0/1816712794 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fbf2403c8e0 con 0x7fbf34107fc0 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 msgr2=0x7fbf3419b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 --2- 172.21.15.143:0/1816712794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3419b450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf24032e10 tx=0x7fbf24032e40 comp rx=0 tx=0).stop 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 shutdown_connections 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 --2- 172.21.15.143:0/1816712794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf34107fc0 0x7fbf3419b450 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:15.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 >> 172.21.15.143:0/1816712794 conn(0x7fbf34101bc0 msgr2=0x7fbf34102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:15.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 shutdown_connections 2024-09-10T10:35:15.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:15.273+0000 7fbf38f1e640 1 -- 172.21.15.143:0/1816712794 wait complete. 2024-09-10T10:35:15.840 INFO:teuthology.orchestra.run.smithi143.stdout:mgr not available, waiting (1/15)... 2024-09-10T10:35:15.956 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.709+0000 7f7103970040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:35:15.956 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.771+0000 7f7103970040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:35:16.300 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:15 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:15.955+0000 7f7103970040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:35:17.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:16 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:16.823+0000 7f7103970040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:35:17.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:16 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:16.889+0000 7f7103970040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:35:17.861 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:17.600+0000 7f7103970040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:35:17.861 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:17.714+0000 7f7103970040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:35:17.861 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:17.859+0000 7f7103970040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:35:18.146 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:17.923+0000 7f7103970040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:35:18.147 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.077+0000 7f7103970040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:35:18.147 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.145+0000 7f7103970040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:35:18.444 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1745893302' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T10:35:18.445 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.209+0000 7f7103970040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:35:18.445 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.376+0000 7f7103970040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:35:18.445 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.443+0000 7f7103970040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:35:18.719 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.510+0000 7f7103970040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:35:18.719 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.596+0000 7f7103970040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:35:18.719 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.657+0000 7f7103970040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:35:18.719 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:18.717+0000 7f7103970040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:35:18.930 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:18.930 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:35:18.930 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:35:18.930 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 0 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T10:35:18.931 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T10:35:18.932 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:18.933 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T10:35:07:959271+0000", 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.934 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T10:35:07.959912+0000", 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf15f17640 1 Processor -- start 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf15f17640 1 -- start start 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf15f17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf15f17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10106de0 con 0x7fdf10106410 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57806/0 (socket says 172.21.15.143:57806) 2024-09-10T10:35:18.935 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/1065422082 learned_addr learned my addr 172.21.15.143:0/1065422082 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.223+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/1065422082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf10107610 con 0x7fdf10106410 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.224+0000 7fdf0f7fe640 1 --2- 172.21.15.143:0/1065422082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10106810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009820 tx=0x7fdf0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a873934255c6110e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.224+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/1065422082 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf00037070 con 0x7fdf10106410 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.224+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/1065422082 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdf000328d0 con 0x7fdf10106410 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.224+0000 7fdf0e7fc640 1 -- 172.21.15.143:0/1065422082 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf000307d0 con 0x7fdf10106410 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 -- 172.21.15.143:0/1065422082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 msgr2=0x7fdf10106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 --2- 172.21.15.143:0/1065422082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10106810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009820 tx=0x7fdf0002a270 comp rx=0 tx=0).stop 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 -- 172.21.15.143:0/1065422082 shutdown_connections 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 --2- 172.21.15.143:0/1065422082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10106810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 -- 172.21.15.143:0/1065422082 >> 172.21.15.143:0/1065422082 conn(0x7fdf10101bc0 msgr2=0x7fdf10103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 -- 172.21.15.143:0/1065422082 shutdown_connections 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 -- 172.21.15.143:0/1065422082 wait complete. 2024-09-10T10:35:18.936 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.225+0000 7fdf15f17640 1 Processor -- start 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf15f17640 1 -- start start 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf15f17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf15f17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1010a9c0 con 0x7fdf10106410 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf0f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10075390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57812/0 (socket says 172.21.15.143:57812) 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/1745893302 learned_addr learned my addr 172.21.15.143:0/1745893302 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf0f7fe640 1 -- 172.21.15.143:0/1745893302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1019e1a0 con 0x7fdf10106410 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.226+0000 7fdf0f7fe640 1 --2- 172.21.15.143:0/1745893302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10075390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009940 tx=0x7fdf00032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf00042070 con 0x7fdf10106410 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdf00030c80 con 0x7fdf10106410 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1019e370 con 0x7fdf10106410 2024-09-10T10:35:18.937 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf00037040 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1019e710 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf00045070 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.227+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf00047a40 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.228+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf10106890 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.230+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdf0003c700 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.291+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdf1019eb50 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.291+0000 7fdf0cff9640 1 -- 172.21.15.143:0/1745893302 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fdf0003c8e0 con 0x7fdf10106410 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 msgr2=0x7fdf10075390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 --2- 172.21.15.143:0/1745893302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10075390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009940 tx=0x7fdf00032e10 comp rx=0 tx=0).stop 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 shutdown_connections 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 --2- 172.21.15.143:0/1745893302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf10106410 0x7fdf10075390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:18.938 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 >> 172.21.15.143:0/1745893302 conn(0x7fdf10101bc0 msgr2=0x7fdf10102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:18.939 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 shutdown_connections 2024-09-10T10:35:18.939 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:18.293+0000 7fdf15f17640 1 -- 172.21.15.143:0/1745893302 wait complete. 2024-09-10T10:35:18.939 INFO:teuthology.orchestra.run.smithi143.stdout:mgr not available, waiting (2/15)... 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:19.547+0000 7f7103970040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: Activating manager daemon x 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: mgrmap e2: x(active, starting, since 0.00285462s) 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:35:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: Manager daemon x is now available 2024-09-10T10:35:19.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:35:19.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:35:19.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' 2024-09-10T10:35:19.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' 2024-09-10T10:35:19.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:19 smithi143 ceph-mon[31057]: from='mgr.14100 172.21.15.143:0/1547911217' entity='mgr.x' 2024-09-10T10:35:21.577 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:21 smithi143 ceph-mon[31057]: mgrmap e3: x(active, since 1.00722s) 2024-09-10T10:35:22.136 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:22.136 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:35:22.136 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:35:22.136 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 0 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-10T10:35:22.137 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.138 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T10:35:07:959271+0000", 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T10:35:22.139 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T10:35:22.140 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T10:35:07.959912+0000", 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.324+0000 7fea6fc23640 1 Processor -- start 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.324+0000 7fea6fc23640 1 -- start start 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.325+0000 7fea6fc23640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6810a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.325+0000 7fea6fc23640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fea68075730 con 0x7fea68108040 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.325+0000 7fea6d9bf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6810a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.325+0000 7fea6d9bf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6810a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57886/0 (socket says 172.21.15.143:57886) 2024-09-10T10:35:22.141 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.325+0000 7fea6d9bf640 1 -- 172.21.15.143:0/106500603 learned_addr learned my addr 172.21.15.143:0/106500603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.325+0000 7fea6d9bf640 1 -- 172.21.15.143:0/106500603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6810a970 con 0x7fea68108040 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.326+0000 7fea6d9bf640 1 --2- 172.21.15.143:0/106500603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6810a430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea64009820 tx=0x7fea6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2b9fb49559a133 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.326+0000 7fea6c9bd640 1 -- 172.21.15.143:0/106500603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea64037070 con 0x7fea68108040 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.326+0000 7fea6c9bd640 1 -- 172.21.15.143:0/106500603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fea640328d0 con 0x7fea68108040 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.326+0000 7fea6c9bd640 1 -- 172.21.15.143:0/106500603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea640307d0 con 0x7fea68108040 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 -- 172.21.15.143:0/106500603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 msgr2=0x7fea6810a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 --2- 172.21.15.143:0/106500603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6810a430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea64009820 tx=0x7fea6402a270 comp rx=0 tx=0).stop 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 -- 172.21.15.143:0/106500603 shutdown_connections 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 --2- 172.21.15.143:0/106500603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6810a430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 -- 172.21.15.143:0/106500603 >> 172.21.15.143:0/106500603 conn(0x7fea68101bc0 msgr2=0x7fea68103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:22.142 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 -- 172.21.15.143:0/106500603 shutdown_connections 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 -- 172.21.15.143:0/106500603 wait complete. 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.327+0000 7fea6fc23640 1 Processor -- start 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6fc23640 1 -- start start 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6fc23640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6fc23640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fea6810b8f0 con 0x7fea68108040 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6d9bf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6d9bf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57888/0 (socket says 172.21.15.143:57888) 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6d9bf640 1 -- 172.21.15.143:0/4053863094 learned_addr learned my addr 172.21.15.143:0/4053863094 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6d9bf640 1 -- 172.21.15.143:0/4053863094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6819b980 con 0x7fea68108040 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6d9bf640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6819b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea64032e10 tx=0x7fea64032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea64042070 con 0x7fea68108040 2024-09-10T10:35:22.143 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fea64030c80 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6819bc10 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.328+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea64037040 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.329+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6819e900 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.329+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fea64045070 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.330+0000 7fea5effd640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7fea54038e10 0x7fea5403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.330+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fea6406bca0 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.330+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea681084a0 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.330+0000 7fea6d1be640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7fea54038e10 0x7fea5403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.330+0000 7fea6d1be640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7fea54038e10 0x7fea5403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fea580098c0 tx=0x7fea58006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.335+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea6404b170 con 0x7fea68108040 2024-09-10T10:35:22.144 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.555+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fea64031ad0 con 0x7fea68108040 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.565+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fea6819ee50 con 0x7fea68108040 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.565+0000 7fea5effd640 1 -- 172.21.15.143:0/4053863094 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fea64031dc0 con 0x7fea68108040 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7fea54038e10 msgr2=0x7fea5403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7fea54038e10 0x7fea5403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fea580098c0 tx=0x7fea58006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 msgr2=0x7fea6819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6819b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea64032e10 tx=0x7fea64032e40 comp rx=0 tx=0).stop 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 shutdown_connections 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7fea54038e10 0x7fea5403b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 --2- 172.21.15.143:0/4053863094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea68108040 0x7fea6819b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.567+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 >> 172.21.15.143:0/4053863094 conn(0x7fea68101bc0 msgr2=0x7fea68102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.568+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 shutdown_connections 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:21.568+0000 7fea6fc23640 1 -- 172.21.15.143:0/4053863094 wait complete. 2024-09-10T10:35:22.145 INFO:teuthology.orchestra.run.smithi143.stdout:mgr is available 2024-09-10T10:35:22.695 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:22 smithi143 ceph-mon[31057]: mgrmap e4: x(active, since 2s) 2024-09-10T10:35:22.695 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:22 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4053863094' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T10:35:24.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:23 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2504476364' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T10:35:24.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:24.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout fsid = 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.143:3300,v1:172.21.15.143:6789] 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T10:35:24.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.517+0000 7f026ed5c640 1 Processor -- start 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.517+0000 7f026ed5c640 1 -- start start 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.517+0000 7f026ed5c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026810a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:24.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.517+0000 7f026ed5c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f026807be50 con 0x7f0268107fb0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.518+0000 7f026caf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026810a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.518+0000 7f026caf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026810a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57894/0 (socket says 172.21.15.143:57894) 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.518+0000 7f026caf8640 1 -- 172.21.15.143:0/1040144571 learned_addr learned my addr 172.21.15.143:0/1040144571 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.518+0000 7f026caf8640 1 -- 172.21.15.143:0/1040144571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026810a8e0 con 0x7f0268107fb0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f026caf8640 1 --2- 172.21.15.143:0/1040144571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026810a3a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f025c009820 tx=0x7f025c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9355215026a9f3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f02677fe640 1 -- 172.21.15.143:0/1040144571 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c037070 con 0x7f0268107fb0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f02677fe640 1 -- 172.21.15.143:0/1040144571 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f025c0328d0 con 0x7f0268107fb0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f02677fe640 1 -- 172.21.15.143:0/1040144571 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c0307d0 con 0x7f0268107fb0 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f026ed5c640 1 -- 172.21.15.143:0/1040144571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 msgr2=0x7f026810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:24.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f026ed5c640 1 --2- 172.21.15.143:0/1040144571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026810a3a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f025c009820 tx=0x7f025c02a270 comp rx=0 tx=0).stop 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f026ed5c640 1 -- 172.21.15.143:0/1040144571 shutdown_connections 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f026ed5c640 1 --2- 172.21.15.143:0/1040144571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026810a3a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.519+0000 7f026ed5c640 1 -- 172.21.15.143:0/1040144571 >> 172.21.15.143:0/1040144571 conn(0x7f0268101bd0 msgr2=0x7f0268103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026ed5c640 1 -- 172.21.15.143:0/1040144571 shutdown_connections 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026ed5c640 1 -- 172.21.15.143:0/1040144571 wait complete. 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026ed5c640 1 Processor -- start 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026ed5c640 1 -- start start 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026ed5c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026ed5c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f026810b8c0 con 0x7f0268107fb0 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026caf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:24.352 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026caf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57906/0 (socket says 172.21.15.143:57906) 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.520+0000 7f026caf8640 1 -- 172.21.15.143:0/2504476364 learned_addr learned my addr 172.21.15.143:0/2504476364 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f026caf8640 1 -- 172.21.15.143:0/2504476364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026819b9a0 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f026caf8640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026819b460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f025c0097f0 tx=0x7f025c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c037040 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f025c032bb0 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026819bc30 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c03a390 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.521+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026819e920 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.522+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f025c03a5d0 con 0x7f0268107fb0 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.522+0000 7f0265ffb640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f0258038f20 0x7f025803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:24.353 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.522+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f025c06c240 con 0x7f0268107fb0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.522+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0268108410 con 0x7f0268107fb0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.522+0000 7f0267fff640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f0258038f20 0x7f025803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.523+0000 7f0267fff640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f0258038f20 0x7f025803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02540098c0 tx=0x7f0254006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.527+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f025c0313a0 con 0x7f0268107fb0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.693+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f026819fff0 con 0x7f0268107fb0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.694+0000 7f0265ffb640 1 -- 172.21.15.143:0/2504476364 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f025c042da0 con 0x7f0268107fb0 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f0258038f20 msgr2=0x7f025803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f0258038f20 0x7f025803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02540098c0 tx=0x7f0254006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 msgr2=0x7f026819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026819b460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f025c0097f0 tx=0x7f025c030d00 comp rx=0 tx=0).stop 2024-09-10T10:35:24.354 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 shutdown_connections 2024-09-10T10:35:24.355 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f0258038f20 0x7f025803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:24.355 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 --2- 172.21.15.143:0/2504476364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0268107fb0 0x7f026819b460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:24.355 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 >> 172.21.15.143:0/2504476364 conn(0x7f0268101bd0 msgr2=0x7f02681027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:24.355 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 shutdown_connections 2024-09-10T10:35:24.355 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:22.697+0000 7f026ed5c640 1 -- 172.21.15.143:0/2504476364 wait complete. 2024-09-10T10:35:24.355 INFO:teuthology.orchestra.run.smithi143.stdout:Enabling cephadm module... 2024-09-10T10:35:27.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:27 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2513470266' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T10:35:27.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: ignoring --setuser ceph since I am not root 2024-09-10T10:35:27.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: ignoring --setgroup ceph since I am not root 2024-09-10T10:35:27.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:27.733+0000 7f432638b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:35:27.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:27.789+0000 7f432638b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:35:28.291 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:27.936+0000 7f432638b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:35:28.291 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:28.001+0000 7f432638b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:35:28.291 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:28.146+0000 7f432638b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:35:28.291 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:28.288+0000 7f432638b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:35:29.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2513470266' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T10:35:29.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-mon[31057]: mgrmap e5: x(active, since 8s) 2024-09-10T10:35:29.051 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:28.600+0000 7f432638b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:35:29.051 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:28.661+0000 7f432638b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:35:29.051 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:28.845+0000 7f432638b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:35:29.828 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:29 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:29.752+0000 7f432638b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:35:29.828 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:29 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:29.823+0000 7f432638b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:35:30.825 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:30 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:30.565+0000 7f432638b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:35:30.825 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:30 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:30.681+0000 7f432638b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:35:31.078 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:30 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:30.822+0000 7f432638b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:35:31.078 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:30 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:30.884+0000 7f432638b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:35:31.078 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.032+0000 7f432638b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:35:31.385 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.101+0000 7f432638b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:35:31.385 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.161+0000 7f432638b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:35:31.385 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.318+0000 7f432638b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:35:31.385 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.383+0000 7f432638b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:35:31.651 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.449+0000 7f432638b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:35:31.651 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.531+0000 7f432638b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:35:31.652 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.590+0000 7f432638b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:35:31.652 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:31 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:31.649+0000 7f432638b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:35:32.800 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:32.438+0000 7f432638b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: Active manager daemon x restarted 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: Activating manager daemon x 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: mgrmap e6: x(active, starting, since 0.00329828s) 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:35:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: Manager daemon x is now available 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:35:32.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:32 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:35:33.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:33 smithi143 ceph-mon[31057]: Found migration_current of "None". Setting to last migration. 2024-09-10T10:35:33.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:33 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:33.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:33 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:33.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:33 smithi143 ceph-mon[31057]: mgrmap e7: x(active, since 1.00519s) 2024-09-10T10:35:33.910 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f7771d35640 1 Processor -- start 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f7771d35640 1 -- start start 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f7771d35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f7771d35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f776c106de0 con 0x7f776c106410 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45790/0 (socket says 172.21.15.143:45790) 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f776b7fe640 1 -- 172.21.15.143:0/934830469 learned_addr learned my addr 172.21.15.143:0/934830469 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.319+0000 7f776b7fe640 1 -- 172.21.15.143:0/934830469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c107610 con 0x7f776c106410 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.320+0000 7f776b7fe640 1 --2- 172.21.15.143:0/934830469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c106810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f775c009820 tx=0x7f775c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc4e94ef24f4013 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.320+0000 7f776a7fc640 1 -- 172.21.15.143:0/934830469 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c037070 con 0x7f776c106410 2024-09-10T10:35:33.911 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.320+0000 7f776a7fc640 1 -- 172.21.15.143:0/934830469 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f775c0328d0 con 0x7f776c106410 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.320+0000 7f776a7fc640 1 -- 172.21.15.143:0/934830469 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c0307d0 con 0x7f776c106410 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 -- 172.21.15.143:0/934830469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 msgr2=0x7f776c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 --2- 172.21.15.143:0/934830469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c106810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f775c009820 tx=0x7f775c02a270 comp rx=0 tx=0).stop 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 -- 172.21.15.143:0/934830469 shutdown_connections 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 --2- 172.21.15.143:0/934830469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c106810 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 -- 172.21.15.143:0/934830469 >> 172.21.15.143:0/934830469 conn(0x7f776c101bc0 msgr2=0x7f776c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 -- 172.21.15.143:0/934830469 shutdown_connections 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.321+0000 7f7771d35640 1 -- 172.21.15.143:0/934830469 wait complete. 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f7771d35640 1 Processor -- start 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f7771d35640 1 -- start start 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f7771d35640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f7771d35640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10a9c0 con 0x7f776c106410 2024-09-10T10:35:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c075390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45794/0 (socket says 172.21.15.143:45794) 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.322+0000 7f776b7fe640 1 -- 172.21.15.143:0/2513470266 learned_addr learned my addr 172.21.15.143:0/2513470266 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f776b7fe640 1 -- 172.21.15.143:0/2513470266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c19e1a0 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f776b7fe640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c075390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f775c0097f0 tx=0x7f775c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c037040 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f775c032bb0 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c19e370 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f775c03a390 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.323+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c19e710 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.324+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f775c03b3f0 con 0x7f776c106410 2024-09-10T10:35:33.913 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.324+0000 7f7768ff9640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f7754038f20 0x7f775403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:33.915 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.324+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c106890 con 0x7f776c106410 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.324+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f775c06c090 con 0x7f776c106410 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.324+0000 7f776affd640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f7754038f20 0x7f775403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.325+0000 7f776affd640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f7754038f20 0x7f775403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77600098c0 tx=0x7f7760006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.329+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f775c040210 con 0x7f776c106410 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.543+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f776c19eb70 con 0x7f776c106410 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.594+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f775c038d40 con 0x7f776c106410 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.596+0000 7f7768ff9640 1 -- 172.21.15.143:0/2513470266 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f775c03b6e0 con 0x7f776c106410 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f7754038f20 msgr2=0x7f775403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f7754038f20 0x7f775403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77600098c0 tx=0x7f7760006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 msgr2=0x7f776c075390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c075390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f775c0097f0 tx=0x7f775c030d00 comp rx=0 tx=0).stop 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 shutdown_connections 2024-09-10T10:35:33.916 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:6800/3171173333,v1:172.21.15.143:6801/3171173333] conn(0x7f7754038f20 0x7f775403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:33.917 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 --2- 172.21.15.143:0/2513470266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c106410 0x7f776c075390 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:33.917 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.597+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 >> 172.21.15.143:0/2513470266 conn(0x7f776c101bc0 msgr2=0x7f776c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:33.917 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.598+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 shutdown_connections 2024-09-10T10:35:33.917 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:27.598+0000 7f7771d35640 1 -- 172.21.15.143:0/2513470266 wait complete. 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:34.344+0000 7f428d6e5640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:10:35:34] ENGINE Error in HTTPServer.serve 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: Traceback (most recent call last): 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self._connections.run(self.expiration_interval) 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self._run(expiration_interval) 2024-09-10T10:35:34.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: s = self.context.wrap_socket( 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: return self.sslsocket_class._create( 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self.do_handshake() 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self._sslobj.do_handshake() 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:35:34.802 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:35 smithi143 ceph-mon[31057]: [10/Sep/2024:10:35:34] ENGINE Bus STARTING 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:35 smithi143 ceph-mon[31057]: [10/Sep/2024:10:35:34] ENGINE Serving on http://172.21.15.143:8765 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:35 smithi143 ceph-mon[31057]: [10/Sep/2024:10:35:34] ENGINE Serving on https://172.21.15.143:7150 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:35 smithi143 ceph-mon[31057]: [10/Sep/2024:10:35:34] ENGINE Bus STARTED 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:35 smithi143 ceph-mon[31057]: [10/Sep/2024:10:35:34] ENGINE Error in HTTPServer.serve 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: Traceback (most recent call last): 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:35:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._run(expiration_interval) 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: s = self.context.wrap_socket( 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: return self.sslsocket_class._create( 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: self.do_handshake() 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:35:35.802 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._sslobj.do_handshake() 2024-09-10T10:35:35.803 INFO:journalctl@ceph.mon.a.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:35:35.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:35 smithi143 ceph-mon[31057]: mgrmap e8: x(active, since 2s) 2024-09-10T10:35:37.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:36 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1812603746' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T10:35:37.098 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:35:37.098 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-10T10:35:37.098 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T10:35:37.098 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T10:35:37.098 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.332+0000 7f3aef0a3640 1 Processor -- start 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.332+0000 7f3aef0a3640 1 -- start start 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.332+0000 7f3aef0a3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae81078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.332+0000 7f3aef0a3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8107e90 con 0x7f3ae81074c0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.333+0000 7f3aece3f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae81078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.333+0000 7f3aece3f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae81078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45892/0 (socket says 172.21.15.143:45892) 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.333+0000 7f3aece3f640 1 -- 172.21.15.143:0/940043727 learned_addr learned my addr 172.21.15.143:0/940043727 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.333+0000 7f3aece3f640 1 -- 172.21.15.143:0/940043727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae81086c0 con 0x7f3ae81074c0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.333+0000 7f3aece3f640 1 --2- 172.21.15.143:0/940043727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae81078c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009820 tx=0x7f3adc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f018de63a100fbad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3ae77fe640 1 -- 172.21.15.143:0/940043727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3adc037070 con 0x7f3ae81074c0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3ae77fe640 1 -- 172.21.15.143:0/940043727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3adc0328d0 con 0x7f3ae81074c0 2024-09-10T10:35:37.099 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3ae77fe640 1 -- 172.21.15.143:0/940043727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3adc0307d0 con 0x7f3ae81074c0 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 -- 172.21.15.143:0/940043727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 msgr2=0x7f3ae81078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 --2- 172.21.15.143:0/940043727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae81078c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009820 tx=0x7f3adc02a270 comp rx=0 tx=0).stop 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 -- 172.21.15.143:0/940043727 shutdown_connections 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 --2- 172.21.15.143:0/940043727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae81078c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 -- 172.21.15.143:0/940043727 >> 172.21.15.143:0/940043727 conn(0x7f3ae8102c50 msgr2=0x7f3ae8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 -- 172.21.15.143:0/940043727 shutdown_connections 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.334+0000 7f3aef0a3640 1 -- 172.21.15.143:0/940043727 wait complete. 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aef0a3640 1 Processor -- start 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aef0a3640 1 -- start start 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aef0a3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aef0a3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810ba70 con 0x7f3ae81074c0 2024-09-10T10:35:37.100 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aece3f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aece3f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45902/0 (socket says 172.21.15.143:45902) 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.335+0000 7f3aece3f640 1 -- 172.21.15.143:0/1812603746 learned_addr learned my addr 172.21.15.143:0/1812603746 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3aece3f640 1 -- 172.21.15.143:0/1812603746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae819b9c0 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3aece3f640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae819b480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009940 tx=0x7f3adc032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3adc042070 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3adc030c80 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae819bc50 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3adc037040 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.336+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae819e940 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.337+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3adc045070 con 0x7f3ae81074c0 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.337+0000 7f3ae5ffb640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3ad8038fc0 0x7f3ad803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:37.101 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.337+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3adc06bff0 con 0x7f3ae81074c0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.337+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae810bbf0 con 0x7f3ae81074c0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.337+0000 7f3ae7fff640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3ad8038fc0 0x7f3ad803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.338+0000 7f3ae7fff640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3ad8038fc0 0x7f3ad803b480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40098c0 tx=0x7f3ad4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.343+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3adc046da0 con 0x7f3ae81074c0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.559+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3ae819eb10 con 0x7f3ae81074c0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.559+0000 7f3ae5ffb640 1 -- 172.21.15.143:0/1812603746 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v8) ==== 56+0+88 (secure 0 0 0) 0x7f3adc03b330 con 0x7f3ae81074c0 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3ad8038fc0 msgr2=0x7f3ad803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3ad8038fc0 0x7f3ad803b480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40098c0 tx=0x7f3ad4006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 msgr2=0x7f3ae819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae819b480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009940 tx=0x7f3adc032e10 comp rx=0 tx=0).stop 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 shutdown_connections 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3ad8038fc0 0x7f3ad803b480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:37.102 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 --2- 172.21.15.143:0/1812603746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ae81074c0 0x7f3ae819b480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:37.103 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.561+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 >> 172.21.15.143:0/1812603746 conn(0x7f3ae8102c50 msgr2=0x7f3ae8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:37.103 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.562+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 shutdown_connections 2024-09-10T10:35:37.103 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:36.562+0000 7f3aef0a3640 1 -- 172.21.15.143:0/1812603746 wait complete. 2024-09-10T10:35:37.103 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for the mgr to restart... 2024-09-10T10:35:37.103 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr epoch 8... 2024-09-10T10:35:38.037 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:37 smithi143 ceph-mon[31057]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T10:35:38.037 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:37 smithi143 ceph-mon[31057]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T10:35:38.045 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:35:38.045 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-10T10:35:38.045 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T10:35:38.045 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d7020640 1 Processor -- start 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d7020640 1 -- start start 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d7020640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d0107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d7020640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0107ed0 con 0x7f97d0107500 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d4dbc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d0107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d4dbc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d0107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35572/0 (socket says 172.21.15.143:35572) 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d4dbc640 1 -- 172.21.15.143:0/235773740 learned_addr learned my addr 172.21.15.143:0/235773740 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.488+0000 7f97d4dbc640 1 -- 172.21.15.143:0/235773740 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d0108700 con 0x7f97d0107500 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.489+0000 7f97d4dbc640 1 --2- 172.21.15.143:0/235773740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d0107900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009820 tx=0x7f97c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808f1bc364951ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.489+0000 7f97cf7fe640 1 -- 172.21.15.143:0/235773740 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97c4037070 con 0x7f97d0107500 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.489+0000 7f97cf7fe640 1 -- 172.21.15.143:0/235773740 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f97c40328d0 con 0x7f97d0107500 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.489+0000 7f97cf7fe640 1 -- 172.21.15.143:0/235773740 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97c40307d0 con 0x7f97d0107500 2024-09-10T10:35:38.046 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 -- 172.21.15.143:0/235773740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 msgr2=0x7f97d0107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 --2- 172.21.15.143:0/235773740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d0107900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009820 tx=0x7f97c402a270 comp rx=0 tx=0).stop 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 -- 172.21.15.143:0/235773740 shutdown_connections 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 --2- 172.21.15.143:0/235773740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d0107900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 -- 172.21.15.143:0/235773740 >> 172.21.15.143:0/235773740 conn(0x7f97d0102c70 msgr2=0x7f97d01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 -- 172.21.15.143:0/235773740 shutdown_connections 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 -- 172.21.15.143:0/235773740 wait complete. 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.490+0000 7f97d7020640 1 Processor -- start 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d7020640 1 -- start start 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d7020640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d7020640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f97d010bab0 con 0x7f97d0107500 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d4dbc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d4dbc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35578/0 (socket says 172.21.15.143:35578) 2024-09-10T10:35:38.047 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d4dbc640 1 -- 172.21.15.143:0/4294540773 learned_addr learned my addr 172.21.15.143:0/4294540773 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d4dbc640 1 -- 172.21.15.143:0/4294540773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d007e090 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d4dbc640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d007db50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009940 tx=0x7f97c4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97c4042070 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f97c4030c80 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d007c280 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.491+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97c4037040 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.492+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d007c7c0 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.493+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f97c4045070 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.493+0000 7f97cdffb640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f97c0038f70 0x7f97c003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.493+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f97c003bb40 con 0x7f97c0038f70 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.493+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f97c406bf30 con 0x7f97d0107500 2024-09-10T10:35:38.048 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.493+0000 7f97cffff640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f97c0038f70 0x7f97c003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.493+0000 7f97cffff640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f97c0038f70 0x7f97c003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098c0 tx=0x7f97bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.494+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f97c003bb40 con 0x7f97c0038f70 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.500+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f97d010bc30 con 0x7f97c0038f70 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.500+0000 7f97cdffb640 1 -- 172.21.15.143:0/4294540773 <== mgr.14118 v2:172.21.15.143:6800/1050213403 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f97d010bc30 con 0x7f97c0038f70 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f97c0038f70 msgr2=0x7f97c003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f97c0038f70 0x7f97c003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098c0 tx=0x7f97bc006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 msgr2=0x7f97d007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d007db50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009940 tx=0x7f97c4032e10 comp rx=0 tx=0).stop 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 shutdown_connections 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f97c0038f70 0x7f97c003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 --2- 172.21.15.143:0/4294540773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0107500 0x7f97d007db50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:38.049 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 >> 172.21.15.143:0/4294540773 conn(0x7f97d0102c70 msgr2=0x7f97d01037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:38.050 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 shutdown_connections 2024-09-10T10:35:38.050 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:37.501+0000 7f97d7020640 1 -- 172.21.15.143:0/4294540773 wait complete. 2024-09-10T10:35:38.050 INFO:teuthology.orchestra.run.smithi143.stdout:mgr epoch 8 is available 2024-09-10T10:35:38.050 INFO:teuthology.orchestra.run.smithi143.stdout:Setting orchestrator backend to cephadm... 2024-09-10T10:35:39.176 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa734612640 1 Processor -- start 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa734612640 1 -- start start 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa734612640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa734612640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c07c890 con 0x7fa72c07de60 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa7323ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa7323ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35594/0 (socket says 172.21.15.143:35594) 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.416+0000 7fa7323ae640 1 -- 172.21.15.143:0/1277818633 learned_addr learned my addr 172.21.15.143:0/1277818633 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.417+0000 7fa7323ae640 1 -- 172.21.15.143:0/1277818633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c07ca10 con 0x7fa72c07de60 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.417+0000 7fa7323ae640 1 --2- 172.21.15.143:0/1277818633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c07c2c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e80 tx=0x7fa72402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed842c1b4ae0a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.417+0000 7fa7313ac640 1 -- 172.21.15.143:0/1277818633 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa724036070 con 0x7fa72c07de60 2024-09-10T10:35:39.177 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.417+0000 7fa7313ac640 1 -- 172.21.15.143:0/1277818633 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa724031890 con 0x7fa72c07de60 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.417+0000 7fa7313ac640 1 -- 172.21.15.143:0/1277818633 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa724007980 con 0x7fa72c07de60 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 -- 172.21.15.143:0/1277818633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 msgr2=0x7fa72c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 --2- 172.21.15.143:0/1277818633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c07c2c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e80 tx=0x7fa72402a7a0 comp rx=0 tx=0).stop 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 -- 172.21.15.143:0/1277818633 shutdown_connections 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 --2- 172.21.15.143:0/1277818633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c07c2c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 -- 172.21.15.143:0/1277818633 >> 172.21.15.143:0/1277818633 conn(0x7fa72c102c50 msgr2=0x7fa72c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 -- 172.21.15.143:0/1277818633 shutdown_connections 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.418+0000 7fa734612640 1 -- 172.21.15.143:0/1277818633 wait complete. 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa734612640 1 Processor -- start 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa734612640 1 -- start start 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa734612640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa734612640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10da90 con 0x7fa72c07de60 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa7323ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa7323ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c19d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35600/0 (socket says 172.21.15.143:35600) 2024-09-10T10:35:39.178 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa7323ae640 1 -- 172.21.15.143:0/3351582219 learned_addr learned my addr 172.21.15.143:0/3351582219 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa7323ae640 1 -- 172.21.15.143:0/3351582219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c19db60 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa7323ae640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c19d620 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa724009680 tx=0x7fa724031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.419+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa724041020 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.420+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c19ddf0 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.420+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c1a0ae0 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.420+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa72403c900 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.420+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa724036040 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.420+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa724044020 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.421+0000 7fa7237fe640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa718038f70 0x7fa71803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.421+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7100052c0 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.421+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa72406d450 con 0x7fa72c07de60 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.421+0000 7fa731bad640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa718038f70 0x7fa71803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:39.179 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.421+0000 7fa731bad640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa718038f70 0x7fa71803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0098c0 tx=0x7fa71c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.425+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa72404b020 con 0x7fa72c07de60 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.607+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa710000bf0 con 0x7fa718038f70 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.612+0000 7fa7237fe640 1 -- 172.21.15.143:0/3351582219 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa710000bf0 con 0x7fa718038f70 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.614+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa718038f70 msgr2=0x7fa71803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.614+0000 7fa734612640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa718038f70 0x7fa71803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0098c0 tx=0x7fa71c006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.614+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 msgr2=0x7fa72c19d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.614+0000 7fa734612640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c19d620 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa724009680 tx=0x7fa724031c70 comp rx=0 tx=0).stop 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.615+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 shutdown_connections 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.615+0000 7fa734612640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa718038f70 0x7fa71803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.615+0000 7fa734612640 1 --2- 172.21.15.143:0/3351582219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa72c07de60 0x7fa72c19d620 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.615+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 >> 172.21.15.143:0/3351582219 conn(0x7fa72c102c50 msgr2=0x7fa72c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.615+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 shutdown_connections 2024-09-10T10:35:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:38.615+0000 7fa734612640 1 -- 172.21.15.143:0/3351582219 wait complete. 2024-09-10T10:35:39.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:39 smithi143 ceph-mon[31057]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:39.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:39 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:39.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:39 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:35:40.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T10:35:40.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.558+0000 7f3b5556d640 1 Processor -- start 2024-09-10T10:35:40.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.558+0000 7f3b5556d640 1 -- start start 2024-09-10T10:35:40.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.559+0000 7f3b5556d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b50107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:40.281 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.559+0000 7f3b5556d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50107ed0 con 0x7f3b50107500 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.559+0000 7f3b4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b50107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.559+0000 7f3b4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b50107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35608/0 (socket says 172.21.15.143:35608) 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.559+0000 7f3b4effd640 1 -- 172.21.15.143:0/1562397238 learned_addr learned my addr 172.21.15.143:0/1562397238 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.559+0000 7f3b4effd640 1 -- 172.21.15.143:0/1562397238 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b50108700 con 0x7f3b50107500 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b4effd640 1 --2- 172.21.15.143:0/1562397238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b50107900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009e80 tx=0x7f3b4002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e37a1ff13fcae22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b4dffb640 1 -- 172.21.15.143:0/1562397238 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40036070 con 0x7f3b50107500 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b4dffb640 1 -- 172.21.15.143:0/1562397238 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3b40031890 con 0x7f3b50107500 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b4dffb640 1 -- 172.21.15.143:0/1562397238 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40007980 con 0x7f3b50107500 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 -- 172.21.15.143:0/1562397238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 msgr2=0x7f3b50107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 --2- 172.21.15.143:0/1562397238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b50107900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009e80 tx=0x7f3b4002a7a0 comp rx=0 tx=0).stop 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 -- 172.21.15.143:0/1562397238 shutdown_connections 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 --2- 172.21.15.143:0/1562397238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b50107900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:40.282 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 -- 172.21.15.143:0/1562397238 >> 172.21.15.143:0/1562397238 conn(0x7f3b50102c70 msgr2=0x7f3b501050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 -- 172.21.15.143:0/1562397238 shutdown_connections 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.560+0000 7f3b5556d640 1 -- 172.21.15.143:0/1562397238 wait complete. 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b5556d640 1 Processor -- start 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b5556d640 1 -- start start 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b5556d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b5019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b5556d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5010bab0 con 0x7f3b50107500 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b5019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b5019b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35618/0 (socket says 172.21.15.143:35618) 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.561+0000 7f3b4effd640 1 -- 172.21.15.143:0/2322589849 learned_addr learned my addr 172.21.15.143:0/2322589849 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b4effd640 1 -- 172.21.15.143:0/2322589849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5019ba00 con 0x7f3b50107500 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b4effd640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b5019b4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b4002acc0 tx=0x7f3b40031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40041030 con 0x7f3b50107500 2024-09-10T10:35:40.283 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3b4003c900 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5019bc90 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b40036040 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.562+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5019e980 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.563+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3b40044070 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.563+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b5010bc30 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.563+0000 7f3b47fff640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3b3c038f70 0x7f3b3c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.563+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b40042070 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.563+0000 7f3b4e7fc640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3b3c038f70 0x7f3b3c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.564+0000 7f3b4e7fc640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3b3c038f70 0x7f3b3c03b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b380098c0 tx=0x7f3b38006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.568+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b4004b070 con 0x7f3b50107500 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.736+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3b50103ee0 con 0x7f3b3c038f70 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.737+0000 7f3b47fff640 1 -- 172.21.15.143:0/2322589849 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3b50103ee0 con 0x7f3b3c038f70 2024-09-10T10:35:40.284 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3b3c038f70 msgr2=0x7f3b3c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3b3c038f70 0x7f3b3c03b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b380098c0 tx=0x7f3b38006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 msgr2=0x7f3b5019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b5019b4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3b4002acc0 tx=0x7f3b40031c70 comp rx=0 tx=0).stop 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 shutdown_connections 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f3b3c038f70 0x7f3b3c03b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 --2- 172.21.15.143:0/2322589849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b50107500 0x7f3b5019b4c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.739+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 >> 172.21.15.143:0/2322589849 conn(0x7f3b50102c70 msgr2=0x7f3b501037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.740+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 shutdown_connections 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:39.740+0000 7f3b5556d640 1 -- 172.21.15.143:0/2322589849 wait complete. 2024-09-10T10:35:40.285 INFO:teuthology.orchestra.run.smithi143.stdout:Generating ssh key... 2024-09-10T10:35:40.991 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-mon[31057]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:41.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: Generating public/private rsa key pair. 2024-09-10T10:35:41.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: Your identification has been saved in /tmp/tmp9gq1tiez/key 2024-09-10T10:35:41.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: Your public key has been saved in /tmp/tmp9gq1tiez/key.pub 2024-09-10T10:35:41.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: The key fingerprint is: 2024-09-10T10:35:41.301 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: SHA256:0jcsZFNefxSdojQSvjAEUDmlOJtwgUFsWt1xUcCJerU ceph-34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: The key's randomart image is: 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: +---[RSA 3072]----+ 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: |ooo+o+=*+*+ . .=| 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: | +. +o=.=+ + o o.| 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: |oo + o.++o+ o o .| 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: |. o = .+Eo.. . | 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: | o .. S.+ | 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: | . o . | 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: | | 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: | | 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: | | 2024-09-10T10:35:41.302 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: +----[SHA256]-----+ 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.690+0000 7f515c75b640 1 Processor -- start 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.690+0000 7f515c75b640 1 -- start start 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.690+0000 7f515c75b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f51541061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.690+0000 7f515c75b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5154075770 con 0x7f5154105df0 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.691+0000 7f515a4f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f51541061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.691+0000 7f515a4f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f51541061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35630/0 (socket says 172.21.15.143:35630) 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.691+0000 7f515a4f7640 1 -- 172.21.15.143:0/1488693552 learned_addr learned my addr 172.21.15.143:0/1488693552 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.691+0000 7f515a4f7640 1 -- 172.21.15.143:0/1488693552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5154106730 con 0x7f5154105df0 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515a4f7640 1 --2- 172.21.15.143:0/1488693552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f51541061f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5144009e80 tx=0x7f514402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f8cee9a9ceffea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:41.573 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f51594f5640 1 -- 172.21.15.143:0/1488693552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5144036070 con 0x7f5154105df0 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f51594f5640 1 -- 172.21.15.143:0/1488693552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5144031890 con 0x7f5154105df0 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f51594f5640 1 -- 172.21.15.143:0/1488693552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5144007980 con 0x7f5154105df0 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 -- 172.21.15.143:0/1488693552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 msgr2=0x7f51541061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 --2- 172.21.15.143:0/1488693552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f51541061f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5144009e80 tx=0x7f514402a7a0 comp rx=0 tx=0).stop 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 -- 172.21.15.143:0/1488693552 shutdown_connections 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 --2- 172.21.15.143:0/1488693552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f51541061f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 -- 172.21.15.143:0/1488693552 >> 172.21.15.143:0/1488693552 conn(0x7f5154101bc0 msgr2=0x7f5154103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 -- 172.21.15.143:0/1488693552 shutdown_connections 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.692+0000 7f515c75b640 1 -- 172.21.15.143:0/1488693552 wait complete. 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515c75b640 1 Processor -- start 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515c75b640 1 -- start start 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515c75b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f515419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515c75b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51541077d0 con 0x7f5154105df0 2024-09-10T10:35:41.574 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515a4f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f515419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515a4f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f515419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35640/0 (socket says 172.21.15.143:35640) 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515a4f7640 1 -- 172.21.15.143:0/1244675222 learned_addr learned my addr 172.21.15.143:0/1244675222 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.693+0000 7f515a4f7640 1 -- 172.21.15.143:0/1244675222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515419db90 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.694+0000 7f515a4f7640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f515419d650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5144009680 tx=0x7f5144031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.694+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5144041020 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.694+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f514403c900 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.694+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5144036040 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.694+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515419de20 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.694+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541a0b10 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.695+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5144044020 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.695+0000 7f51537fe640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f513c038fc0 0x7f513c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.695+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f514406d480 con 0x7f5154105df0 2024-09-10T10:35:41.575 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.695+0000 7f5159cf6640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f513c038fc0 0x7f513c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.695+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51541061f0 con 0x7f5154105df0 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.695+0000 7f5159cf6640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f513c038fc0 0x7f513c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51480098c0 tx=0x7f5148006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.701+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f514404b020 con 0x7f5154105df0 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.870+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f515402d970 con 0x7f513c038fc0 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.996+0000 7f51537fe640 1 -- 172.21.15.143:0/1244675222 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f515402d970 con 0x7f513c038fc0 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f513c038fc0 msgr2=0x7f513c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f513c038fc0 0x7f513c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51480098c0 tx=0x7f5148006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 msgr2=0x7f515419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f515419d650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5144009680 tx=0x7f5144031c70 comp rx=0 tx=0).stop 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 shutdown_connections 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f513c038fc0 0x7f513c03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 --2- 172.21.15.143:0/1244675222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5154105df0 0x7f515419d650 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:41.576 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 >> 172.21.15.143:0/1244675222 conn(0x7f5154101bc0 msgr2=0x7f5154194350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:41.577 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 shutdown_connections 2024-09-10T10:35:41.577 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:40.998+0000 7f515c75b640 1 -- 172.21.15.143:0/1244675222 wait complete. 2024-09-10T10:35:42.139 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:41 smithi143 ceph-mon[31057]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:42.139 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:41 smithi143 ceph-mon[31057]: Generating ssh key... 2024-09-10T10:35:42.140 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:41 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:42.140 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:41 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:42.741 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:42.741 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff715cf2640 1 Processor -- start 2024-09-10T10:35:42.741 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff715cf2640 1 -- start start 2024-09-10T10:35:42.742 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff715cf2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff7101078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:42.742 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff715cf2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff710107e90 con 0x7ff7101074c0 2024-09-10T10:35:42.742 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff7101078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:42.742 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff7101078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35648/0 (socket says 172.21.15.143:35648) 2024-09-10T10:35:42.742 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.960+0000 7ff70f7fe640 1 -- 172.21.15.143:0/266803389 learned_addr learned my addr 172.21.15.143:0/266803389 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.961+0000 7ff70f7fe640 1 -- 172.21.15.143:0/266803389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7101086c0 con 0x7ff7101074c0 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.961+0000 7ff70f7fe640 1 --2- 172.21.15.143:0/266803389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff7101078c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff700009820 tx=0x7ff70002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9e5064bce801ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.961+0000 7ff70e7fc640 1 -- 172.21.15.143:0/266803389 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff700037070 con 0x7ff7101074c0 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.961+0000 7ff70e7fc640 1 -- 172.21.15.143:0/266803389 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff7000328d0 con 0x7ff7101074c0 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff70e7fc640 1 -- 172.21.15.143:0/266803389 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7000307d0 con 0x7ff7101074c0 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 -- 172.21.15.143:0/266803389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 msgr2=0x7ff7101078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:42.743 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 --2- 172.21.15.143:0/266803389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff7101078c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff700009820 tx=0x7ff70002a270 comp rx=0 tx=0).stop 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 -- 172.21.15.143:0/266803389 shutdown_connections 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 --2- 172.21.15.143:0/266803389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff7101078c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 -- 172.21.15.143:0/266803389 >> 172.21.15.143:0/266803389 conn(0x7ff710102c50 msgr2=0x7ff710105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 -- 172.21.15.143:0/266803389 shutdown_connections 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.962+0000 7ff715cf2640 1 -- 172.21.15.143:0/266803389 wait complete. 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff715cf2640 1 Processor -- start 2024-09-10T10:35:42.744 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff715cf2640 1 -- start start 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff715cf2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff71019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff715cf2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff71010ba70 con 0x7ff7101074c0 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff71019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff71019b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35650/0 (socket says 172.21.15.143:35650) 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff70f7fe640 1 -- 172.21.15.143:0/910962274 learned_addr learned my addr 172.21.15.143:0/910962274 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.963+0000 7ff70f7fe640 1 -- 172.21.15.143:0/910962274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71019b9c0 con 0x7ff7101074c0 2024-09-10T10:35:42.745 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.964+0000 7ff70f7fe640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff71019b480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff700009940 tx=0x7ff700032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.964+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff700042070 con 0x7ff7101074c0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.964+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff700030c80 con 0x7ff7101074c0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.964+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71019bc50 con 0x7ff7101074c0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.964+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff700037040 con 0x7ff7101074c0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.964+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71019e940 con 0x7ff7101074c0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.965+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff700045070 con 0x7ff7101074c0 2024-09-10T10:35:42.746 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.965+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71010bbf0 con 0x7ff7101074c0 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.965+0000 7ff70cff9640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ff6f8038fc0 0x7ff6f803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.965+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff70006bff0 con 0x7ff7101074c0 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.965+0000 7ff70effd640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ff6f8038fc0 0x7ff6f803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.965+0000 7ff70effd640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ff6f8038fc0 0x7ff6f803b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7040098c0 tx=0x7ff704006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:41.970+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff70003ac60 con 0x7ff7101074c0 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.138+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7ff710103e60 con 0x7ff6f8038fc0 2024-09-10T10:35:42.747 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.139+0000 7ff70cff9640 1 -- 172.21.15.143:0/910962274 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7ff710103e60 con 0x7ff6f8038fc0 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ff6f8038fc0 msgr2=0x7ff6f803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ff6f8038fc0 0x7ff6f803b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7040098c0 tx=0x7ff704006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 msgr2=0x7ff71019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff71019b480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff700009940 tx=0x7ff700032e10 comp rx=0 tx=0).stop 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 shutdown_connections 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ff6f8038fc0 0x7ff6f803b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:42.748 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 --2- 172.21.15.143:0/910962274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7101074c0 0x7ff71019b480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:42.749 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 >> 172.21.15.143:0/910962274 conn(0x7ff710102c50 msgr2=0x7ff710103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:42.749 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 shutdown_connections 2024-09-10T10:35:42.749 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:42.141+0000 7ff715cf2640 1 -- 172.21.15.143:0/910962274 wait complete. 2024-09-10T10:35:42.749 INFO:teuthology.orchestra.run.smithi143.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:35:42.749 INFO:teuthology.orchestra.run.smithi143.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T10:35:42.750 INFO:teuthology.orchestra.run.smithi143.stdout:Adding host smithi143... 2024-09-10T10:35:43.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:42 smithi143 ceph-mon[31057]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:44.095 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:43 smithi143 ceph-mon[31057]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "addr": "172.21.15.143", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:45.024 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:44 smithi143 ceph-mon[31057]: Deploying cephadm binary to smithi143 2024-09-10T10:35:47.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Added host 'smithi143' with addr '172.21.15.143' 2024-09-10T10:35:47.197 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.140+0000 7ffade055640 1 Processor -- start 2024-09-10T10:35:47.197 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffade055640 1 -- start start 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffade055640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad810a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffade055640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffad80757f0 con 0x7ffad8107f80 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffad77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad810a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffad77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad810a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35662/0 (socket says 172.21.15.143:35662) 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffad77fe640 1 -- 172.21.15.143:0/3970337463 learned_addr learned my addr 172.21.15.143:0/3970337463 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.141+0000 7ffad77fe640 1 -- 172.21.15.143:0/3970337463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad810a8b0 con 0x7ffad8107f80 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.142+0000 7ffad77fe640 1 --2- 172.21.15.143:0/3970337463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad810a370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009820 tx=0x7ffac802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cdfdd5627058a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.142+0000 7ffad6ffd640 1 -- 172.21.15.143:0/3970337463 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac8037070 con 0x7ffad8107f80 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.142+0000 7ffad6ffd640 1 -- 172.21.15.143:0/3970337463 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffac80328d0 con 0x7ffad8107f80 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.142+0000 7ffad6ffd640 1 -- 172.21.15.143:0/3970337463 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac80307d0 con 0x7ffad8107f80 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 -- 172.21.15.143:0/3970337463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 msgr2=0x7ffad810a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 --2- 172.21.15.143:0/3970337463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad810a370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009820 tx=0x7ffac802a270 comp rx=0 tx=0).stop 2024-09-10T10:35:47.198 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 -- 172.21.15.143:0/3970337463 shutdown_connections 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 --2- 172.21.15.143:0/3970337463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad810a370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 -- 172.21.15.143:0/3970337463 >> 172.21.15.143:0/3970337463 conn(0x7ffad8101bc0 msgr2=0x7ffad8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 -- 172.21.15.143:0/3970337463 shutdown_connections 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 -- 172.21.15.143:0/3970337463 wait complete. 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 Processor -- start 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.143+0000 7ffade055640 1 -- start start 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffade055640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffade055640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffad810b8f0 con 0x7ffad8107f80 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35676/0 (socket says 172.21.15.143:35676) 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad77fe640 1 -- 172.21.15.143:0/2952041198 learned_addr learned my addr 172.21.15.143:0/2952041198 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad77fe640 1 -- 172.21.15.143:0/2952041198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad819b980 con 0x7ffad8107f80 2024-09-10T10:35:47.199 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad77fe640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad819b440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffac80097f0 tx=0x7ffac8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac8037040 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffac8032bb0 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.144+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad819bc10 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.145+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac803a390 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.145+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad819e900 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.146+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ffac803a5f0 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.146+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad81083e0 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.146+0000 7ffad57fa640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffac4038f70 0x7ffac403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.146+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffac806c3a0 con 0x7ffad8107f80 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.146+0000 7ffaceffd640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffac4038f70 0x7ffac403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.146+0000 7ffaceffd640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffac4038f70 0x7ffac403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffac00098c0 tx=0x7ffac0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:47.200 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.151+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffac8032dd0 con 0x7ffad8107f80 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:43.320+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi143", "addr": "172.21.15.143", "target": ["mon-mgr", ""]}) -- 0x7ffad8102e60 con 0x7ffac4038f70 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.448+0000 7ffad57fa640 1 -- 172.21.15.143:0/2952041198 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffad8102e60 con 0x7ffac4038f70 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffac4038f70 msgr2=0x7ffac403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffac4038f70 0x7ffac403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffac00098c0 tx=0x7ffac0006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 msgr2=0x7ffad819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad819b440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffac80097f0 tx=0x7ffac8030d00 comp rx=0 tx=0).stop 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 shutdown_connections 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffac4038f70 0x7ffac403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 --2- 172.21.15.143:0/2952041198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad8107f80 0x7ffad819b440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 >> 172.21.15.143:0/2952041198 conn(0x7ffad8101bc0 msgr2=0x7ffad8102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.451+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 shutdown_connections 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:46.452+0000 7ffade055640 1 -- 172.21.15.143:0/2952041198 wait complete. 2024-09-10T10:35:47.201 INFO:teuthology.orchestra.run.smithi143.stdout:Deploying unmanaged mon service... 2024-09-10T10:35:47.700 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:47 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:47.701 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:47 smithi143 ceph-mon[31057]: Added host smithi143 2024-09-10T10:35:47.701 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:47 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:35:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T10:35:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a41324640 1 Processor -- start 2024-09-10T10:35:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a41324640 1 -- start start 2024-09-10T10:35:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a41324640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c1067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a41324640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c106dc0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.836 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a3affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c1067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a3affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c1067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33146/0 (socket says 172.21.15.143:33146) 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.699+0000 7f7a3affd640 1 -- 172.21.15.143:0/3602198772 learned_addr learned my addr 172.21.15.143:0/3602198772 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.700+0000 7f7a3affd640 1 -- 172.21.15.143:0/3602198772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c1075f0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a3affd640 1 --2- 172.21.15.143:0/3602198772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c1067f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009820 tx=0x7f7a2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece02c26af683bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a39ffb640 1 -- 172.21.15.143:0/3602198772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c037070 con 0x7f7a3c1063f0 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a39ffb640 1 -- 172.21.15.143:0/3602198772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7a2c0328d0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a39ffb640 1 -- 172.21.15.143:0/3602198772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c0307d0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a41324640 1 -- 172.21.15.143:0/3602198772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 msgr2=0x7f7a3c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a41324640 1 --2- 172.21.15.143:0/3602198772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c1067f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009820 tx=0x7f7a2c02a270 comp rx=0 tx=0).stop 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a41324640 1 -- 172.21.15.143:0/3602198772 shutdown_connections 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a41324640 1 --2- 172.21.15.143:0/3602198772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c1067f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.701+0000 7f7a41324640 1 -- 172.21.15.143:0/3602198772 >> 172.21.15.143:0/3602198772 conn(0x7f7a3c101ba0 msgr2=0x7f7a3c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.702+0000 7f7a41324640 1 -- 172.21.15.143:0/3602198772 shutdown_connections 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.702+0000 7f7a41324640 1 -- 172.21.15.143:0/3602198772 wait complete. 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.702+0000 7f7a41324640 1 Processor -- start 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.702+0000 7f7a41324640 1 -- start start 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.702+0000 7f7a41324640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c07ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.702+0000 7f7a41324640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c10a9a0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a3affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c07ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a3affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c07ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33150/0 (socket says 172.21.15.143:33150) 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a3affd640 1 -- 172.21.15.143:0/2992589776 learned_addr learned my addr 172.21.15.143:0/2992589776 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:48.838 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a3affd640 1 -- 172.21.15.143:0/2992589776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c19bfa0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a3affd640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c07ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0097f0 tx=0x7f7a2c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c037040 con 0x7f7a3c1063f0 2024-09-10T10:35:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7a2c032bb0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c1a05b0 con 0x7f7a3c1063f0 2024-09-10T10:35:48.839 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c03a390 con 0x7f7a3c1063f0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.703+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c1a0950 con 0x7f7a3c1063f0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.704+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7a2c03a580 con 0x7f7a3c1063f0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.704+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3c106870 con 0x7f7a3c1063f0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.704+0000 7f7a23fff640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f7a24038fc0 0x7f7a2403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.704+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a2c06c190 con 0x7f7a3c1063f0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.704+0000 7f7a3a7fc640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f7a24038fc0 0x7f7a2403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.705+0000 7f7a3a7fc640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f7a24038fc0 0x7f7a2403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7a300098c0 tx=0x7f7a30006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:48.840 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.709+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a2c040210 con 0x7f7a3c1063f0 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.890+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7a3c102da0 con 0x7f7a24038fc0 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.897+0000 7f7a23fff640 1 -- 172.21.15.143:0/2992589776 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7a3c102da0 con 0x7f7a24038fc0 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f7a24038fc0 msgr2=0x7f7a2403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f7a24038fc0 0x7f7a2403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7a300098c0 tx=0x7f7a30006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 msgr2=0x7f7a3c07ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c07ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0097f0 tx=0x7f7a2c030d00 comp rx=0 tx=0).stop 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 shutdown_connections 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f7a24038fc0 0x7f7a2403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 --2- 172.21.15.143:0/2992589776 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a3c1063f0 0x7f7a3c07ba30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 >> 172.21.15.143:0/2992589776 conn(0x7f7a3c101ba0 msgr2=0x7f7a3c102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 shutdown_connections 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:47.899+0000 7f7a41324640 1 -- 172.21.15.143:0/2992589776 wait complete. 2024-09-10T10:35:48.841 INFO:teuthology.orchestra.run.smithi143.stdout:Deploying unmanaged mgr service... 2024-09-10T10:35:49.047 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:48 smithi143 ceph-mon[31057]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:49.047 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:48 smithi143 ceph-mon[31057]: Saving service mon spec with placement count:5 2024-09-10T10:35:49.047 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:48 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:50.247 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T10:35:50.247 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.233+0000 7f92d0033640 1 Processor -- start 2024-09-10T10:35:50.247 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.233+0000 7f92d0033640 1 -- start start 2024-09-10T10:35:50.247 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.234+0000 7f92d0033640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:50.248 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.234+0000 7f92d0033640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f92c807c8b0 con 0x7f92c807de80 2024-09-10T10:35:50.248 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.234+0000 7f92cddcf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:50.248 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.234+0000 7f92cddcf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33162/0 (socket says 172.21.15.143:33162) 2024-09-10T10:35:50.248 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.234+0000 7f92cddcf640 1 -- 172.21.15.143:0/143603871 learned_addr learned my addr 172.21.15.143:0/143603871 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:50.248 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.234+0000 7f92cddcf640 1 -- 172.21.15.143:0/143603871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c807ca30 con 0x7f92c807de80 2024-09-10T10:35:50.248 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92cddcf640 1 --2- 172.21.15.143:0/143603871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c807c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009820 tx=0x7f92bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1878f448f8330c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:50.249 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92ccdcd640 1 -- 172.21.15.143:0/143603871 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92bc037070 con 0x7f92c807de80 2024-09-10T10:35:50.249 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92ccdcd640 1 -- 172.21.15.143:0/143603871 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f92bc0328d0 con 0x7f92c807de80 2024-09-10T10:35:50.249 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92ccdcd640 1 -- 172.21.15.143:0/143603871 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92bc0307d0 con 0x7f92c807de80 2024-09-10T10:35:50.249 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92d0033640 1 -- 172.21.15.143:0/143603871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 msgr2=0x7f92c807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:50.249 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92d0033640 1 --2- 172.21.15.143:0/143603871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c807c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009820 tx=0x7f92bc02a270 comp rx=0 tx=0).stop 2024-09-10T10:35:50.250 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92d0033640 1 -- 172.21.15.143:0/143603871 shutdown_connections 2024-09-10T10:35:50.250 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92d0033640 1 --2- 172.21.15.143:0/143603871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c807c2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:50.250 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.235+0000 7f92d0033640 1 -- 172.21.15.143:0/143603871 >> 172.21.15.143:0/143603871 conn(0x7f92c8102c50 msgr2=0x7f92c8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:50.250 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92d0033640 1 -- 172.21.15.143:0/143603871 shutdown_connections 2024-09-10T10:35:50.250 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92d0033640 1 -- 172.21.15.143:0/143603871 wait complete. 2024-09-10T10:35:50.250 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92d0033640 1 Processor -- start 2024-09-10T10:35:50.251 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92d0033640 1 -- start start 2024-09-10T10:35:50.251 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92d0033640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:50.251 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92d0033640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f92c810dad0 con 0x7f92c807de80 2024-09-10T10:35:50.251 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92cddcf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:50.251 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92cddcf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33166/0 (socket says 172.21.15.143:33166) 2024-09-10T10:35:50.252 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.236+0000 7f92cddcf640 1 -- 172.21.15.143:0/3659738086 learned_addr learned my addr 172.21.15.143:0/3659738086 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:50.252 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92cddcf640 1 -- 172.21.15.143:0/3659738086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c819dba0 con 0x7f92c807de80 2024-09-10T10:35:50.252 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92cddcf640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c819d660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f92bc032e10 tx=0x7f92bc032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:50.252 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92bc042070 con 0x7f92c807de80 2024-09-10T10:35:50.252 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f92bc030c80 con 0x7f92c807de80 2024-09-10T10:35:50.252 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c819de30 con 0x7f92c807de80 2024-09-10T10:35:50.253 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92bc037040 con 0x7f92c807de80 2024-09-10T10:35:50.253 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.237+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81a0b20 con 0x7f92c807de80 2024-09-10T10:35:50.253 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.238+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f92bc045070 con 0x7f92c807de80 2024-09-10T10:35:50.253 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.238+0000 7f92baffd640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f92b4038fc0 0x7f92b403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:50.253 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.238+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c807c360 con 0x7f92c807de80 2024-09-10T10:35:50.253 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.238+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92bc06c040 con 0x7f92c807de80 2024-09-10T10:35:50.254 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.238+0000 7f92cd5ce640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f92b4038fc0 0x7f92b403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:50.254 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.239+0000 7f92cd5ce640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f92b4038fc0 0x7f92b403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f92c40098c0 tx=0x7f92c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:50.254 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.243+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92bc045360 con 0x7f92c807de80 2024-09-10T10:35:50.254 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.426+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f92c8103f00 con 0x7f92b4038fc0 2024-09-10T10:35:50.254 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.430+0000 7f92baffd640 1 -- 172.21.15.143:0/3659738086 <== mgr.14118 v2:172.21.15.143:6800/1050213403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f92c8103f00 con 0x7f92b4038fc0 2024-09-10T10:35:50.254 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f92b4038fc0 msgr2=0x7f92b403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f92b4038fc0 0x7f92b403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f92c40098c0 tx=0x7f92c4006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 msgr2=0x7f92c819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c819d660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f92bc032e10 tx=0x7f92bc032e40 comp rx=0 tx=0).stop 2024-09-10T10:35:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 shutdown_connections 2024-09-10T10:35:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f92b4038fc0 0x7f92b403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 --2- 172.21.15.143:0/3659738086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92c807de80 0x7f92c819d660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:50.256 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 >> 172.21.15.143:0/3659738086 conn(0x7f92c8102c50 msgr2=0x7f92c81037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:50.256 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 shutdown_connections 2024-09-10T10:35:50.256 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:49.432+0000 7f92d0033640 1 -- 172.21.15.143:0/3659738086 wait complete. 2024-09-10T10:35:50.541 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:50 smithi143 ceph-mon[31057]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:35:50.542 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:50 smithi143 ceph-mon[31057]: Saving service mgr spec with placement count:2 2024-09-10T10:35:50.542 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:50 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:50.542 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:50 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:50.542 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:50 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:51.966 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:51 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1124231290' entity='client.admin' 2024-09-10T10:35:51.966 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:51 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:51.973 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4afa43a640 1 Processor -- start 2024-09-10T10:35:51.973 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4afa43a640 1 -- start start 2024-09-10T10:35:51.973 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4afa43a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af41078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:51.973 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4afa43a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4107e90 con 0x7f4af41074c0 2024-09-10T10:35:51.973 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af41078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af41078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33182/0 (socket says 172.21.15.143:33182) 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.621+0000 7f4af3fff640 1 -- 172.21.15.143:0/2852114782 learned_addr learned my addr 172.21.15.143:0/2852114782 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.622+0000 7f4af3fff640 1 -- 172.21.15.143:0/2852114782 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af41086c0 con 0x7f4af41074c0 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.622+0000 7f4af3fff640 1 --2- 172.21.15.143:0/2852114782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af41078c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009820 tx=0x7f4ae402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79907f0414c58818 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.622+0000 7f4af2ffd640 1 -- 172.21.15.143:0/2852114782 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae4037070 con 0x7f4af41074c0 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4af2ffd640 1 -- 172.21.15.143:0/2852114782 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ae40328f0 con 0x7f4af41074c0 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4af2ffd640 1 -- 172.21.15.143:0/2852114782 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae4030740 con 0x7f4af41074c0 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 -- 172.21.15.143:0/2852114782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 msgr2=0x7f4af41078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 --2- 172.21.15.143:0/2852114782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af41078c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009820 tx=0x7f4ae402a270 comp rx=0 tx=0).stop 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 -- 172.21.15.143:0/2852114782 shutdown_connections 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 --2- 172.21.15.143:0/2852114782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af41078c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:51.974 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 -- 172.21.15.143:0/2852114782 >> 172.21.15.143:0/2852114782 conn(0x7f4af4102c50 msgr2=0x7f4af4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 -- 172.21.15.143:0/2852114782 shutdown_connections 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.623+0000 7f4afa43a640 1 -- 172.21.15.143:0/2852114782 wait complete. 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4afa43a640 1 Processor -- start 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4afa43a640 1 -- start start 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4afa43a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4afa43a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410ba70 con 0x7f4af41074c0 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4af3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33184/0 (socket says 172.21.15.143:33184) 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4af3fff640 1 -- 172.21.15.143:0/1124231290 learned_addr learned my addr 172.21.15.143:0/1124231290 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4af3fff640 1 -- 172.21.15.143:0/1124231290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af419b9d0 con 0x7f4af41074c0 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.624+0000 7f4af3fff640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af419b490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009940 tx=0x7f4ae4030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.625+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae4042070 con 0x7f4af41074c0 2024-09-10T10:35:51.975 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.625+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ae4030e30 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.625+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af419bc60 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.625+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae4037040 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.625+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af419e950 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.626+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4ae4045070 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.626+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af410bbf0 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.626+0000 7f4af17fa640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4adc038f70 0x7f4adc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.626+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ae406c020 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.626+0000 7f4af37fe640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4adc038f70 0x7f4adc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.626+0000 7f4af37fe640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4adc038f70 0x7f4adc03b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098c0 tx=0x7f4ae8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.630+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ae404b170 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.818+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4af4107940 con 0x7f4af41074c0 2024-09-10T10:35:51.976 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.823+0000 7f4af17fa640 1 -- 172.21.15.143:0/1124231290 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4ae402fd90 con 0x7f4af41074c0 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4adc038f70 msgr2=0x7f4adc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4adc038f70 0x7f4adc03b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098c0 tx=0x7f4ae8006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 msgr2=0x7f4af419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af419b490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009940 tx=0x7f4ae4030cc0 comp rx=0 tx=0).stop 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 shutdown_connections 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4adc038f70 0x7f4adc03b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 --2- 172.21.15.143:0/1124231290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af41074c0 0x7f4af419b490 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 >> 172.21.15.143:0/1124231290 conn(0x7f4af4102c50 msgr2=0x7f4af41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 shutdown_connections 2024-09-10T10:35:51.977 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:50.825+0000 7f4afa43a640 1 -- 172.21.15.143:0/1124231290 wait complete. 2024-09-10T10:35:53.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.367+0000 7f33bb3f6640 1 Processor -- start 2024-09-10T10:35:53.148 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.368+0000 7f33bb3f6640 1 -- start start 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.368+0000 7f33bb3f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b410a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.368+0000 7f33bb3f6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f33b40757b0 con 0x7f33b4107fc0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.368+0000 7f33b9192640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b410a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.368+0000 7f33b9192640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b410a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33188/0 (socket says 172.21.15.143:33188) 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.368+0000 7f33b9192640 1 -- 172.21.15.143:0/2056799402 learned_addr learned my addr 172.21.15.143:0/2056799402 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.369+0000 7f33b9192640 1 -- 172.21.15.143:0/2056799402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b410a8f0 con 0x7f33b4107fc0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.369+0000 7f33b9192640 1 --2- 172.21.15.143:0/2056799402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b410a3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009820 tx=0x7f33b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0452f07bb1c9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.369+0000 7f33abfff640 1 -- 172.21.15.143:0/2056799402 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b0037070 con 0x7f33b4107fc0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.369+0000 7f33abfff640 1 -- 172.21.15.143:0/2056799402 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33b00328f0 con 0x7f33b4107fc0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.369+0000 7f33abfff640 1 -- 172.21.15.143:0/2056799402 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b0030740 con 0x7f33b4107fc0 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 -- 172.21.15.143:0/2056799402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 msgr2=0x7f33b410a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 --2- 172.21.15.143:0/2056799402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b410a3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009820 tx=0x7f33b002a270 comp rx=0 tx=0).stop 2024-09-10T10:35:53.149 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 -- 172.21.15.143:0/2056799402 shutdown_connections 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 --2- 172.21.15.143:0/2056799402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b410a3b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 -- 172.21.15.143:0/2056799402 >> 172.21.15.143:0/2056799402 conn(0x7f33b4101bc0 msgr2=0x7f33b4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 -- 172.21.15.143:0/2056799402 shutdown_connections 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.370+0000 7f33bb3f6640 1 -- 172.21.15.143:0/2056799402 wait complete. 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.371+0000 7f33bb3f6640 1 Processor -- start 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.371+0000 7f33bb3f6640 1 -- start start 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33bb3f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33bb3f6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f33b410b910 con 0x7f33b4107fc0 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33b9192640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33b9192640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33190/0 (socket says 172.21.15.143:33190) 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33b9192640 1 -- 172.21.15.143:0/534738908 learned_addr learned my addr 172.21.15.143:0/534738908 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33b9192640 1 -- 172.21.15.143:0/534738908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b419b9b0 con 0x7f33b4107fc0 2024-09-10T10:35:53.150 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33b9192640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b419b470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33b0030cc0 tx=0x7f33b0030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b0042070 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b419bc40 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33b0030e60 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b0037040 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.372+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b419e930 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.373+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f33b0045070 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.374+0000 7f33aa7fc640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f33a0038f70 0x7f33a003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.374+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f33b006c110 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.374+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b4108420 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.374+0000 7f33b8991640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f33a0038f70 0x7f33a003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.374+0000 7f33b8991640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f33a0038f70 0x7f33a003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33a40098c0 tx=0x7f33a4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.379+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33b0045360 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.572+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f33b419eb00 con 0x7f33b4107fc0 2024-09-10T10:35:53.151 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.575+0000 7f33aa7fc640 1 -- 172.21.15.143:0/534738908 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f33b002ed90 con 0x7f33b4107fc0 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f33a0038f70 msgr2=0x7f33a003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f33a0038f70 0x7f33a003b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33a40098c0 tx=0x7f33a4006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 msgr2=0x7f33b419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b419b470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33b0030cc0 tx=0x7f33b0030cf0 comp rx=0 tx=0).stop 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 shutdown_connections 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f33a0038f70 0x7f33a003b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 --2- 172.21.15.143:0/534738908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33b4107fc0 0x7f33b419b470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 >> 172.21.15.143:0/534738908 conn(0x7f33b4101bc0 msgr2=0x7f33b4102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 shutdown_connections 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:52.577+0000 7f33bb3f6640 1 -- 172.21.15.143:0/534738908 wait complete. 2024-09-10T10:35:53.152 INFO:teuthology.orchestra.run.smithi143.stdout:Enabling the dashboard module... 2024-09-10T10:35:53.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:53 smithi143 ceph-mon[31057]: from='mgr.14118 172.21.15.143:0/2221649586' entity='mgr.x' 2024-09-10T10:35:53.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:53 smithi143 ceph-mon[31057]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:35:53.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:53 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/534738908' entity='client.admin' 2024-09-10T10:35:54.790 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2567486695' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T10:35:54.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: ignoring --setuser ceph since I am not root 2024-09-10T10:35:54.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: ignoring --setgroup ceph since I am not root 2024-09-10T10:35:54.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:54.584+0000 7f546ad90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:35:54.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:54.641+0000 7f546ad90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:35:54.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:54.789+0000 7f546ad90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:35:55.050 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:54.858+0000 7f546ad90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:35:55.051 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:55.007+0000 7f546ad90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:35:55.170 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.594+0000 7ffaccb5c640 1 Processor -- start 2024-09-10T10:35:55.170 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.594+0000 7ffaccb5c640 1 -- start start 2024-09-10T10:35:55.170 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.595+0000 7ffaccb5c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac8106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:55.170 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.595+0000 7ffaccb5c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8106de0 con 0x7ffac8106410 2024-09-10T10:35:55.170 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.595+0000 7ffac659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac8106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.595+0000 7ffac659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac8106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33192/0 (socket says 172.21.15.143:33192) 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.595+0000 7ffac659c640 1 -- 172.21.15.143:0/3603388360 learned_addr learned my addr 172.21.15.143:0/3603388360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.595+0000 7ffac659c640 1 -- 172.21.15.143:0/3603388360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac8107610 con 0x7ffac8106410 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffac659c640 1 --2- 172.21.15.143:0/3603388360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac8106810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009820 tx=0x7ffab802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa9b74d3fdc6981 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffac5d9b640 1 -- 172.21.15.143:0/3603388360 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8037070 con 0x7ffac8106410 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffac5d9b640 1 -- 172.21.15.143:0/3603388360 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffab80328f0 con 0x7ffac8106410 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffac5d9b640 1 -- 172.21.15.143:0/3603388360 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8030740 con 0x7ffac8106410 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffaccb5c640 1 -- 172.21.15.143:0/3603388360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 msgr2=0x7ffac8106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffaccb5c640 1 --2- 172.21.15.143:0/3603388360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac8106810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009820 tx=0x7ffab802a270 comp rx=0 tx=0).stop 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffaccb5c640 1 -- 172.21.15.143:0/3603388360 shutdown_connections 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffaccb5c640 1 --2- 172.21.15.143:0/3603388360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac8106810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.596+0000 7ffaccb5c640 1 -- 172.21.15.143:0/3603388360 >> 172.21.15.143:0/3603388360 conn(0x7ffac8101bc0 msgr2=0x7ffac8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:55.171 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.597+0000 7ffaccb5c640 1 -- 172.21.15.143:0/3603388360 shutdown_connections 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.597+0000 7ffaccb5c640 1 -- 172.21.15.143:0/3603388360 wait complete. 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.597+0000 7ffaccb5c640 1 Processor -- start 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.597+0000 7ffaccb5c640 1 -- start start 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.597+0000 7ffaccb5c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.597+0000 7ffaccb5c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffac810a9c0 con 0x7ffac8106410 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffac659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffac659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33204/0 (socket says 172.21.15.143:33204) 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffac659c640 1 -- 172.21.15.143:0/2567486695 learned_addr learned my addr 172.21.15.143:0/2567486695 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffac659c640 1 -- 172.21.15.143:0/2567486695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac819b9f0 con 0x7ffac8106410 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffac659c640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac819b4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009940 tx=0x7ffab8030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8042070 con 0x7ffac8106410 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffab8030e30 con 0x7ffac8106410 2024-09-10T10:35:55.172 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac819bc80 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8037040 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.598+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac819e970 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.599+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ffab8045070 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.599+0000 7ffabf7fe640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffab4038f70 0x7ffab403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.599+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac810ab40 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.599+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffab806c170 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.599+0000 7ffabffff640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffab4038f70 0x7ffab403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.600+0000 7ffabffff640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffab4038f70 0x7ffab403b430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffab00098c0 tx=0x7ffab0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.604+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffab803be10 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:53.817+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ffac819eb40 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.433+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ffab806b790 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.435+0000 7ffabf7fe640 1 -- 172.21.15.143:0/2567486695 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ffab803c650 con 0x7ffac8106410 2024-09-10T10:35:55.173 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffab4038f70 msgr2=0x7ffab403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffab4038f70 0x7ffab403b430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffab00098c0 tx=0x7ffab0006db0 comp rx=0 tx=0).stop 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 msgr2=0x7ffac819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac819b4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009940 tx=0x7ffab8030cc0 comp rx=0 tx=0).stop 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 shutdown_connections 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7ffab4038f70 0x7ffab403b430 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 --2- 172.21.15.143:0/2567486695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffac8106410 0x7ffac819b4b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 >> 172.21.15.143:0/2567486695 conn(0x7ffac8101bc0 msgr2=0x7ffac81026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 shutdown_connections 2024-09-10T10:35:55.174 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:54.436+0000 7ffaccb5c640 1 -- 172.21.15.143:0/2567486695 wait complete. 2024-09-10T10:35:55.435 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:55.150+0000 7f546ad90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:35:55.732 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2567486695' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T10:35:55.732 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-mon[31057]: mgrmap e9: x(active, since 21s) 2024-09-10T10:35:55.732 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:55.474+0000 7f546ad90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:35:55.732 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:55.538+0000 7f546ad90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:35:55.732 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:55.730+0000 7f546ad90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:35:56.345 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:35:56.345 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.543+0000 7f465e48d640 1 Processor -- start 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.543+0000 7f465e48d640 1 -- start start 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.543+0000 7f465e48d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f4658106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.543+0000 7f465e48d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4658106de0 con 0x7f4658106410 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.544+0000 7f4657fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f4658106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.544+0000 7f4657fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f4658106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33232/0 (socket says 172.21.15.143:33232) 2024-09-10T10:35:56.346 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.544+0000 7f4657fff640 1 -- 172.21.15.143:0/499810691 learned_addr learned my addr 172.21.15.143:0/499810691 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.544+0000 7f4657fff640 1 -- 172.21.15.143:0/499810691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4658107610 con 0x7f4658106410 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f4657fff640 1 --2- 172.21.15.143:0/499810691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f4658106810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4648009820 tx=0x7f464802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fba22afcbb49e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f4656ffd640 1 -- 172.21.15.143:0/499810691 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4648037070 con 0x7f4658106410 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f4656ffd640 1 -- 172.21.15.143:0/499810691 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46480328f0 con 0x7f4658106410 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f4656ffd640 1 -- 172.21.15.143:0/499810691 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4648030740 con 0x7f4658106410 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f465e48d640 1 -- 172.21.15.143:0/499810691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 msgr2=0x7f4658106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f465e48d640 1 --2- 172.21.15.143:0/499810691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f4658106810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4648009820 tx=0x7f464802a270 comp rx=0 tx=0).stop 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f465e48d640 1 -- 172.21.15.143:0/499810691 shutdown_connections 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f465e48d640 1 --2- 172.21.15.143:0/499810691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f4658106810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.545+0000 7f465e48d640 1 -- 172.21.15.143:0/499810691 >> 172.21.15.143:0/499810691 conn(0x7f4658101bc0 msgr2=0x7f4658103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:56.347 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.546+0000 7f465e48d640 1 -- 172.21.15.143:0/499810691 shutdown_connections 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.546+0000 7f465e48d640 1 -- 172.21.15.143:0/499810691 wait complete. 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.546+0000 7f465e48d640 1 Processor -- start 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.546+0000 7f465e48d640 1 -- start start 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.546+0000 7f465e48d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f465819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.546+0000 7f465e48d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f465810a9c0 con 0x7f4658106410 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f4657fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f465819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f4657fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f465819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33234/0 (socket says 172.21.15.143:33234) 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f4657fff640 1 -- 172.21.15.143:0/738233350 learned_addr learned my addr 172.21.15.143:0/738233350 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f4657fff640 1 -- 172.21.15.143:0/738233350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465819a8f0 con 0x7f4658106410 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f4657fff640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f465819a3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4648009940 tx=0x7f4648030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:35:56.348 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4648042070 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4648030e30 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4648037040 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465819ab80 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.547+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465819d870 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.548+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f4648045070 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.548+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465810ab40 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.549+0000 7f46557fa640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 0x7f464403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.549+0000 7f46577fe640 1 -- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 msgr2=0x7f464403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.143:6800/1050213403 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.549+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f464806c1a0 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.549+0000 7f46577fe640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 0x7f464403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.553+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4648045360 con 0x7f4658106410 2024-09-10T10:35:56.349 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.750+0000 7f46577fe640 1 -- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 msgr2=0x7f464403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.143:6800/1050213403 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.750+0000 7f46577fe640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 0x7f464403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.774+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f465819da40 con 0x7f4658106410 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.774+0000 7f46557fa640 1 -- 172.21.15.143:0/738233350 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f464803be10 con 0x7f4658106410 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 msgr2=0x7f464403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 0x7f464403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 msgr2=0x7f465819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f465819a3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4648009940 tx=0x7f4648030cc0 comp rx=0 tx=0).stop 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 shutdown_connections 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7f4644038fc0 0x7f464403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 --2- 172.21.15.143:0/738233350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4658106410 0x7f465819a3b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:35:56.350 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 >> 172.21.15.143:0/738233350 conn(0x7f4658101bc0 msgr2=0x7f46581026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:35:56.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 shutdown_connections 2024-09-10T10:35:56.351 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:55.777+0000 7f465e48d640 1 -- 172.21.15.143:0/738233350 wait complete. 2024-09-10T10:35:56.351 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for the mgr to restart... 2024-09-10T10:35:56.351 INFO:teuthology.orchestra.run.smithi143.stdout:Waiting for mgr epoch 9... 2024-09-10T10:35:56.713 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:56 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/738233350' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T10:35:56.713 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:56 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:56.641+0000 7f546ad90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:35:56.713 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:56 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:56.712+0000 7f546ad90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:35:57.679 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:57.422+0000 7f546ad90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:35:57.679 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:57.537+0000 7f546ad90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:35:57.958 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:57.677+0000 7f546ad90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:35:57.958 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:57.739+0000 7f546ad90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:35:57.958 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:57.889+0000 7f546ad90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:35:57.958 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:57.957+0000 7f546ad90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:35:58.241 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.017+0000 7f546ad90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:35:58.241 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.175+0000 7f546ad90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:35:58.508 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.239+0000 7f546ad90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:35:58.508 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.310+0000 7f546ad90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:35:58.508 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.396+0000 7f546ad90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:35:58.508 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.458+0000 7f546ad90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:35:58.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:58 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:58.521+0000 7f546ad90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:35:59.336+0000 7f546ad90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: Active manager daemon x restarted 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: Activating manager daemon x 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: mgrmap e10: x(active, starting, since 0.00345141s) 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:35:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:35:59.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:35:59.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: Manager daemon x is now available 2024-09-10T10:35:59.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:35:59.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:35:59.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:35:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:36:01.190 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:36:01.190 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.734+0000 7fa0ff2e0640 1 Processor -- start 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0ff2e0640 1 -- start start 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0ff2e0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0ff2e0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8106dc0 con 0x7fa0f81063f0 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0fd07c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0fd07c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40734/0 (socket says 172.21.15.143:40734) 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0fd07c640 1 -- 172.21.15.143:0/910631355 learned_addr learned my addr 172.21.15.143:0/910631355 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.735+0000 7fa0fd07c640 1 -- 172.21.15.143:0/910631355 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f81075f0 con 0x7fa0f81063f0 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.736+0000 7fa0fd07c640 1 --2- 172.21.15.143:0/910631355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81067f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400bac0 tx=0x7fa0f402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f713c6a98a63f912 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.736+0000 7fa0effff640 1 -- 172.21.15.143:0/910631355 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f40326d0 con 0x7fa0f81063f0 2024-09-10T10:36:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.736+0000 7fa0effff640 1 -- 172.21.15.143:0/910631355 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0f4038070 con 0x7fa0f81063f0 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.736+0000 7fa0effff640 1 -- 172.21.15.143:0/910631355 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f4033c40 con 0x7fa0f81063f0 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/910631355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 msgr2=0x7fa0f81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/910631355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81067f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400bac0 tx=0x7fa0f402c920 comp rx=0 tx=0).stop 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/910631355 shutdown_connections 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/910631355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81067f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/910631355 >> 172.21.15.143:0/910631355 conn(0x7fa0f8101ba0 msgr2=0x7fa0f8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/910631355 shutdown_connections 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.737+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/910631355 wait complete. 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0ff2e0640 1 Processor -- start 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0ff2e0640 1 -- start start 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0ff2e0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0ff2e0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f807b130 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0fd07c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0fd07c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40746/0 (socket says 172.21.15.143:40746) 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0fd07c640 1 -- 172.21.15.143:0/3950557017 learned_addr learned my addr 172.21.15.143:0/3950557017 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0fd07c640 1 -- 172.21.15.143:0/3950557017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f8199830 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.738+0000 7fa0fd07c640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81992f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b2c0 tx=0x7fa0f40327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.739+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f4042070 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.739+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f8199ac0 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.739+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f819c7b0 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.739+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0f4032d20 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.739+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f403c3f0 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.740+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa0f403c590 con 0x7fa0f81063f0 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.740+0000 7fa0ee7fc640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:01.193 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.740+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 --> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0e403bb90 con 0x7fa0e4038fc0 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.740+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa0f4030050 con 0x7fa0f81063f0 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.740+0000 7fa0fc87b640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 msgr2=0x7fa0e403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.143:6800/1050213403 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.740+0000 7fa0fc87b640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.941+0000 7fa0fc87b640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 msgr2=0x7fa0e403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.143:6800/1050213403 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:56.941+0000 7fa0fc87b640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:57.342+0000 7fa0fc87b640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 msgr2=0x7fa0e403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.143:6800/1050213403 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:57.342+0000 7fa0fc87b640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:58.143+0000 7fa0fc87b640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 msgr2=0x7fa0e403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.143:6800/1050213403 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:58.143+0000 7fa0fc87b640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:59.340+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fa0f406da90 con 0x7fa0f81063f0 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:59.340+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 msgr2=0x7fa0e403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:35:59.340+0000 7fa0ee7fc640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:01.194 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.341+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa0f403cbf0 con 0x7fa0f81063f0 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.341+0000 7fa0ee7fc640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa0e403ca80 0x7fa0e403ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.341+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0f404eeb0 con 0x7fa0e403ca80 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.342+0000 7fa0fc87b640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa0e403ca80 0x7fa0e403ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.342+0000 7fa0fc87b640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa0e403ca80 0x7fa0e403ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8003980 tx=0x7fa0e8006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.343+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa0f404eeb0 con 0x7fa0e403ca80 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.350+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa0dc0025a0 con 0x7fa0e403ca80 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ee7fc640 1 -- 172.21.15.143:0/3950557017 <== mgr.14150 v2:172.21.15.143:6800/1095035473 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa0dc0025a0 con 0x7fa0e403ca80 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa0e403ca80 msgr2=0x7fa0e403ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa0e403ca80 0x7fa0e403ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8003980 tx=0x7fa0e8006e10 comp rx=0 tx=0).stop 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 msgr2=0x7fa0f81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81992f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b2c0 tx=0x7fa0f40327d0 comp rx=0 tx=0).stop 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 shutdown_connections 2024-09-10T10:36:01.195 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa0e403ca80 0x7fa0e403ee70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:6800/1050213403,v1:172.21.15.143:6801/1050213403] conn(0x7fa0e4038fc0 0x7fa0e403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 --2- 172.21.15.143:0/3950557017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0f81063f0 0x7fa0f81992f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 >> 172.21.15.143:0/3950557017 conn(0x7fa0f8101ba0 msgr2=0x7fa0f818f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 shutdown_connections 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:00.351+0000 7fa0ff2e0640 1 -- 172.21.15.143:0/3950557017 wait complete. 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:mgr epoch 9 is available 2024-09-10T10:36:01.196 INFO:teuthology.orchestra.run.smithi143.stdout:Generating a dashboard self-signed certificate... 2024-09-10T10:36:01.442 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-mon[31057]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T10:36:01.442 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-mon[31057]: mgrmap e11: x(active, since 1.00636s) 2024-09-10T10:36:01.442 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-mon[31057]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T10:36:01.442 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:01.442 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:36:01.327+0000 7f53d0852640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:10:36:01] ENGINE Error in HTTPServer.serve 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: Traceback (most recent call last): 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self._connections.run(self.expiration_interval) 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self._run(expiration_interval) 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: s = self.context.wrap_socket( 2024-09-10T10:36:01.443 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: return self.sslsocket_class._create( 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self.do_handshake() 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: self._sslobj.do_handshake() 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:36:01.444 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:36:01 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.619+0000 7f68253cc640 1 Processor -- start 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.619+0000 7f68253cc640 1 -- start start 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.619+0000 7f68253cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682010a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.619+0000 7f68253cc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f68200757b0 con 0x7f6820107fc0 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.620+0000 7f681effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682010a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.620+0000 7f681effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682010a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40816/0 (socket says 172.21.15.143:40816) 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.620+0000 7f681effd640 1 -- 172.21.15.143:0/1586175983 learned_addr learned my addr 172.21.15.143:0/1586175983 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.620+0000 7f681effd640 1 -- 172.21.15.143:0/1586175983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682010a8f0 con 0x7f6820107fc0 2024-09-10T10:36:02.429 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f681effd640 1 --2- 172.21.15.143:0/1586175983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682010a3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6810009e80 tx=0x7f681002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292b6c871f17218d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f681dffb640 1 -- 172.21.15.143:0/1586175983 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810036070 con 0x7f6820107fc0 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f681dffb640 1 -- 172.21.15.143:0/1586175983 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68100318b0 con 0x7f6820107fc0 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f681dffb640 1 -- 172.21.15.143:0/1586175983 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810007930 con 0x7f6820107fc0 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f68253cc640 1 -- 172.21.15.143:0/1586175983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 msgr2=0x7f682010a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f68253cc640 1 --2- 172.21.15.143:0/1586175983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682010a3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6810009e80 tx=0x7f681002a7a0 comp rx=0 tx=0).stop 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f68253cc640 1 -- 172.21.15.143:0/1586175983 shutdown_connections 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f68253cc640 1 --2- 172.21.15.143:0/1586175983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682010a3b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.621+0000 7f68253cc640 1 -- 172.21.15.143:0/1586175983 >> 172.21.15.143:0/1586175983 conn(0x7f6820101bc0 msgr2=0x7f6820103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f68253cc640 1 -- 172.21.15.143:0/1586175983 shutdown_connections 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f68253cc640 1 -- 172.21.15.143:0/1586175983 wait complete. 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f68253cc640 1 Processor -- start 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f68253cc640 1 -- start start 2024-09-10T10:36:02.430 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f68253cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f68253cc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f682010b910 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.622+0000 7f681effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f681effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40820/0 (socket says 172.21.15.143:40820) 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f681effd640 1 -- 172.21.15.143:0/2544273167 learned_addr learned my addr 172.21.15.143:0/2544273167 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f681effd640 1 -- 172.21.15.143:0/2544273167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682019b9b0 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f681effd640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682019b470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6810031c90 tx=0x7f6810031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810036050 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682019bc40 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6810042070 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f681003daa0 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.623+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682019e930 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.624+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6810047460 con 0x7f6820107fc0 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.624+0000 7f6817fff640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f680c038eb0 0x7f680c03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:02.431 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.624+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6820108420 con 0x7f6820107fc0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.624+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f681006d4c0 con 0x7f6820107fc0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.624+0000 7f681e7fc640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f680c038eb0 0x7f680c03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.625+0000 7f681e7fc640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f680c038eb0 0x7f680c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68080098c0 tx=0x7f6808006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.629+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6810044310 con 0x7f6820107fc0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.788+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f681004dcf0 con 0x7f6820107fc0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.825+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6820102e80 con 0x7f680c038eb0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.906+0000 7f6817fff640 1 -- 172.21.15.143:0/2544273167 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6820102e80 con 0x7f680c038eb0 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f680c038eb0 msgr2=0x7f680c03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f680c038eb0 0x7f680c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68080098c0 tx=0x7f6808006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 msgr2=0x7f682019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682019b470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6810031c90 tx=0x7f6810031cc0 comp rx=0 tx=0).stop 2024-09-10T10:36:02.432 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 shutdown_connections 2024-09-10T10:36:02.433 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f680c038eb0 0x7f680c03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:02.433 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 --2- 172.21.15.143:0/2544273167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6820107fc0 0x7f682019b470 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:02.433 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 >> 172.21.15.143:0/2544273167 conn(0x7f6820101bc0 msgr2=0x7f6820102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:02.433 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 shutdown_connections 2024-09-10T10:36:02.433 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:01.908+0000 7f68253cc640 1 -- 172.21.15.143:0/2544273167 wait complete. 2024-09-10T10:36:02.433 INFO:teuthology.orchestra.run.smithi143.stdout:Creating initial admin user... 2024-09-10T10:36:02.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: [10/Sep/2024:10:36:01] ENGINE Bus STARTING 2024-09-10T10:36:02.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: [10/Sep/2024:10:36:01] ENGINE Serving on http://172.21.15.143:8765 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: [10/Sep/2024:10:36:01] ENGINE Serving on https://172.21.15.143:7150 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: [10/Sep/2024:10:36:01] ENGINE Bus STARTED 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: [10/Sep/2024:10:36:01] ENGINE Error in HTTPServer.serve 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: Traceback (most recent call last): 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._run(expiration_interval) 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:36:02.669 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: s = self.context.wrap_socket( 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: return self.sslsocket_class._create( 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: self.do_handshake() 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: self._sslobj.do_handshake() 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: mgrmap e12: x(active, since 2s) 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:02.670 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:03.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:03 smithi143 ceph-mon[31057]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:03.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:03 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:04.544 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AeVbh1s3Jav.bXPokvCjMOM8v.TcnR6QPcSl4A72LNTaWbUJ0tjJi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725964563, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T10:36:04.544 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.871+0000 7ff493ab5640 1 Processor -- start 2024-09-10T10:36:04.544 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.871+0000 7ff493ab5640 1 -- start start 2024-09-10T10:36:04.544 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.871+0000 7ff493ab5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c10a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:04.544 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.871+0000 7ff493ab5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c0757b0 con 0x7ff48c107fc0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.872+0000 7ff491851640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c10a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.872+0000 7ff491851640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c10a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40828/0 (socket says 172.21.15.143:40828) 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.872+0000 7ff491851640 1 -- 172.21.15.143:0/1279051485 learned_addr learned my addr 172.21.15.143:0/1279051485 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.872+0000 7ff491851640 1 -- 172.21.15.143:0/1279051485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c10a8f0 con 0x7ff48c107fc0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.872+0000 7ff491851640 1 --2- 172.21.15.143:0/1279051485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c10a3b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff488009820 tx=0x7ff48802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3a2849e9714aea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.873+0000 7ff49084f640 1 -- 172.21.15.143:0/1279051485 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488037070 con 0x7ff48c107fc0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.873+0000 7ff49084f640 1 -- 172.21.15.143:0/1279051485 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4880328f0 con 0x7ff48c107fc0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.873+0000 7ff49084f640 1 -- 172.21.15.143:0/1279051485 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488030740 con 0x7ff48c107fc0 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.873+0000 7ff493ab5640 1 -- 172.21.15.143:0/1279051485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 msgr2=0x7ff48c10a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.873+0000 7ff493ab5640 1 --2- 172.21.15.143:0/1279051485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c10a3b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff488009820 tx=0x7ff48802a270 comp rx=0 tx=0).stop 2024-09-10T10:36:04.545 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 -- 172.21.15.143:0/1279051485 shutdown_connections 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 --2- 172.21.15.143:0/1279051485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c10a3b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 -- 172.21.15.143:0/1279051485 >> 172.21.15.143:0/1279051485 conn(0x7ff48c101bc0 msgr2=0x7ff48c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 -- 172.21.15.143:0/1279051485 shutdown_connections 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 -- 172.21.15.143:0/1279051485 wait complete. 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 Processor -- start 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 -- start start 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c107840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.874+0000 7ff493ab5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10b910 con 0x7ff48c107fc0 2024-09-10T10:36:04.546 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff491851640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff491851640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40840/0 (socket says 172.21.15.143:40840) 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff491851640 1 -- 172.21.15.143:0/2093322346 learned_addr learned my addr 172.21.15.143:0/2093322346 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff491851640 1 -- 172.21.15.143:0/2093322346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c107d80 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff491851640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c107840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4880097f0 tx=0x7ff488030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488042070 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff488030d80 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488037040 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c105f70 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.875+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1064b0 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.876+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff488045070 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.876+0000 7ff47effd640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff478038fc0 0x7ff47803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.876+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff48806d0a0 con 0x7ff48c107fc0 2024-09-10T10:36:04.547 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.877+0000 7ff491050640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff478038fc0 0x7ff47803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.877+0000 7ff491050640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff478038fc0 0x7ff47803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098c0 tx=0x7ff480006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.877+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c108420 con 0x7ff48c107fc0 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:02.882+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff488045520 con 0x7ff48c107fc0 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.071+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff48c106b10 con 0x7ff478038fc0 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.294+0000 7ff47effd640 1 -- 172.21.15.143:0/2093322346 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff48c106b10 con 0x7ff478038fc0 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.296+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff478038fc0 msgr2=0x7ff47803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.296+0000 7ff493ab5640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff478038fc0 0x7ff47803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098c0 tx=0x7ff480006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.296+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 msgr2=0x7ff48c107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c107840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4880097f0 tx=0x7ff488030c70 comp rx=0 tx=0).stop 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 shutdown_connections 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff478038fc0 0x7ff47803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 --2- 172.21.15.143:0/2093322346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff48c107fc0 0x7ff48c107840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:04.548 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 >> 172.21.15.143:0/2093322346 conn(0x7ff48c101bc0 msgr2=0x7ff48c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:04.549 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 shutdown_connections 2024-09-10T10:36:04.549 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:03.297+0000 7ff493ab5640 1 -- 172.21.15.143:0/2093322346 wait complete. 2024-09-10T10:36:04.549 INFO:teuthology.orchestra.run.smithi143.stdout:Fetching dashboard port number... 2024-09-10T10:36:04.667 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:04 smithi143 ceph-mon[31057]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:05.635 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:05 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4082001651' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T10:36:05.646 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T10:36:05.646 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.949+0000 7f7a6a784640 1 Processor -- start 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a6a784640 1 -- start start 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a6a784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a641078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a6a784640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64107e90 con 0x7f7a641074c0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a641078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a641078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40856/0 (socket says 172.21.15.143:40856) 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a63fff640 1 -- 172.21.15.143:0/23772082 learned_addr learned my addr 172.21.15.143:0/23772082 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.950+0000 7f7a63fff640 1 -- 172.21.15.143:0/23772082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a641086c0 con 0x7f7a641074c0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.951+0000 7f7a63fff640 1 --2- 172.21.15.143:0/23772082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a641078c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009820 tx=0x7f7a5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763fb5a3c85f2ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.951+0000 7f7a62ffd640 1 -- 172.21.15.143:0/23772082 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54037070 con 0x7f7a641074c0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.951+0000 7f7a62ffd640 1 -- 172.21.15.143:0/23772082 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a540328f0 con 0x7f7a641074c0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.951+0000 7f7a62ffd640 1 -- 172.21.15.143:0/23772082 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54030740 con 0x7f7a641074c0 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 -- 172.21.15.143:0/23772082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 msgr2=0x7f7a641078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:05.647 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 --2- 172.21.15.143:0/23772082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a641078c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009820 tx=0x7f7a5402a270 comp rx=0 tx=0).stop 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 -- 172.21.15.143:0/23772082 shutdown_connections 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 --2- 172.21.15.143:0/23772082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a641078c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 -- 172.21.15.143:0/23772082 >> 172.21.15.143:0/23772082 conn(0x7f7a64102c50 msgr2=0x7f7a64105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 -- 172.21.15.143:0/23772082 shutdown_connections 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 -- 172.21.15.143:0/23772082 wait complete. 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 Processor -- start 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.952+0000 7f7a6a784640 1 -- start start 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a6a784640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a6419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a6a784640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6410ba70 con 0x7f7a641074c0 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a6419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a63fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a6419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40868/0 (socket says 172.21.15.143:40868) 2024-09-10T10:36:05.648 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a63fff640 1 -- 172.21.15.143:0/4082001651 learned_addr learned my addr 172.21.15.143:0/4082001651 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a63fff640 1 -- 172.21.15.143:0/4082001651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6419b9d0 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a63fff640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a6419b490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009940 tx=0x7f7a54030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54042070 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a54030e30 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6419bc60 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54037040 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.953+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6419e950 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.954+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7a54045070 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.954+0000 7f7a617fa640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a50039020 0x7f7a5003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.954+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6410bbf0 con 0x7f7a641074c0 2024-09-10T10:36:05.649 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.954+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7a5406c120 con 0x7f7a641074c0 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.954+0000 7f7a637fe640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a50039020 0x7f7a5003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.955+0000 7f7a637fe640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a50039020 0x7f7a5003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098c0 tx=0x7f7a4c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:04.959+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a54045520 con 0x7f7a641074c0 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.126+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7a64107940 con 0x7f7a641074c0 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.126+0000 7f7a617fa640 1 -- 172.21.15.143:0/4082001651 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7a5403add0 con 0x7f7a641074c0 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a50039020 msgr2=0x7f7a5003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a50039020 0x7f7a5003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098c0 tx=0x7f7a4c006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 msgr2=0x7f7a6419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a6419b490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009940 tx=0x7f7a54030cc0 comp rx=0 tx=0).stop 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 shutdown_connections 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a50039020 0x7f7a5003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 --2- 172.21.15.143:0/4082001651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a641074c0 0x7f7a6419b490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:05.650 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.128+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 >> 172.21.15.143:0/4082001651 conn(0x7f7a64102c50 msgr2=0x7f7a641037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:05.651 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.129+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 shutdown_connections 2024-09-10T10:36:05.651 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:05.129+0000 7f7a6a784640 1 -- 172.21.15.143:0/4082001651 wait complete. 2024-09-10T10:36:05.651 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:36:05.651 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout disabled 2024-09-10T10:36:05.656 INFO:teuthology.orchestra.run.smithi143.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:36:05.656 INFO:teuthology.orchestra.run.smithi143.stdout:systemctl: stdout inactive 2024-09-10T10:36:05.656 INFO:teuthology.orchestra.run.smithi143.stdout:firewalld.service is not enabled 2024-09-10T10:36:05.656 INFO:teuthology.orchestra.run.smithi143.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout:Ceph Dashboard is now available at: 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout: URL: https://smithi143.front.sepia.ceph.com:8443/ 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout: User: admin 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout: Password: kp3fti13q7 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:05.658 INFO:teuthology.orchestra.run.smithi143.stdout:Saving cluster configuration to /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config directory 2024-09-10T10:36:06.608 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:06 smithi143 ceph-mon[31057]: mgrmap e13: x(active, since 6s) 2024-09-10T10:36:06.608 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:06 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1751511335' entity='client.admin' 2024-09-10T10:36:07.022 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.055+0000 7f7803ef9640 1 Processor -- start 2024-09-10T10:36:07.022 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.055+0000 7f7803ef9640 1 -- start start 2024-09-10T10:36:07.022 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.055+0000 7f7803ef9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc1061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:07.022 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.055+0000 7f7803ef9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc07be10 con 0x7f77fc105dd0 2024-09-10T10:36:07.023 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.056+0000 7f7801c95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc1061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:07.023 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.056+0000 7f7801c95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc1061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40872/0 (socket says 172.21.15.143:40872) 2024-09-10T10:36:07.023 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.056+0000 7f7801c95640 1 -- 172.21.15.143:0/2526147273 learned_addr learned my addr 172.21.15.143:0/2526147273 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:07.023 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.056+0000 7f7801c95640 1 -- 172.21.15.143:0/2526147273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc106710 con 0x7f77fc105dd0 2024-09-10T10:36:07.023 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7801c95640 1 --2- 172.21.15.143:0/2526147273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc1061d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009e80 tx=0x7f77ec02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10078d807fccce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:07.024 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7800c93640 1 -- 172.21.15.143:0/2526147273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ec036070 con 0x7f77fc105dd0 2024-09-10T10:36:07.024 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7800c93640 1 -- 172.21.15.143:0/2526147273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77ec0318b0 con 0x7f77fc105dd0 2024-09-10T10:36:07.024 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7800c93640 1 -- 172.21.15.143:0/2526147273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ec007930 con 0x7f77fc105dd0 2024-09-10T10:36:07.024 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 -- 172.21.15.143:0/2526147273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 msgr2=0x7f77fc1061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:07.024 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 --2- 172.21.15.143:0/2526147273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc1061d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009e80 tx=0x7f77ec02a7a0 comp rx=0 tx=0).stop 2024-09-10T10:36:07.024 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 -- 172.21.15.143:0/2526147273 shutdown_connections 2024-09-10T10:36:07.025 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 --2- 172.21.15.143:0/2526147273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc1061d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:07.025 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 -- 172.21.15.143:0/2526147273 >> 172.21.15.143:0/2526147273 conn(0x7f77fc101ba0 msgr2=0x7f77fc103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:07.025 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 -- 172.21.15.143:0/2526147273 shutdown_connections 2024-09-10T10:36:07.025 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.057+0000 7f7803ef9640 1 -- 172.21.15.143:0/2526147273 wait complete. 2024-09-10T10:36:07.025 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7803ef9640 1 Processor -- start 2024-09-10T10:36:07.025 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7803ef9640 1 -- start start 2024-09-10T10:36:07.026 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7803ef9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:07.026 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7803ef9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1077f0 con 0x7f77fc105dd0 2024-09-10T10:36:07.026 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7801c95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:07.026 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7801c95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40874/0 (socket says 172.21.15.143:40874) 2024-09-10T10:36:07.026 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7801c95640 1 -- 172.21.15.143:0/1751511335 learned_addr learned my addr 172.21.15.143:0/1751511335 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:07.026 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.058+0000 7f7801c95640 1 -- 172.21.15.143:0/1751511335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc19dbb0 con 0x7f77fc105dd0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.059+0000 7f7801c95640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc19d670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77ec031c90 tx=0x7f77ec031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.059+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ec036050 con 0x7f77fc105dd0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.059+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77ec042070 con 0x7f77fc105dd0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.059+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc19de40 con 0x7f77fc105dd0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.059+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77ec03daa0 con 0x7f77fc105dd0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.059+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a0b30 con 0x7f77fc105dd0 2024-09-10T10:36:07.027 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.060+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f77ec03dc40 con 0x7f77fc105dd0 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.060+0000 7f77faffd640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77e40390c0 0x7f77e403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.060+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f77ec06d6c0 con 0x7f77fc105dd0 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.060+0000 7f7801494640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77e40390c0 0x7f77e403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.060+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc1061d0 con 0x7f77fc105dd0 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.060+0000 7f7801494640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77e40390c0 0x7f77e403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098c0 tx=0x7f77f0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.065+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77ec044310 con 0x7f77fc105dd0 2024-09-10T10:36:07.028 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.314+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f77fc02d8c0 con 0x7f77fc105dd0 2024-09-10T10:36:07.029 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.317+0000 7f77faffd640 1 -- 172.21.15.143:0/1751511335 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f77ec036c30 con 0x7f77fc105dd0 2024-09-10T10:36:07.029 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T10:36:07.029 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77e40390c0 msgr2=0x7f77e403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:07.029 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77e40390c0 0x7f77e403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098c0 tx=0x7f77f0006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:07.029 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 msgr2=0x7f77fc19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc19d670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77ec031c90 tx=0x7f77ec031cc0 comp rx=0 tx=0).stop 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 shutdown_connections 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77e40390c0 0x7f77e403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 --2- 172.21.15.143:0/1751511335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77fc105dd0 0x7f77fc19d670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 >> 172.21.15.143:0/1751511335 conn(0x7f77fc101ba0 msgr2=0x7f77fc194330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 shutdown_connections 2024-09-10T10:36:07.030 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:06.319+0000 7f7803ef9640 1 -- 172.21.15.143:0/1751511335 wait complete. 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: ceph telemetry on 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.031 INFO:teuthology.orchestra.run.smithi143.stdout:For more information see: 2024-09-10T10:36:07.032 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.032 INFO:teuthology.orchestra.run.smithi143.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T10:36:07.032 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:07.032 INFO:teuthology.orchestra.run.smithi143.stdout:Bootstrap complete. 2024-09-10T10:36:07.057 INFO:tasks.cephadm:Fetching config... 2024-09-10T10:36:07.057 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:07.057 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T10:36:07.074 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T10:36:07.074 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:07.074 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T10:36:07.130 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T10:36:07.130 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:07.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T10:36:07.194 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T10:36:07.195 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:07.195 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T10:36:07.251 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T10:36:07.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T10:36:07.334 INFO:teuthology.orchestra.run.smithi143.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:07.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T10:36:07.396 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:07.409 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T10:36:07.546 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7qU4dYjvhNL4EQL/zMsF4Jdw2LbqQ0BEEb8x7O8wWONRQ6aS9thyumeAOZgLkgAV0K2drK0nieQvvBWBkPuh8QrzaajMP2Qu0h7fTb9UNdHlfrOFC+qT0Fi6PBrbApPrbclCjIFj5VLfAexWVM1Z9Qpx9+ry91hYWwSxugki37oc0y7trDx3ycKtjThD/lW7EXdtGsEgObQO6yzd8VxuwlNRj1tws5h2XgqazS3Cri2ZRZBf6ZIYmlYyLPOZffcw5FI6HqFHQOTlg7Qzm0GpgfcUaNUA39FdqXzP1Nq9gaB3BIMyNRXDuOS+umzFL+Bd4YIjyPtTndhSGfyTz7j3UXf55InZpCyX1R7C5Su2XG34JSVFqD8V+ysaLfJEz5/xPjrAmaobLgNGzjW6n/JzAOkrvtyDMSQBXMiMWbaFyKuEUG/d2ozwGM294JF5lyetKZUvvTQPggCYX4zapvP4dBJ+r7kQaL1X/uK4fCgdn0fbDa/SYXUcuMc0DfC5kPok= ceph-34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:07.559 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T10:36:07.809 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.323+0000 7f53835a5640 1 -- 172.21.15.143:0/790430378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 msgr2=0x7f537c104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.323+0000 7f53835a5640 1 --2- 172.21.15.143:0/790430378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c104540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5378009880 tx=0x7f537802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.323+0000 7f53835a5640 1 -- 172.21.15.143:0/790430378 shutdown_connections 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.323+0000 7f53835a5640 1 --2- 172.21.15.143:0/790430378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c104540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.323+0000 7f53835a5640 1 -- 172.21.15.143:0/790430378 >> 172.21.15.143:0/790430378 conn(0x7f537c0fbcb0 msgr2=0x7f537c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f53835a5640 1 -- 172.21.15.143:0/790430378 shutdown_connections 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f53835a5640 1 -- 172.21.15.143:0/790430378 wait complete. 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f53835a5640 1 Processor -- start 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f53835a5640 1 -- start start 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f53835a5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f53835a5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f537c1075d0 con 0x7f537c102150 2024-09-10T10:36:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f5381341640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f5381341640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48432/0 (socket says 172.21.15.143:48432) 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.324+0000 7f5381341640 1 -- 172.21.15.143:0/2522468531 learned_addr learned my addr 172.21.15.143:0/2522468531 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f5381341640 1 -- 172.21.15.143:0/2522468531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c1971b0 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f5381341640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c196c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5378009230 tx=0x7f53780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f537803f070 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5378003aa0 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c197440 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5378035030 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.325+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c19a130 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.326+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5378042070 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.326+0000 7f53727fc640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5368039060 0x7f536803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.326+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c107d50 con 0x7f537c102150 2024-09-10T10:36:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.326+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f537806d010 con 0x7f537c102150 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.326+0000 7f5380b40640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5368039060 0x7f536803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.326+0000 7f5380b40640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5368039060 0x7f536803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f536c0098c0 tx=0x7f536c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.331+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f537803b4b0 con 0x7f537c102150 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.501+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f537c1025b0 con 0x7f537c102150 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.504+0000 7f53727fc640 1 -- 172.21.15.143:0/2522468531 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f537c1025b0 con 0x7f537c102150 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5368039060 msgr2=0x7f536803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5368039060 0x7f536803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f536c0098c0 tx=0x7f536c006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 msgr2=0x7f537c196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c196c70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5378009230 tx=0x7f53780048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 shutdown_connections 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5368039060 0x7f536803b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 --2- 172.21.15.143:0/2522468531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f537c102150 0x7f537c196c70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 >> 172.21.15.143:0/2522468531 conn(0x7f537c0fbcb0 msgr2=0x7f537c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 shutdown_connections 2024-09-10T10:36:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:08.506+0000 7f53835a5640 1 -- 172.21.15.143:0/2522468531 wait complete. 2024-09-10T10:36:09.503 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T10:36:09.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T10:36:09.737 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2522468531' entity='client.admin' 2024-09-10T10:36:09.737 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:09.737 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:09.737 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:36:09.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:09.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:09.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:09.967 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:10.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 -- 172.21.15.143:0/396523888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 msgr2=0x7f1474101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:10.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 --2- 172.21.15.143:0/396523888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474101a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c20 tx=0x7f146c02aa90 comp rx=0 tx=0).stop 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 -- 172.21.15.143:0/396523888 shutdown_connections 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 --2- 172.21.15.143:0/396523888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474101a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 -- 172.21.15.143:0/396523888 >> 172.21.15.143:0/396523888 conn(0x7f14740fcdf0 msgr2=0x7f14740ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 -- 172.21.15.143:0/396523888 shutdown_connections 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.433+0000 7f147aac2640 1 -- 172.21.15.143:0/396523888 wait complete. 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147aac2640 1 Processor -- start 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147aac2640 1 -- start start 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147aac2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147aac2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1474107600 con 0x7f1474101640 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147885e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:10.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147885e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48464/0 (socket says 172.21.15.143:48464) 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147885e640 1 -- 172.21.15.143:0/1849047882 learned_addr learned my addr 172.21.15.143:0/1849047882 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.434+0000 7f147885e640 1 -- 172.21.15.143:0/1849047882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740763f0 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.435+0000 7f147885e640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f146c006010 tx=0x7f146c030ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.435+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f146c047070 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.435+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f146c02fa70 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.435+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14740745e0 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.435+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f146c033050 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.435+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1474074b20 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.436+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f146c04a070 con 0x7f1474101640 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.436+0000 7f1471ffb640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14600390c0 0x7f146003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:10.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.436+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f146c03e030 con 0x7f1474101640 2024-09-10T10:36:10.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.436+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1474107d80 con 0x7f1474101640 2024-09-10T10:36:10.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.436+0000 7f1473fff640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14600390c0 0x7f146003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:10.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.437+0000 7f1473fff640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14600390c0 0x7f146003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f14640098c0 tx=0x7f1464006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:10.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.441+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f146c038cf0 con 0x7f1474101640 2024-09-10T10:36:10.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.617+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f147418ff80 con 0x7f14600390c0 2024-09-10T10:36:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.621+0000 7f1471ffb640 1 -- 172.21.15.143:0/1849047882 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f147418ff80 con 0x7f14600390c0 2024-09-10T10:36:10.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14600390c0 msgr2=0x7f146003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:10.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14600390c0 0x7f146003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f14640098c0 tx=0x7f1464006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 msgr2=0x7f1474075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f146c006010 tx=0x7f146c030ee0 comp rx=0 tx=0).stop 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 shutdown_connections 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14600390c0 0x7f146003b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 --2- 172.21.15.143:0/1849047882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1474101640 0x7f1474075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 >> 172.21.15.143:0/1849047882 conn(0x7f14740fcdf0 msgr2=0x7f14740fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 shutdown_connections 2024-09-10T10:36:10.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:10.623+0000 7f147aac2640 1 -- 172.21.15.143:0/1849047882 wait complete. 2024-09-10T10:36:11.492 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-09-10T10:36:11.492 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:36:11.492 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-09-10T10:36:11.512 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:36:11.512 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:11.573 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-09-10T10:36:11.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch host add smithi144 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: Updating smithi143:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: Updating smithi143:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:11.998 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 -- 172.21.15.143:0/3605895241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 msgr2=0x7f6a5c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 --2- 172.21.15.143:0/3605895241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c100d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009880 tx=0x7f6a4c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 -- 172.21.15.143:0/3605895241 shutdown_connections 2024-09-10T10:36:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 --2- 172.21.15.143:0/3605895241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c100d70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 -- 172.21.15.143:0/3605895241 >> 172.21.15.143:0/3605895241 conn(0x7f6a5c076250 msgr2=0x7f6a5c076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 -- 172.21.15.143:0/3605895241 shutdown_connections 2024-09-10T10:36:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 -- 172.21.15.143:0/3605895241 wait complete. 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 Processor -- start 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.412+0000 7f6a63fef640 1 -- start start 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a63fef640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a63fef640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10aa40 con 0x7f6a5c100990 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a61d8b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a61d8b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48476/0 (socket says 172.21.15.143:48476) 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a61d8b640 1 -- 172.21.15.143:0/3911187856 learned_addr learned my addr 172.21.15.143:0/3911187856 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a61d8b640 1 -- 172.21.15.143:0/3911187856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c198420 con 0x7f6a5c100990 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a61d8b640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c197ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009230 tx=0x7f6a4c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:12.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c03f070 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a4c004d10 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c035030 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c198710 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.413+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c198bd0 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.414+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6a4c042070 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.414+0000 7f6a5affd640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a440390c0 0x7f6a4403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.414+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a4c06cc50 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.414+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c101c40 con 0x7f6a5c100990 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.414+0000 7f6a6158a640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a440390c0 0x7f6a4403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.415+0000 7f6a6158a640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a440390c0 0x7f6a4403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a500098c0 tx=0x7f6a50006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.419+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a4c039350 con 0x7f6a5c100990 2024-09-10T10:36:12.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:12.588+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7f6a5c100df0 con 0x7f6a440390c0 2024-09-10T10:36:13.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:13 smithi143 ceph-mon[31057]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:13 smithi143 ceph-mon[31057]: Deploying cephadm binary to smithi144 2024-09-10T10:36:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.703+0000 7f6a5affd640 1 -- 172.21.15.143:0/3911187856 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6a5c100df0 con 0x7f6a440390c0 2024-09-10T10:36:15.705 INFO:teuthology.orchestra.run.smithi143.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-09-10T10:36:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a440390c0 msgr2=0x7f6a4403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a440390c0 0x7f6a4403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a500098c0 tx=0x7f6a50006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 msgr2=0x7f6a5c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c197ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009230 tx=0x7f6a4c0048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 shutdown_connections 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a440390c0 0x7f6a4403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 --2- 172.21.15.143:0/3911187856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5c100990 0x7f6a5c197ee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 >> 172.21.15.143:0/3911187856 conn(0x7f6a5c076250 msgr2=0x7f6a5c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.705+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 shutdown_connections 2024-09-10T10:36:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:15.706+0000 7f6a63fef640 1 -- 172.21.15.143:0/3911187856 wait complete. 2024-09-10T10:36:16.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T10:36:16.546 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:16.978 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:16.978 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:16 smithi143 ceph-mon[31057]: Added host smithi144 2024-09-10T10:36:16.978 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:16.978 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:16.978 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:17.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 -- 172.21.15.143:0/1668319186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 msgr2=0x7fce84106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:17.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 --2- 172.21.15.143:0/1668319186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce84106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce78009880 tx=0x7fce7802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:17.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 -- 172.21.15.143:0/1668319186 shutdown_connections 2024-09-10T10:36:17.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 --2- 172.21.15.143:0/1668319186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce84106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:17.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 -- 172.21.15.143:0/1668319186 >> 172.21.15.143:0/1668319186 conn(0x7fce840fc1a0 msgr2=0x7fce840fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 -- 172.21.15.143:0/1668319186 shutdown_connections 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.976+0000 7fce8b24d640 1 -- 172.21.15.143:0/1668319186 wait complete. 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce8b24d640 1 Processor -- start 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce8b24d640 1 -- start start 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce8b24d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce841970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce8b24d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410aa70 con 0x7fce84106590 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce88fe9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce841970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce88fe9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce841970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35660/0 (socket says 172.21.15.143:35660) 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce88fe9640 1 -- 172.21.15.143:0/1610670773 learned_addr learned my addr 172.21.15.143:0/1610670773 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.977+0000 7fce88fe9640 1 -- 172.21.15.143:0/1610670773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce841975f0 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.978+0000 7fce88fe9640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce841970b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce78009230 tx=0x7fce780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.978+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce7803f070 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.978+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce78003aa0 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.978+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce84197880 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.978+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce78035030 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.978+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce84197ce0 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.979+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fce78042070 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.979+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce84101c00 con 0x7fce84106590 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.979+0000 7fce81ffb640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fce74039060 0x7fce7403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.979+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce7806ccb0 con 0x7fce84106590 2024-09-10T10:36:17.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.979+0000 7fce83fff640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fce74039060 0x7fce7403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:17.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.979+0000 7fce83fff640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fce74039060 0x7fce7403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce700098c0 tx=0x7fce70006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:17.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:16.984+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce78072020 con 0x7fce84106590 2024-09-10T10:36:17.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.156+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce84198470 con 0x7fce74039060 2024-09-10T10:36:17.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.157+0000 7fce81ffb640 1 -- 172.21.15.143:0/1610670773 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fce84198470 con 0x7fce74039060 2024-09-10T10:36:17.158 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:17.158 INFO:teuthology.orchestra.run.smithi143.stdout:[{"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-09-10T10:36:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fce74039060 msgr2=0x7fce7403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fce74039060 0x7fce7403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce700098c0 tx=0x7fce70006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 msgr2=0x7fce841970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:17.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce841970b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce78009230 tx=0x7fce780048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 shutdown_connections 2024-09-10T10:36:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fce74039060 0x7fce7403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 --2- 172.21.15.143:0/1610670773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce84106590 0x7fce841970b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 >> 172.21.15.143:0/1610670773 conn(0x7fce840fc1a0 msgr2=0x7fce84069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 shutdown_connections 2024-09-10T10:36:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:17.159+0000 7fce8b24d640 1 -- 172.21.15.143:0/1610670773 wait complete. 2024-09-10T10:36:17.782 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-09-10T10:36:17.783 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:36:17.783 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-09-10T10:36:17.801 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:36:17.802 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:17.860 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-09-10T10:36:17.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch host add smithi178 2024-09-10T10:36:18.104 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:18.354 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:18.354 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:18 smithi143 ceph-mon[31057]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:36:18.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 -- 172.21.15.143:0/4088432214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 msgr2=0x7f5c84106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:18.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 --2- 172.21.15.143:0/4088432214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84106970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009880 tx=0x7f5c7402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 -- 172.21.15.143:0/4088432214 shutdown_connections 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 --2- 172.21.15.143:0/4088432214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84106970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 -- 172.21.15.143:0/4088432214 >> 172.21.15.143:0/4088432214 conn(0x7f5c840fc1a0 msgr2=0x7f5c840fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 -- 172.21.15.143:0/4088432214 shutdown_connections 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.533+0000 7f5c89471640 1 -- 172.21.15.143:0/4088432214 wait complete. 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c89471640 1 Processor -- start 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c89471640 1 -- start start 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c89471640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c89471640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8410aa70 con 0x7f5c84106590 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c82ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c82ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35672/0 (socket says 172.21.15.143:35672) 2024-09-10T10:36:18.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c82ffd640 1 -- 172.21.15.143:0/2191216068 learned_addr learned my addr 172.21.15.143:0/2191216068 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:18.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.534+0000 7f5c82ffd640 1 -- 172.21.15.143:0/2191216068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c841964b0 con 0x7f5c84106590 2024-09-10T10:36:18.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.535+0000 7f5c82ffd640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84195f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009230 tx=0x7f5c740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.535+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c7403f070 con 0x7f5c84106590 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.535+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c84196740 con 0x7f5c84106590 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.535+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c74004d10 con 0x7f5c84106590 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.535+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c74035030 con 0x7f5c84106590 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.535+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c84196ba0 con 0x7f5c84106590 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.536+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5c74042070 con 0x7f5c84106590 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.536+0000 7f5c80ff9640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c70039060 0x7f5c7003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:18.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.536+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c84101c00 con 0x7f5c84106590 2024-09-10T10:36:18.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.536+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c7406cca0 con 0x7f5c84106590 2024-09-10T10:36:18.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.536+0000 7f5c7bfff640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c70039060 0x7f5c7003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:18.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.536+0000 7f5c7bfff640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c70039060 0x7f5c7003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0098c0 tx=0x7f5c6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.541+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c740055a0 con 0x7f5c84106590 2024-09-10T10:36:18.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:18.712+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}) -- 0x7f5c841972d0 con 0x7f5c70039060 2024-09-10T10:36:20.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:21.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: Deploying cephadm binary to smithi178 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: Updating smithi144:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: Updating smithi144:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:21.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.837+0000 7f5c80ff9640 1 -- 172.21.15.143:0/2191216068 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5c841972d0 con 0x7f5c70039060 2024-09-10T10:36:21.838 INFO:teuthology.orchestra.run.smithi143.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-09-10T10:36:21.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c70039060 msgr2=0x7f5c7003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:21.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c70039060 0x7f5c7003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0098c0 tx=0x7f5c6c006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:21.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 msgr2=0x7f5c84195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:21.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84195f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009230 tx=0x7f5c740048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:21.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 shutdown_connections 2024-09-10T10:36:21.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c70039060 0x7f5c7003b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:21.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 --2- 172.21.15.143:0/2191216068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c84106590 0x7f5c84195f70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:21.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 >> 172.21.15.143:0/2191216068 conn(0x7f5c840fc1a0 msgr2=0x7f5c840fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:21.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 shutdown_connections 2024-09-10T10:36:21.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:21.839+0000 7f5c89471640 1 -- 172.21.15.143:0/2191216068 wait complete. 2024-09-10T10:36:22.426 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:22 smithi143 ceph-mon[31057]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:22.426 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:22.426 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:22.426 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:22.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T10:36:22.689 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 -- 172.21.15.143:0/1443445126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 msgr2=0x7faed8100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 --2- 172.21.15.143:0/1443445126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8100d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faed4009880 tx=0x7faed402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 -- 172.21.15.143:0/1443445126 shutdown_connections 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 --2- 172.21.15.143:0/1443445126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8100d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 -- 172.21.15.143:0/1443445126 >> 172.21.15.143:0/1443445126 conn(0x7faed8076250 msgr2=0x7faed8076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 -- 172.21.15.143:0/1443445126 shutdown_connections 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.133+0000 7faedf337640 1 -- 172.21.15.143:0/1443445126 wait complete. 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedf337640 1 Processor -- start 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedf337640 1 -- start start 2024-09-10T10:36:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedf337640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedf337640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faed810aa40 con 0x7faed8100990 2024-09-10T10:36:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedd0d3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedd0d3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35688/0 (socket says 172.21.15.143:35688) 2024-09-10T10:36:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedd0d3640 1 -- 172.21.15.143:0/3159473137 learned_addr learned my addr 172.21.15.143:0/3159473137 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.134+0000 7faedd0d3640 1 -- 172.21.15.143:0/3159473137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed8198420 con 0x7faed8100990 2024-09-10T10:36:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.135+0000 7faedd0d3640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8197ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faed4009230 tx=0x7faed40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:23.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.135+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faed403f070 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.135+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faed4004d10 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.135+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed8198710 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.135+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faed4035030 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.135+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed8198bd0 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.136+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7faed4042070 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.136+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed8101c40 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.136+0000 7faece7fc640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7faec4039060 0x7faec403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.136+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faed406cdc0 con 0x7faed8100990 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.136+0000 7faedc8d2640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7faec4039060 0x7faec403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.136+0000 7faedc8d2640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7faec4039060 0x7faec403b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faec80098c0 tx=0x7faec8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:23.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.141+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faed40303a0 con 0x7faed8100990 2024-09-10T10:36:23.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.323+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faed8069cb0 con 0x7faec4039060 2024-09-10T10:36:23.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.324+0000 7faece7fc640 1 -- 172.21.15.143:0/3159473137 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7faed8069cb0 con 0x7faec4039060 2024-09-10T10:36:23.325 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:23.325 INFO:teuthology.orchestra.run.smithi143.stdout:[{"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7faec4039060 msgr2=0x7faec403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7faec4039060 0x7faec403b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faec80098c0 tx=0x7faec8006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 msgr2=0x7faed8197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8197ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faed4009230 tx=0x7faed40048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 shutdown_connections 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7faec4039060 0x7faec403b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 --2- 172.21.15.143:0/3159473137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed8100990 0x7faed8197ee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 >> 172.21.15.143:0/3159473137 conn(0x7faed8076250 msgr2=0x7faed81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:23.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 shutdown_connections 2024-09-10T10:36:23.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:23.326+0000 7faedf337640 1 -- 172.21.15.143:0/3159473137 wait complete. 2024-09-10T10:36:23.624 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:23 smithi143 ceph-mon[31057]: Added host smithi178 2024-09-10T10:36:23.625 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:23 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:23.625 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:23 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:23.928 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T10:36:23.928 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T10:36:24.166 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:24.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:24 smithi143 ceph-mon[31057]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:36:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:24 smithi143 ceph-mon[31057]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 -- 172.21.15.143:0/1682580718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 msgr2=0x7fdfc0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 --2- 172.21.15.143:0/1682580718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009880 tx=0x7fdfb002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 -- 172.21.15.143:0/1682580718 shutdown_connections 2024-09-10T10:36:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 --2- 172.21.15.143:0/1682580718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 -- 172.21.15.143:0/1682580718 >> 172.21.15.143:0/1682580718 conn(0x7fdfc00fc1a0 msgr2=0x7fdfc00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 -- 172.21.15.143:0/1682580718 shutdown_connections 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 -- 172.21.15.143:0/1682580718 wait complete. 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.600+0000 7fdfc5c28640 1 Processor -- start 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfc5c28640 1 -- start start 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfc5c28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfc5c28640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010aa70 con 0x7fdfc0106590 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35698/0 (socket says 172.21.15.143:35698) 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbf7fe640 1 -- 172.21.15.143:0/2168761799 learned_addr learned my addr 172.21.15.143:0/2168761799 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbf7fe640 1 -- 172.21.15.143:0/2168761799 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc0104040 con 0x7fdfc0106590 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbf7fe640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0102150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009230 tx=0x7fdfb00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb003f070 con 0x7fdfc0106590 2024-09-10T10:36:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.601+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfb0004d10 con 0x7fdfc0106590 2024-09-10T10:36:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.602+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc0102690 con 0x7fdfc0106590 2024-09-10T10:36:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.602+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb0035030 con 0x7fdfc0106590 2024-09-10T10:36:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.602+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc0102b50 con 0x7fdfc0106590 2024-09-10T10:36:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.602+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fdfb0042070 con 0x7fdfc0106590 2024-09-10T10:36:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.603+0000 7fdfbcff9640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdfa80390c0 0x7fdfa803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.603+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc0101c00 con 0x7fdfc0106590 2024-09-10T10:36:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.603+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdfb006ccd0 con 0x7fdfc0106590 2024-09-10T10:36:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.603+0000 7fdfbeffd640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdfa80390c0 0x7fdfa803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.603+0000 7fdfbeffd640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdfa80390c0 0x7fdfa803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40098c0 tx=0x7fdfb4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:24.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.607+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfb0039350 con 0x7fdfc0106590 2024-09-10T10:36:24.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:24.779+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdfc0103680 con 0x7fdfc0106590 2024-09-10T10:36:25.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.405+0000 7fdfbcff9640 1 -- 172.21.15.143:0/2168761799 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fdfb002f300 con 0x7fdfc0106590 2024-09-10T10:36:25.406 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2024-09-10T10:36:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdfa80390c0 msgr2=0x7fdfa803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:25.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdfa80390c0 0x7fdfa803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40098c0 tx=0x7fdfb4006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 msgr2=0x7fdfc0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0102150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009230 tx=0x7fdfb00048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 shutdown_connections 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdfa80390c0 0x7fdfa803b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 --2- 172.21.15.143:0/2168761799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc0106590 0x7fdfc0102150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 >> 172.21.15.143:0/2168761799 conn(0x7fdfc00fc1a0 msgr2=0x7fdfc00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 shutdown_connections 2024-09-10T10:36:25.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:25.408+0000 7fdfc5c28640 1 -- 172.21.15.143:0/2168761799 wait complete. 2024-09-10T10:36:25.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:25 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2168761799' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T10:36:25.991 INFO:tasks.cephadm:Adding mon.a on smithi143 2024-09-10T10:36:25.991 INFO:tasks.cephadm:Adding mon.b on smithi144 2024-09-10T10:36:25.991 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-09-10T10:36:25.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c' 2024-09-10T10:36:26.241 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T10:36:26.296 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2168761799' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:36:26.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:26.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:26.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/293372271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 msgr2=0x7f6c98104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 --2- 172.21.15.178:0/293372271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98104510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009880 tx=0x7f6c8802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/293372271 shutdown_connections 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 --2- 172.21.15.178:0/293372271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98104510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/293372271 >> 172.21.15.178:0/293372271 conn(0x7f6c980fbca0 msgr2=0x7f6c980fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/293372271 shutdown_connections 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/293372271 wait complete. 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 Processor -- start 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.817+0000 7f6c9e9cf640 1 -- start start 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c9e9cf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:26.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c9e9cf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981075e0 con 0x7f6c98102120 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c97fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c97fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.178:38442/0 (socket says 172.21.15.178:38442) 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c97fff640 1 -- 172.21.15.178:0/56547276 learned_addr learned my addr 172.21.15.178:0/56547276 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c97fff640 1 -- 172.21.15.178:0/56547276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c981971f0 con 0x7f6c98102120 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c97fff640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98196cb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009230 tx=0x7f6c880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.818+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8803f070 con 0x7f6c98102120 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.819+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c88003aa0 con 0x7f6c98102120 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.819+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c98197480 con 0x7f6c98102120 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.819+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c88035030 con 0x7f6c98102120 2024-09-10T10:36:26.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.819+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9819a170 con 0x7f6c98102120 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.820+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6c88042070 con 0x7f6c98102120 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.820+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c98107d60 con 0x7f6c98102120 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.820+0000 7f6c957fa640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6c80039060 0x7f6c8003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.820+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6c8806cf90 con 0x7f6c98102120 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.820+0000 7f6c977fe640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6c80039060 0x7f6c8003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.820+0000 7f6c977fe640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6c80039060 0x7f6c8003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0098c0 tx=0x7f6c8c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:26.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:26.824+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c88039350 con 0x7f6c98102120 2024-09-10T10:36:27.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.010+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}) -- 0x7f6c9802d8f0 con 0x7f6c80039060 2024-09-10T10:36:27.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.015+0000 7f6c957fa640 1 -- 172.21.15.178:0/56547276 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6c9802d8f0 con 0x7f6c80039060 2024-09-10T10:36:27.017 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.017+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6c80039060 msgr2=0x7f6c8003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.017+0000 7f6c9e9cf640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6c80039060 0x7f6c8003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0098c0 tx=0x7f6c8c006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.017+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 msgr2=0x7f6c98196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.017+0000 7f6c9e9cf640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98196cb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009230 tx=0x7f6c880048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.017+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 shutdown_connections 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.018+0000 7f6c9e9cf640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6c80039060 0x7f6c8003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.018+0000 7f6c9e9cf640 1 --2- 172.21.15.178:0/56547276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c98102120 0x7f6c98196cb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.018+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 >> 172.21.15.178:0/56547276 conn(0x7f6c980fbca0 msgr2=0x7f6c980fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.018+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 shutdown_connections 2024-09-10T10:36:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:27.018+0000 7f6c9e9cf640 1 -- 172.21.15.178:0/56547276 wait complete. 2024-09-10T10:36:27.605 DEBUG:teuthology.orchestra.run.smithi144:mon.b> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T10:36:27.607 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T10:36:27.610 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:36:27.610 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:27.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:27.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:27.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:28.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:28 smithi143 ceph-mon[31057]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:36:28.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:28 smithi143 ceph-mon[31057]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:28.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:28 smithi143 ceph-mon[31057]: Saving service mon spec with placement smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c;count:3 2024-09-10T10:36:28.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:28 smithi143 ceph-mon[31057]: Deploying daemon mon.c on smithi178 2024-09-10T10:36:28.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:28 smithi143 ceph-mon[31057]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:28.748 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 -- 172.21.15.178:0/174855397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 msgr2=0x7f3c40107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 --2- 172.21.15.178:0/174855397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009880 tx=0x7f3c3402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 -- 172.21.15.178:0/174855397 shutdown_connections 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 --2- 172.21.15.178:0/174855397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 -- 172.21.15.178:0/174855397 >> 172.21.15.178:0/174855397 conn(0x7f3c40076250 msgr2=0x7f3c40076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 -- 172.21.15.178:0/174855397 shutdown_connections 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.084+0000 7f3c46f68640 1 -- 172.21.15.178:0/174855397 wait complete. 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c46f68640 1 Processor -- start 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c46f68640 1 -- start start 2024-09-10T10:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c46f68640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:30.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c46f68640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010bb90 con 0x7f3c40107690 2024-09-10T10:36:30.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c44d04640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:30.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c44d04640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.178:38474/0 (socket says 172.21.15.178:38474) 2024-09-10T10:36:30.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.085+0000 7f3c44d04640 1 -- 172.21.15.178:0/3789999960 learned_addr learned my addr 172.21.15.178:0/3789999960 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:36:30.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c44d04640 1 -- 172.21.15.178:0/3789999960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c40197650 con 0x7f3c40107690 2024-09-10T10:36:30.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c44d04640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009230 tx=0x7f3c340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c3403f070 con 0x7f3c40107690 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c34004d10 con 0x7f3c40107690 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401978e0 con 0x7f3c40107690 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c34035030 con 0x7f3c40107690 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.086+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c40197d40 con 0x7f3c40107690 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.087+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3c34042070 con 0x7f3c40107690 2024-09-10T10:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.087+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c40102d30 con 0x7f3c40107690 2024-09-10T10:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.087+0000 7f3c3dffb640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c30039060 0x7f3c3003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.087+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3c3406cd90 con 0x7f3c40107690 2024-09-10T10:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.088+0000 7f3c3ffff640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c30039060 0x7f3c3003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.088+0000 7f3c3ffff640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c30039060 0x7f3c3003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098c0 tx=0x7f3c2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:30.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.092+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c34042360 con 0x7f3c40107690 2024-09-10T10:36:30.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.342+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3c401984d0 con 0x7f3c40107690 2024-09-10T10:36:30.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.343+0000 7f3c3dffb640 1 -- 172.21.15.178:0/3789999960 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f3c3402fdc0 con 0x7f3c40107690 2024-09-10T10:36:30.345 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:36:30.345 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:35:05.055889Z","created":"2024-09-10T10:35:05.055889Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T10:36:30.345 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c30039060 msgr2=0x7f3c3003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c30039060 0x7f3c3003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098c0 tx=0x7f3c2c006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 msgr2=0x7f3c40197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009230 tx=0x7f3c340048f0 comp rx=0 tx=0).stop 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 shutdown_connections 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c30039060 0x7f3c3003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 --2- 172.21.15.178:0/3789999960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c40107690 0x7f3c40197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 >> 172.21.15.178:0/3789999960 conn(0x7f3c40076250 msgr2=0x7f3c400fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:30.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 shutdown_connections 2024-09-10T10:36:30.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:30.345+0000 7f3c46f68640 1 -- 172.21.15.178:0/3789999960 wait complete. 2024-09-10T10:36:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:30 smithi143 ceph-mon[31057]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:30 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/3789999960' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:31.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 podman[34792]: 2024-09-10 10:36:31.548990639 +0000 UTC m=+0.356334899 container start 4cf4bb79c3bbbc12d4c5845abdfd4dca3338ee0338e0e994d40249f7c02c6329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T10:36:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T10:36:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T10:36:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: pidfile_write: ignore empty --pid-file 2024-09-10T10:36:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: load: jerasure load: lrc 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: RocksDB version: 7.9.2 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Git sha 0 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: DB SUMMARY 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: DB Session ID: 5XXSIDSGGBHJLEC4W6J6 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: CURRENT file: CURRENT 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.error_if_exists: 0 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.create_if_missing: 0 2024-09-10T10:36:31.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.paranoid_checks: 1 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.env: 0x5622039d2940 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.info_log: 0x5622056516e0 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.statistics: (nil) 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.use_fsync: 0 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_log_file_size: 0 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T10:36:31.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.allow_fallocate: 1 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.use_direct_reads: 0 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.db_log_dir: 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.wal_dir: 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T10:36:31.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.write_buffer_manager: 0x562205660dc0 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T10:36:31.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.unordered_write: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.row_cache: None 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.wal_filter: None 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.two_write_queues: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T10:36:31.930 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.wal_compression: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.atomic_flush: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.log_readahead_size: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T10:36:31.931 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_background_jobs: 2 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_background_compactions: -1 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_subcompactions: 1 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T10:36:31.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T10:36:31.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T10:36:31.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T10:36:31.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T10:36:31.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_open_files: -1 2024-09-10T10:36:31.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T10:36:31.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_background_flushes: -1 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Compression algorithms supported: 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kZSTD supported: 0 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kXpressCompression supported: 0 2024-09-10T10:36:31.935 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kBZip2Compression supported: 0 2024-09-10T10:36:31.936 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T10:36:31.936 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kLZ4Compression supported: 1 2024-09-10T10:36:31.936 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kZlibCompression supported: 1 2024-09-10T10:36:31.936 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T10:36:31.936 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: kSnappyCompression supported: 1 2024-09-10T10:36:31.936 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T10:36:31.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T10:36:31.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T10:36:31.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T10:36:31.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T10:36:31.937 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.merge_operator: 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_filter: None 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562205651320) 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_type: 0 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_index_type: 0 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_shortening: 1 2024-09-10T10:36:31.938 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: checksum: 4 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: no_block_cache: 0 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache: 0x5622056451f0 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_options: 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: capacity : 536870912 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_shard_bits : 4 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: strict_capacity_limit : 0 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_compressed: (nil) 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: persistent_cache: (nil) 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size: 4096 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size_deviation: 10 2024-09-10T10:36:31.939 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_restart_interval: 16 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_block_restart_interval: 1 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: metadata_block_size: 4096 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: partition_filters: 0 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: use_delta_encoding: 1 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: filter_policy: bloomfilter 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: whole_key_filtering: 1 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: verify_compression: 0 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: format_version: 5 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_index_compression: 1 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_align: 0 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: max_auto_readahead_size: 262144 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: prepopulate_block_cache: 0 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: initial_auto_readahead_size: 8192 2024-09-10T10:36:31.940 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression: NoCompression 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.num_levels: 7 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T10:36:31.941 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T10:36:31.942 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T10:36:31.943 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T10:36:31.944 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.inplace_update_support: 0 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T10:36:31.945 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.bloom_locality: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.max_successive_merges: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.ttl: 2592000 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.enable_blob_files: false 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.min_blob_size: 0 2024-09-10T10:36:31.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T10:36:31.947 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 25264262-0bb1-4d93-ae4f-6a0c51716c85 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964591578309, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964591578883, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725964591, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "25264262-0bb1-4d93-ae4f-6a0c51716c85", "db_session_id": "5XXSIDSGGBHJLEC4W6J6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964591579010, "job": 1, "event": "recovery_finished"} 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562205672e00 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: DB pointer 0x56220576e000 2024-09-10T10:36:31.948 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** DB Stats ** 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T10:36:31.949 INFO:journalctl@ceph.mon.c.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T10:36:31.950 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval compaction: 0.00 GB write, 0.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache BinnedLRUCache@0x5622056451f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: starting mon.c rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:31.951 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(???) e0 preinit fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: e1 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: btime 2024-09-10T10:35:07:959271+0000 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: legacy client fscid: -1 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout: No filesystems configured 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T10:36:31.952 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2168761799' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2168761799' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:31.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: Saving service mon spec with placement smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c;count:3 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: Deploying daemon mon.c on smithi178 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/3789999960' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 ceph-mon[34813]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 bash[34792]: 4cf4bb79c3bbbc12d4c5845abdfd4dca3338ee0338e0e994d40249f7c02c6329 2024-09-10T10:36:31.955 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:31 smithi178 systemd[1]: Started Ceph mon.c for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:32.144 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:36:32.145 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:36:32.405 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:36:36.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:36.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: mon.a calling monitor election 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: mon.c calling monitor election 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: monmap epoch 2 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: last_changed 2024-09-10T10:36:31.605008+0000 2024-09-10T10:36:36.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: min_mon_release 19 (squid) 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: election_strategy: 1 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: fsmap 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: mgrmap e13: x(active, since 37s) 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: overall HEALTH_OK 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:36.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:37.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:37.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:37.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: mon.a calling monitor election 2024-09-10T10:36:37.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: mon.c calling monitor election 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: monmap epoch 2 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: last_changed 2024-09-10T10:36:31.605008+0000 2024-09-10T10:36:37.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: min_mon_release 19 (squid) 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: election_strategy: 1 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: fsmap 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: mgrmap e13: x(active, since 37s) 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: overall HEALTH_OK 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:37.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 -- 172.21.15.178:0/214871336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 msgr2=0x7fd9e00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 --2- 172.21.15.178:0/214871336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9e00059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9f41016a0 tx=0x7fd9f002bfe0 comp rx=0 tx=0).stop 2024-09-10T10:36:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 -- 172.21.15.178:0/214871336 shutdown_connections 2024-09-10T10:36:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 --2- 172.21.15.178:0/214871336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9e00059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 -- 172.21.15.178:0/214871336 >> 172.21.15.178:0/214871336 conn(0x7fd9f40fc220 msgr2=0x7fd9f40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:37.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 -- 172.21.15.178:0/214871336 shutdown_connections 2024-09-10T10:36:37.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.224+0000 7fd9fc501640 1 -- 172.21.15.178:0/214871336 wait complete. 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.225+0000 7fd9fc501640 1 Processor -- start 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.225+0000 7fd9fc501640 1 -- start start 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.225+0000 7fd9fc501640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9f4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.225+0000 7fd9fc501640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.225+0000 7fd9fc501640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4108520 con 0x7fd9e0003530 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.225+0000 7fd9fc501640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41083a0 con 0x7fd9f4196340 2024-09-10T10:36:37.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33618/0 (socket says 172.21.15.178:33618) 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9fa29d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9f4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 -- 172.21.15.178:0/1896611515 learned_addr learned my addr 172.21.15.178:0/1896611515 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 -- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 msgr2=0x7fd9f41967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 -- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 msgr2=0x7fd9f41967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9f9a9c640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9fa29d640 1 -- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 msgr2=0x7fd9f41967a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9fa29d640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9fa29d640 1 -- 172.21.15.178:0/1896611515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f406b230 con 0x7fd9e0003530 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.226+0000 7fd9fa29d640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9f4195e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0036040 tx=0x7fd9f0004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.227+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd9f00348a0 con 0x7fd9e0003530 2024-09-10T10:36:37.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.227+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9f0042070 con 0x7fd9e0003530 2024-09-10T10:36:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.227+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41a3470 con 0x7fd9e0003530 2024-09-10T10:36:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.227+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd9f003d480 con 0x7fd9e0003530 2024-09-10T10:36:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.227+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41a39a0 con 0x7fd9e0003530 2024-09-10T10:36:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.228+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd9f003d900 con 0x7fd9e0003530 2024-09-10T10:36:37.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.228+0000 7fd9eb7fe640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd9d8038e50 0x7fd9d803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:37.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.228+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd9f006e220 con 0x7fd9e0003530 2024-09-10T10:36:37.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.228+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f4101ca0 con 0x7fd9e0003530 2024-09-10T10:36:37.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.228+0000 7fd9f9a9c640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd9d8038e50 0x7fd9d803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:37.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.229+0000 7fd9f9a9c640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd9d8038e50 0x7fd9d803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009100 tx=0x7fd9e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.234+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9f0045360 con 0x7fd9e0003530 2024-09-10T10:36:37.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.468+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd9f406aac0 con 0x7fd9e0003530 2024-09-10T10:36:37.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.469+0000 7fd9eb7fe640 1 -- 172.21.15.178:0/1896611515 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fd9f0043270 con 0x7fd9e0003530 2024-09-10T10:36:37.470 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:36:37.470 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:36:31.605008Z","created":"2024-09-10T10:35:05.055889Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T10:36:37.471 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-10T10:36:37.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd9d8038e50 msgr2=0x7fd9d803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:37.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd9d8038e50 0x7fd9d803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009100 tx=0x7fd9e4008040 comp rx=0 tx=0).stop 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 msgr2=0x7fd9f4195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9f4195e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0036040 tx=0x7fd9f0004c10 comp rx=0 tx=0).stop 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 shutdown_connections 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd9d8038e50 0x7fd9d803b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9f4196340 0x7fd9f41967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 --2- 172.21.15.178:0/1896611515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9e0003530 0x7fd9f4195e00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 >> 172.21.15.178:0/1896611515 conn(0x7fd9f40fc220 msgr2=0x7fd9f40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 shutdown_connections 2024-09-10T10:36:37.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:37.471+0000 7fd9fc501640 1 -- 172.21.15.178:0/1896611515 wait complete. 2024-09-10T10:36:37.819 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:37 smithi178 ceph-mon[34813]: Deploying daemon mon.b on smithi144 2024-09-10T10:36:37.820 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:37 smithi178 ceph-mon[34813]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:37.820 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:37 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/1896611515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:37.820 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:37 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:38.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:37 smithi143 ceph-mon[31057]: Deploying daemon mon.b on smithi144 2024-09-10T10:36:38.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:37 smithi143 ceph-mon[31057]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:38.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:37 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/1896611515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:38.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:37 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:39.099 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:36:39.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:36:39.351 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:36:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.792+0000 7fc8b6474640 1 -- 172.21.15.178:0/435485059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 msgr2=0x7fc8b0106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.792+0000 7fc8b6474640 1 --2- 172.21.15.178:0/435485059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b0106a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009880 tx=0x7fc8a002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 -- 172.21.15.178:0/435485059 shutdown_connections 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 --2- 172.21.15.178:0/435485059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b0106a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 -- 172.21.15.178:0/435485059 >> 172.21.15.178:0/435485059 conn(0x7fc8b00fc280 msgr2=0x7fc8b00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 -- 172.21.15.178:0/435485059 shutdown_connections 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 -- 172.21.15.178:0/435485059 wait complete. 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 Processor -- start 2024-09-10T10:36:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.793+0000 7fc8b6474640 1 -- start start 2024-09-10T10:36:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8b6474640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8b6474640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8b6474640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010ab30 con 0x7fc8b0106650 2024-09-10T10:36:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8b6474640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010a9b0 con 0x7fc8b0198610 2024-09-10T10:36:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33634/0 (socket says 172.21.15.178:33634) 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 -- 172.21.15.178:0/330431890 learned_addr learned my addr 172.21.15.178:0/330431890 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8affff640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 -- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 msgr2=0x7fc8b0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 -- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 msgr2=0x7fc8b0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8af7fe640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8affff640 1 -- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 msgr2=0x7fc8b0198a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8affff640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8affff640 1 -- 172.21.15.178:0/330431890 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b01a1140 con 0x7fc8b0106650 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.794+0000 7fc8affff640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b01980d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0006010 tx=0x7fc8a0004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.795+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc8a0035030 con 0x7fc8b0106650 2024-09-10T10:36:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.795+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8a00053a0 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.795+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b01a1310 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.795+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc8a0031830 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.795+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b01a17c0 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.796+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc8a0042070 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.796+0000 7fc8ad7fa640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc898038e00 0x7fc89803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.796+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc8a006cd60 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.796+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b0101cc0 con 0x7fc8b0106650 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.796+0000 7fc8af7fe640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc898038e00 0x7fc89803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.797+0000 7fc8af7fe640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc898038e00 0x7fc89803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009100 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:39.801+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8a0039350 con 0x7fc8b0106650 2024-09-10T10:36:40.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.028+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc8b01a1a70 con 0x7fc8b0106650 2024-09-10T10:36:40.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.029+0000 7fc8ad7fa640 1 -- 172.21.15.178:0/330431890 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fc8a002f300 con 0x7fc8b0106650 2024-09-10T10:36:40.030 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:36:40.031 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:36:31.605008Z","created":"2024-09-10T10:35:05.055889Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T10:36:40.031 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc898038e00 msgr2=0x7fc89803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc898038e00 0x7fc89803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009100 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 msgr2=0x7fc8b01980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b01980d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0006010 tx=0x7fc8a0004e40 comp rx=0 tx=0).stop 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 shutdown_connections 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc898038e00 0x7fc89803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8b0198610 0x7fc8b0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:40.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 --2- 172.21.15.178:0/330431890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8b0106650 0x7fc8b01980d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:40.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 >> 172.21.15.178:0/330431890 conn(0x7fc8b00fc280 msgr2=0x7fc8b00fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:40.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 shutdown_connections 2024-09-10T10:36:40.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:40.031+0000 7fc8b6474640 1 -- 172.21.15.178:0/330431890 wait complete. 2024-09-10T10:36:40.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:40 smithi144 systemd[1]: Started Ceph mon.b for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:41.597 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:36:41.597 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:36:41.849 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.451+0000 7fcaff373640 1 -- 172.21.15.178:0/3089825128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcae0003530 msgr2=0x7fcae00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.451+0000 7fcaff373640 1 --2- 172.21.15.178:0/3089825128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcae0003530 0x7fcae00059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8101170 tx=0x7fcaf002cd90 comp rx=0 tx=0).stop 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.451+0000 7fcaff373640 1 -- 172.21.15.178:0/3089825128 shutdown_connections 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.451+0000 7fcaff373640 1 --2- 172.21.15.178:0/3089825128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcae0003530 0x7fcae00059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.451+0000 7fcaff373640 1 -- 172.21.15.178:0/3089825128 >> 172.21.15.178:0/3089825128 conn(0x7fcaf80fb9a0 msgr2=0x7fcaf80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.452+0000 7fcaff373640 1 -- 172.21.15.178:0/3089825128 shutdown_connections 2024-09-10T10:36:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.452+0000 7fcaff373640 1 -- 172.21.15.178:0/3089825128 wait complete. 2024-09-10T10:36:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.452+0000 7fcaff373640 1 Processor -- start 2024-09-10T10:36:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.452+0000 7fcaff373640 1 -- start start 2024-09-10T10:36:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcaff373640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 0x7fcaf81972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcaff373640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcaf8197810 0x7fcaf819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcaff373640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcaff373640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8107430 con 0x7fcaf8197810 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcaff373640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81072b0 con 0x7fcae0003530 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcaff373640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81075b0 con 0x7fcaf819c0e0 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56206/0 (socket says 172.21.15.178:56206) 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 -- 172.21.15.178:0/3940885579 learned_addr learned my addr 172.21.15.178:0/3940885579 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafc90e640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcaf8197810 0x7fcaf819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd10f640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 0x7fcaf81972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 msgr2=0x7fcaf819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 msgr2=0x7fcaf819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd10f640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 msgr2=0x7fcaf81972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T10:36:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd10f640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 msgr2=0x7fcaf81972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd10f640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 0x7fcaf81972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd910640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafd10f640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 0x7fcaf81972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafc90e640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 msgr2=0x7fcaf819c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafc90e640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafc90e640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 msgr2=0x7fcaf81972d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafc90e640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 0x7fcaf81972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.453+0000 7fcafc90e640 1 -- 172.21.15.178:0/3940885579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81a3e50 con 0x7fcaf8197810 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.454+0000 7fcafc90e640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcaf8197810 0x7fcaf819bba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b9b0 tx=0x7fcaec00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.454+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00cba0 con 0x7fcaf8197810 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.454+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcaec003e40 con 0x7fcaf8197810 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.454+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec014430 con 0x7fcaf8197810 2024-09-10T10:36:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.454+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf81a4080 con 0x7fcaf8197810 2024-09-10T10:36:45.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.454+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81a4590 con 0x7fcaf8197810 2024-09-10T10:36:45.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.455+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcaec0145d0 con 0x7fcaf8197810 2024-09-10T10:36:45.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.455+0000 7fcaf67fc640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcae40394f0 0x7fcae403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:45.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.455+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf81084c0 con 0x7fcaf8197810 2024-09-10T10:36:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.455+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcaec04bbd0 con 0x7fcaf8197810 2024-09-10T10:36:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.455+0000 7fcafd10f640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcae40394f0 0x7fcae403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.456+0000 7fcafd10f640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcae40394f0 0x7fcae403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005c80 tx=0x7fcaf0016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:45.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.461+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcaec004000 con 0x7fcaf8197810 2024-09-10T10:36:45.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: mon.a calling monitor election 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: mon.c calling monitor election 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: mon.b calling monitor election 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: monmap epoch 3 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: last_changed 2024-09-10T10:36:40.254748+0000 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: min_mon_release 19 (squid) 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: election_strategy: 1 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: fsmap 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: mgrmap e13: x(active, since 45s) 2024-09-10T10:36:45.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: overall HEALTH_OK 2024-09-10T10:36:45.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: mon.a calling monitor election 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: mon.c calling monitor election 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: mon.b calling monitor election 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: monmap epoch 3 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: last_changed 2024-09-10T10:36:40.254748+0000 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: min_mon_release 19 (squid) 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: election_strategy: 1 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T10:36:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: fsmap 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: mgrmap e13: x(active, since 45s) 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: overall HEALTH_OK 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.688 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.712+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcaf81a4890 con 0x7fcaf8197810 2024-09-10T10:36:45.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.712+0000 7fcaf67fc640 1 -- 172.21.15.178:0/3940885579 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7fcaec0112a0 con 0x7fcaf8197810 2024-09-10T10:36:45.714 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:36:45.714 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:36:40.254748Z","created":"2024-09-10T10:35:05.055889Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T10:36:45.714 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcae40394f0 msgr2=0x7fcae403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcae40394f0 0x7fcae403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005c80 tx=0x7fcaf0016000 comp rx=0 tx=0).stop 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcaf8197810 msgr2=0x7fcaf819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcaf8197810 0x7fcaf819bba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b9b0 tx=0x7fcaec00be70 comp rx=0 tx=0).stop 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 shutdown_connections 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcae40394f0 0x7fcae403b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf819c0e0 0x7fcaf819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcaf8197810 0x7fcaf819bba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 --2- 172.21.15.178:0/3940885579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0003530 0x7fcaf81972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:45.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 >> 172.21.15.178:0/3940885579 conn(0x7fcaf80fb9a0 msgr2=0x7fcaf80fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:45.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 shutdown_connections 2024-09-10T10:36:45.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:45.715+0000 7fcaff373640 1 -- 172.21.15.178:0/3940885579 wait complete. 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: mon.a calling monitor election 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: mon.c calling monitor election 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: mon.b calling monitor election 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: monmap epoch 3 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: last_changed 2024-09-10T10:36:40.254748+0000 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: created 2024-09-10T10:35:05.055889+0000 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: min_mon_release 19 (squid) 2024-09-10T10:36:45.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: election_strategy: 1 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: 0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: fsmap 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: mgrmap e13: x(active, since 45s) 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: overall HEALTH_OK 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:45.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:46.351 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T10:36:46.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T10:36:46.608 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:46.612 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:46 smithi178 ceph-mon[34813]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:46.612 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:46.612 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:46 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/3940885579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:46.612 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:46 smithi144 ceph-mon[34230]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:46 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/3940885579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:46.729 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:46 smithi143 ceph-mon[31057]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:46.729 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:46.729 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:46 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/3940885579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:36:46.729 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:36:47.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 -- 172.21.15.143:0/2295669146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1720107690 msgr2=0x7f1720107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 --2- 172.21.15.143:0/2295669146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1720107690 0x7f1720107a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1710009880 tx=0x7f171002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 -- 172.21.15.143:0/2295669146 shutdown_connections 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 --2- 172.21.15.143:0/2295669146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1720107690 0x7f1720107a70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 -- 172.21.15.143:0/2295669146 >> 172.21.15.143:0/2295669146 conn(0x7f1720076250 msgr2=0x7f1720076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 -- 172.21.15.143:0/2295669146 shutdown_connections 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.164+0000 7f1725ad3640 1 -- 172.21.15.143:0/2295669146 wait complete. 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.165+0000 7f1725ad3640 1 Processor -- start 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.165+0000 7f1725ad3640 1 -- start start 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.165+0000 7f1725ad3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1720107690 0x7f17201972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.165+0000 7f1725ad3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1720197800 0x7f172019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.165+0000 7f1725ad3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172019c0d0 0x7f172019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f1725ad3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f172010bb90 con 0x7f172019c0d0 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f1725ad3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f172010ba10 con 0x7f1720107690 2024-09-10T10:36:47.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f1725ad3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f172010bd10 con 0x7f1720197800 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172019c0d0 0x7f172019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1720107690 0x7f17201972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172019c0d0 0x7f172019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34490/0 (socket says 172.21.15.143:34490) 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171ffff640 1 -- 172.21.15.143:0/3152575668 learned_addr learned my addr 172.21.15.143:0/3152575668 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171effd640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1720197800 0x7f172019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 -- 172.21.15.143:0/3152575668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1720197800 msgr2=0x7f172019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1720197800 0x7f172019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 -- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172019c0d0 msgr2=0x7f172019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172019c0d0 0x7f172019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 -- 172.21.15.143:0/3152575668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17201a3e40 con 0x7f1720107690 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171f7fe640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1720107690 0x7f17201972c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f1710003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:47.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.166+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171003f070 con 0x7f1720107690 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.167+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1710031d60 con 0x7f1720107690 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.167+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1720100730 con 0x7f1720107690 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.167+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710035030 con 0x7f1720107690 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.167+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1720100ce0 con 0x7f1720107690 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.168+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1710042070 con 0x7f1720107690 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.168+0000 7f171cff9640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17080394f0 0x7f170803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.168+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1720107af0 con 0x7f1720107690 2024-09-10T10:36:47.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.168+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f171006cc30 con 0x7f1720107690 2024-09-10T10:36:47.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.168+0000 7f171effd640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17080394f0 0x7f170803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.169+0000 7f171effd640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17080394f0 0x7f170803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1714009a30 tx=0x7f1714009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:47.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.173+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f171003bdb0 con 0x7f1720107690 2024-09-10T10:36:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.357+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1720100f40 con 0x7f1720107690 2024-09-10T10:36:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.358+0000 7f171cff9640 1 -- 172.21.15.143:0/3152575668 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f171003a8e0 con 0x7f1720107690 2024-09-10T10:36:47.360 INFO:teuthology.orchestra.run.smithi143.stdout:# minimal ceph.conf for 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:47.360 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2024-09-10T10:36:47.360 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:47.361 INFO:teuthology.orchestra.run.smithi143.stdout: mon_host = [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.360+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17080394f0 msgr2=0x7f170803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.360+0000 7f1725ad3640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17080394f0 0x7f170803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1714009a30 tx=0x7f1714009190 comp rx=0 tx=0).stop 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.360+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1720107690 msgr2=0x7f17201972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.360+0000 7f1725ad3640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1720107690 0x7f17201972c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f1710003cd0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 shutdown_connections 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17080394f0 0x7f170803b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f172019c0d0 0x7f172019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1720197800 0x7f172019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 --2- 172.21.15.143:0/3152575668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1720107690 0x7f17201972c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 >> 172.21.15.143:0/3152575668 conn(0x7f1720076250 msgr2=0x7f17200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 shutdown_connections 2024-09-10T10:36:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:47.361+0000 7f1725ad3640 1 -- 172.21.15.143:0/3152575668 wait complete. 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:47 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3152575668' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:47.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:47 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3152575668' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:47.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:47 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3152575668' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:47.995 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T10:36:47.995 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:47.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T10:36:48.024 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:48.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:48.089 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:36:48.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T10:36:48.118 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:36:48.118 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:48.184 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:36:48.184 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T10:36:48.212 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:36:48.212 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:48.279 INFO:tasks.cephadm:Adding mgr.x on smithi143 2024-09-10T10:36:48.279 INFO:tasks.cephadm:Adding mgr.y on smithi144 2024-09-10T10:36:48.279 INFO:tasks.cephadm:Adding mgr.z on smithi178 2024-09-10T10:36:48.279 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi143=x;smithi144=y;smithi178=z' 2024-09-10T10:36:48.555 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: Updating smithi144:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: Updating smithi143:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:48.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: Updating smithi144:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: Updating smithi143:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:48.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: Updating smithi143:/etc/ceph/ceph.conf 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: Updating smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: Updating smithi144:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: Updating smithi143:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:36:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:48.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:48.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.021+0000 7f29e0a32640 1 -- 172.21.15.178:0/3267993699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc1016e0 msgr2=0x7f29dc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.021+0000 7f29e0a32640 1 --2- 172.21.15.178:0/3267993699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc1016e0 0x7f29dc101ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009880 tx=0x7f29cc02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.021+0000 7f29e0a32640 1 -- 172.21.15.178:0/3267993699 shutdown_connections 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.021+0000 7f29e0a32640 1 --2- 172.21.15.178:0/3267993699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc1016e0 0x7f29dc101ae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.021+0000 7f29e0a32640 1 -- 172.21.15.178:0/3267993699 >> 172.21.15.178:0/3267993699 conn(0x7f29dc0fce90 msgr2=0x7f29dc0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 -- 172.21.15.178:0/3267993699 shutdown_connections 2024-09-10T10:36:49.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 -- 172.21.15.178:0/3267993699 wait complete. 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 Processor -- start 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 -- start start 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29dc1016e0 0x7f29dc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 0x7f29dc19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc197890 0x7f29dc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc1076a0 con 0x7f29dc197890 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc107520 con 0x7f29dc1016e0 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.022+0000 7f29e0a32640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc107820 con 0x7f29dc197410 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 0x7f29dc19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 0x7f29dc19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56214/0 (socket says 172.21.15.178:56214) 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 -- 172.21.15.178:0/1759628479 learned_addr learned my addr 172.21.15.178:0/1759628479 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29da59c640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29dc1016e0 0x7f29dc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29dad9d640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc197890 0x7f29dc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 -- 172.21.15.178:0/1759628479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29dc1016e0 msgr2=0x7f29dc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29dc1016e0 0x7f29dc196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 -- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc197890 msgr2=0x7f29dc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc197890 0x7f29dc19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 -- 172.21.15.178:0/1759628479 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc19f2d0 con 0x7f29dc197410 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d3fff640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 0x7f29dc19c480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29c400b520 tx=0x7f29c400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4004230 con 0x7f29dc197410 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29c40043d0 con 0x7f29dc197410 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc19f5c0 con 0x7f29dc197410 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4003cf0 con 0x7f29dc197410 2024-09-10T10:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.023+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc19fbd0 con 0x7f29dc197410 2024-09-10T10:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.024+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f29c400e070 con 0x7f29dc197410 2024-09-10T10:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.025+0000 7f29d37fe640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f29c80394f0 0x7f29c803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.025+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc190460 con 0x7f29dc197410 2024-09-10T10:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.025+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f29c404b960 con 0x7f29dc197410 2024-09-10T10:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.025+0000 7f29da59c640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f29c80394f0 0x7f29c803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.025+0000 7f29da59c640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f29c80394f0 0x7f29c803b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:49.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.029+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29c4014020 con 0x7f29dc197410 2024-09-10T10:36:49.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.205+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi143=x;smithi144=y;smithi178=z", "target": ["mon-mgr", ""]}) -- 0x7f29dc0626c0 con 0x7f29c80394f0 2024-09-10T10:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.210+0000 7f29d37fe640 1 -- 172.21.15.178:0/1759628479 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f29dc0626c0 con 0x7f29c80394f0 2024-09-10T10:36:49.211 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f29c80394f0 msgr2=0x7f29c803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f29c80394f0 0x7f29c803b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc0314a0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 msgr2=0x7f29dc19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 0x7f29dc19c480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29c400b520 tx=0x7f29c400b9e0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 shutdown_connections 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f29c80394f0 0x7f29c803b9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29dc197890 0x7f29dc19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29dc197410 0x7f29dc19c480 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 --2- 172.21.15.178:0/1759628479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29dc1016e0 0x7f29dc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 >> 172.21.15.178:0/1759628479 conn(0x7f29dc0fce90 msgr2=0x7f29dc0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 shutdown_connections 2024-09-10T10:36:49.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:36:49.212+0000 7f29e0a32640 1 -- 172.21.15.178:0/1759628479 wait complete. 2024-09-10T10:36:49.655 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T10:36:49.655 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: Reconfiguring daemon mon.a on smithi143 2024-09-10T10:36:49.655 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.655 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.655 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:49.655 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:49.656 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:49.656 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.656 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T10:36:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: Reconfiguring daemon mon.a on smithi143 2024-09-10T10:36:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:49.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:49.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:49.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: Reconfiguring daemon mon.a on smithi143 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:49.810 DEBUG:teuthology.orchestra.run.smithi144:mgr.y> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T10:36:49.812 DEBUG:teuthology.orchestra.run.smithi178:mgr.z> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T10:36:49.815 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T10:36:49.815 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:36:49.815 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:36:49.833 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:36:49.834 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-10 10:36:10.278791524 +0000 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-10 10:32:00.673095536 +0000 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-10 10:32:00.673095536 +0000 2024-09-10T10:36:49.892 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-10 10:32:00.673095536 +0000 2024-09-10T10:36:49.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:36:49.959 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-10T10:36:49.959 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-10T10:36:49.959 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000201982 s, 2.5 MB/s 2024-09-10T10:36:49.960 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:36:50.019 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-09-10T10:36:50.076 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:36:50.076 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:50.076 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T10:36:50.076 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:50.076 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:50.077 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-10 10:36:10.311790651 +0000 2024-09-10T10:36:50.077 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-10 10:32:01.023086076 +0000 2024-09-10T10:36:50.077 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-10 10:32:01.023086076 +0000 2024-09-10T10:36:50.077 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-10 10:32:01.023086076 +0000 2024-09-10T10:36:50.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:36:50.140 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-10T10:36:50.141 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-10T10:36:50.141 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.00020937 s, 2.4 MB/s 2024-09-10T10:36:50.142 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:36:50.198 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-10 10:36:10.344789778 +0000 2024-09-10T10:36:50.255 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-10 10:32:01.371076670 +0000 2024-09-10T10:36:50.256 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-10 10:32:01.371076670 +0000 2024-09-10T10:36:50.256 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-10 10:32:01.371076670 +0000 2024-09-10T10:36:50.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:36:50.319 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-10T10:36:50.319 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-10T10:36:50.319 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000208784 s, 2.5 MB/s 2024-09-10T10:36:50.320 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:36:50.377 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-09-10T10:36:50.433 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-10 10:36:10.379788852 +0000 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-10 10:32:01.705067642 +0000 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-10 10:32:01.705067642 +0000 2024-09-10T10:36:50.434 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-09-10 10:32:01.705067642 +0000 2024-09-10T10:36:50.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:36:50.497 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-09-10T10:36:50.498 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-09-10T10:36:50.498 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000182873 s, 2.8 MB/s 2024-09-10T10:36:50.499 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:36:50.555 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T10:36:50.555 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:36:50.572 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:36:50.572 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 10:36:39.719603117 +0000 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 10:32:01.635101032 +0000 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 10:32:01.635101032 +0000 2024-09-10T10:36:50.630 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 10:32:01.635101032 +0000 2024-09-10T10:36:50.631 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:36:50.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: Reconfiguring mon.b (monmap changed)... 2024-09-10T10:36:50.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: Reconfiguring daemon mon.b on smithi144 2024-09-10T10:36:50.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi143=x;smithi144=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:50.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: Saving service mgr spec with placement smithi143=x;smithi144=y;smithi178=z;count:3 2024-09-10T10:36:50.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:50.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:50.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:50.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:50.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:50.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:50.658 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T10:36:50.658 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T10:36:50.659 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.00024902 s, 2.1 MB/s 2024-09-10T10:36:50.660 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:36:50.717 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 10:36:39.719603117 +0000 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 10:32:01.982095185 +0000 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 10:32:01.982095185 +0000 2024-09-10T10:36:50.772 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 10:32:01.982095185 +0000 2024-09-10T10:36:50.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:36:50.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: Reconfiguring mon.b (monmap changed)... 2024-09-10T10:36:50.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: Reconfiguring daemon mon.b on smithi144 2024-09-10T10:36:50.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi143=x;smithi144=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:50.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: Saving service mgr spec with placement smithi143=x;smithi144=y;smithi178=z;count:3 2024-09-10T10:36:50.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:50.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:50.779 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:50.779 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:50.779 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:50.779 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:50.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: Reconfiguring mon.b (monmap changed)... 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: Reconfiguring daemon mon.b on smithi144 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi143=x;smithi144=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: Saving service mgr spec with placement smithi143=x;smithi144=y;smithi178=z;count:3 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:36:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:50.835 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T10:36:50.835 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T10:36:50.835 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000189219 s, 2.7 MB/s 2024-09-10T10:36:50.836 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:36:50.893 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 10:36:39.720603100 +0000 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 10:32:02.325089405 +0000 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 10:32:02.325089405 +0000 2024-09-10T10:36:50.949 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 10:32:02.325089405 +0000 2024-09-10T10:36:50.950 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:36:51.012 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T10:36:51.012 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T10:36:51.012 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000259711 s, 2.0 MB/s 2024-09-10T10:36:51.014 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:36:51.070 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-09-10T10:36:51.125 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:36:51.125 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:51.125 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T10:36:51.126 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:51.126 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:51.126 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 10:36:39.720603100 +0000 2024-09-10T10:36:51.126 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 10:32:02.648083963 +0000 2024-09-10T10:36:51.126 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 10:32:02.648083963 +0000 2024-09-10T10:36:51.126 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 10:32:02.648083963 +0000 2024-09-10T10:36:51.126 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:36:51.190 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T10:36:51.190 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T10:36:51.190 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000204499 s, 2.5 MB/s 2024-09-10T10:36:51.191 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:36:51.247 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T10:36:51.247 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:36:51.264 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:36:51.265 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 10:36:30.882427176 +0000 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 10:32:01.934100041 +0000 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 10:32:01.934100041 +0000 2024-09-10T10:36:51.320 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 10:32:01.934100041 +0000 2024-09-10T10:36:51.321 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:36:51.383 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T10:36:51.383 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T10:36:51.384 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000110329 s, 4.6 MB/s 2024-09-10T10:36:51.385 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:36:51.442 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-09-10T10:36:51.497 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:36:51.497 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:51.497 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T10:36:51.497 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:51.498 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:51.498 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 10:36:30.883427158 +0000 2024-09-10T10:36:51.498 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 10:32:02.278093977 +0000 2024-09-10T10:36:51.498 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 10:32:02.278093977 +0000 2024-09-10T10:36:51.498 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 10:32:02.278093977 +0000 2024-09-10T10:36:51.498 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:36:51.563 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T10:36:51.564 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T10:36:51.564 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 8.71e-05 s, 5.9 MB/s 2024-09-10T10:36:51.565 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:36:51.620 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-09-10T10:36:51.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:51 smithi178 ceph-mon[34813]: Reconfiguring mon.c (monmap changed)... 2024-09-10T10:36:51.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:51 smithi178 ceph-mon[34813]: Reconfiguring daemon mon.c on smithi178 2024-09-10T10:36:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:36:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:51.677 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T10:36:51.677 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:51.677 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:51.677 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 10:36:30.883427158 +0000 2024-09-10T10:36:51.678 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 10:32:02.592088442 +0000 2024-09-10T10:36:51.678 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 10:32:02.592088442 +0000 2024-09-10T10:36:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 10:32:02.592088442 +0000 2024-09-10T10:36:51.678 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:36:51.741 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T10:36:51.741 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T10:36:51.741 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000115073 s, 4.4 MB/s 2024-09-10T10:36:51.743 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:36:51.800 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-09-10T10:36:51.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:51 smithi143 ceph-mon[31057]: Reconfiguring mon.c (monmap changed)... 2024-09-10T10:36:51.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:51 smithi143 ceph-mon[31057]: Reconfiguring daemon mon.c on smithi178 2024-09-10T10:36:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 10:36:30.883427158 +0000 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 10:32:02.914082766 +0000 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 10:32:02.914082766 +0000 2024-09-10T10:36:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 10:32:02.914082766 +0000 2024-09-10T10:36:51.856 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:36:51.919 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T10:36:51.919 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T10:36:51.919 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000110549 s, 4.6 MB/s 2024-09-10T10:36:51.920 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:36:51.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:51 smithi144 ceph-mon[34230]: Reconfiguring mon.c (monmap changed)... 2024-09-10T10:36:51.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:51 smithi144 ceph-mon[34230]: Reconfiguring daemon mon.c on smithi178 2024-09-10T10:36:51.977 INFO:tasks.cephadm:Deploying osd.0 on smithi143 with /dev/vg_nvme/lv_4... 2024-09-10T10:36:51.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T10:36:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:52.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:52.686 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:52.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:53.425 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 systemd[1]: Starting Ceph mgr.z for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:36:53.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:53 smithi143 ceph-mon[31057]: Deploying daemon mgr.z on smithi178 2024-09-10T10:36:53.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:53 smithi178 ceph-mon[34813]: Deploying daemon mgr.z on smithi178 2024-09-10T10:36:53.749 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 podman[36612]: 2024-09-10 10:36:53.427975372 +0000 UTC m=+0.016240132 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:36:53.749 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 podman[36612]: 2024-09-10 10:36:53.545456442 +0000 UTC m=+0.133721196 container create 0a2a6923d0d59979e50e28b27fdf0d48023e030dc50c3b56f2b3fa3e8ddd22e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-10T10:36:53.749 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 podman[36612]: 2024-09-10 10:36:53.74804788 +0000 UTC m=+0.336312634 container init 0a2a6923d0d59979e50e28b27fdf0d48023e030dc50c3b56f2b3fa3e8ddd22e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T10:36:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:53 smithi144 ceph-mon[34230]: Deploying daemon mgr.z on smithi178 2024-09-10T10:36:54.080 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 podman[36612]: 2024-09-10 10:36:53.750926062 +0000 UTC m=+0.339190812 container start 0a2a6923d0d59979e50e28b27fdf0d48023e030dc50c3b56f2b3fa3e8ddd22e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T10:36:54.080 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 ceph-mgr[36631]: -- 172.21.15.178:0/2072623649 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558beb4dd0e0 con 0x558beb4bb400 2024-09-10T10:36:54.080 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 bash[36612]: 0a2a6923d0d59979e50e28b27fdf0d48023e030dc50c3b56f2b3fa3e8ddd22e7 2024-09-10T10:36:54.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 systemd[1]: Started Ceph mgr.z for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:54.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:53.872+0000 7fdc15c1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:36:54.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:53.932+0000 7fdc15c1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:54.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.079+0000 7fdc15c1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:36:54.424 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.145+0000 7fdc15c1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:36:54.425 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.291+0000 7fdc15c1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:54.636 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:54.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:54.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:54.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:54.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:36:54.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:54.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:54.709 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:36:54.736 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_4 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:54.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:54.750 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.432+0000 7fdc15c1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:36:54.750 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.747+0000 7fdc15c1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:36:54.980 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:36:55.174 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.808+0000 7fdc15c1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:36:55.174 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:54.992+0000 7fdc15c1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:36:55.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 -- 172.21.15.143:0/1260396677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c0101d10 msgr2=0x7f98c0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 --2- 172.21.15.143:0/1260396677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c0101d10 0x7f98c0102190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009900 tx=0x7f98b402a410 comp rx=0 tx=0).stop 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 -- 172.21.15.143:0/1260396677 shutdown_connections 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 --2- 172.21.15.143:0/1260396677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c01026d0 0x7f98c0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 --2- 172.21.15.143:0/1260396677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c0101d10 0x7f98c0102190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 --2- 172.21.15.143:0/1260396677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98c0100b10 0x7f98c0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 -- 172.21.15.143:0/1260396677 >> 172.21.15.143:0/1260396677 conn(0x7f98c00fc2a0 msgr2=0x7f98c00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 -- 172.21.15.143:0/1260396677 shutdown_connections 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.417+0000 7f98c4838640 1 -- 172.21.15.143:0/1260396677 wait complete. 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 Processor -- start 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 -- start start 2024-09-10T10:36:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c0100b10 0x7f98c0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98c0101d10 0x7f98c0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 0x7f98c019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010c6c0 con 0x7f98c01026d0 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010c540 con 0x7f98c0100b10 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98c4838640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010c840 con 0x7f98c0101d10 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 0x7f98c019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98be59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c0100b10 0x7f98c0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bdd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98c0101d10 0x7f98c0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 0x7f98c019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34504/0 (socket says 172.21.15.143:34504) 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98be59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c0100b10 0x7f98c0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56916/0 (socket says 172.21.15.143:56916) 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 -- 172.21.15.143:0/1692922691 learned_addr learned my addr 172.21.15.143:0/1692922691 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:36:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 -- 172.21.15.143:0/1692922691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98c0101d10 msgr2=0x7f98c0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98c0101d10 0x7f98c0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 -- 172.21.15.143:0/1692922691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c0100b10 msgr2=0x7f98c0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c0100b10 0x7f98c0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.418+0000 7f98bed9d640 1 -- 172.21.15.143:0/1692922691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c019fb00 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.419+0000 7f98bed9d640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 0x7f98c019f490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f98b800b970 tx=0x7f98b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.419+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8019070 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.419+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c019fdf0 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.419+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98b8014440 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.419+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b80075d0 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.419+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01a0320 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.420+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f98b801c070 con 0x7f98c01026d0 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.420+0000 7f98af7fe640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98a80392c0 0x7f98a803b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.420+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c01a05b0 con 0x7f98c01026d0 2024-09-10T10:36:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.420+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98b804b940 con 0x7f98c01026d0 2024-09-10T10:36:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.420+0000 7f98be59c640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98a80392c0 0x7f98a803b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.420+0000 7f98be59c640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98a80392c0 0x7f98a803b780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.425+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98b800e0a0 con 0x7f98c01026d0 2024-09-10T10:36:55.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:55.605+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f98c006af50 con 0x7f98a80392c0 2024-09-10T10:36:55.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:55 smithi144 ceph-mon[34230]: Deploying daemon mgr.y on smithi144 2024-09-10T10:36:55.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:55 smithi143 ceph-mon[31057]: Deploying daemon mgr.y on smithi144 2024-09-10T10:36:55.875 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:55 smithi178 ceph-mon[34813]: Deploying daemon mgr.y on smithi144 2024-09-10T10:36:56.174 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:55.872+0000 7fdc15c1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:36:56.175 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:55.939+0000 7fdc15c1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:36:56.436 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:56.330+0000 7f912a407040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:56.722 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:56.491+0000 7f912a407040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:36:56.722 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:56.560+0000 7f912a407040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:56.720+0000 7f912a407040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.723 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:36:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:36:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:56.806 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:56.674+0000 7fdc15c1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:36:56.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:56.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:36:56.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:36:56.806 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:56.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:56.807 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:57.015 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:56.874+0000 7f912a407040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:36:57.102 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:56.803+0000 7fdc15c1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:36:57.103 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:56.948+0000 7fdc15c1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:57.103 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.013+0000 7fdc15c1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:36:57.424 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.168+0000 7fdc15c1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:36:57.425 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.237+0000 7fdc15c1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:36:57.425 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.297+0000 7fdc15c1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:36:57.436 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:57.200+0000 7f912a407040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:36:57.436 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:57.263+0000 7f912a407040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:36:57.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-mon[34813]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.465+0000 7fdc15c1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.532+0000 7fdc15c1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.600+0000 7fdc15c1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.684+0000 7fdc15c1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:57.747 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.744+0000 7fdc15c1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:36:57.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:57 smithi143 ceph-mon[31057]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:57.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:57.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:57.456+0000 7f912a407040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:57 smithi144 ceph-mon[34230]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.174 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:57.804+0000 7fdc15c1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.677 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/933778921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b681e90c-4078-4a6f-8d49-607d1b7505b1"}]: dispatch 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/933778921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b681e90c-4078-4a6f-8d49-607d1b7505b1"}]': finished 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:36:58.678 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:58 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2392879238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:58.351+0000 7f912a407040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:58.418+0000 7f912a407040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/933778921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b681e90c-4078-4a6f-8d49-607d1b7505b1"}]: dispatch 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/933778921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b681e90c-4078-4a6f-8d49-607d1b7505b1"}]': finished 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:36:58.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:58 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2392879238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:36:58.925 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z[36627]: 2024-09-10T10:36:58.593+0000 7fdc15c1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:36:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:36:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:58.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:36:58.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:58.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:36:58.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:36:58.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:58.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/933778921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b681e90c-4078-4a6f-8d49-607d1b7505b1"}]: dispatch 2024-09-10T10:36:58.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/933778921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b681e90c-4078-4a6f-8d49-607d1b7505b1"}]': finished 2024-09-10T10:36:58.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T10:36:58.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:36:58.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:58 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2392879238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:36:59.397 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.138+0000 7f912a407040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:36:59.397 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.253+0000 7f912a407040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:36:59.397 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.395+0000 7f912a407040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:59.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:36:59.499+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f98b8007770 con 0x7f98c01026d0 2024-09-10T10:36:59.674 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.456+0000 7f912a407040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:36:59.674 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.605+0000 7f912a407040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:36:59.674 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.673+0000 7f912a407040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: Reconfiguring daemon mgr.x on smithi143 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: Standby manager daemon z started 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:36:59.675 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: Reconfiguring daemon mgr.x on smithi143 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: Standby manager daemon z started 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:36:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:36:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: Reconfiguring daemon mgr.x on smithi143 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: Standby manager daemon z started 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.178:0/1626956428' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:36:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:36:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:36:59.936 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.733+0000 7f912a407040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:36:59.936 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.888+0000 7f912a407040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:37:00.219 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:36:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:36:59.954+0000 7f912a407040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:37:00.219 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:37:00.019+0000 7f912a407040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:37:00.219 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:37:00.101+0000 7f912a407040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:37:00.219 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:37:00.159+0000 7f912a407040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:37:00.220 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:37:00.218+0000 7f912a407040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:37:00.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:00 smithi143 ceph-mon[31057]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:00 smithi143 ceph-mon[31057]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-10T10:37:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:37:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:00 smithi178 ceph-mon[34813]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:00 smithi178 ceph-mon[34813]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-10T10:37:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:37:00.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-mon[34230]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-mon[34230]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-10T10:37:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:37:01.436 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:37:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y[35598]: 2024-09-10T10:37:01.004+0000 7f912a407040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:37:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:01.505+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f98b8007af0 con 0x7f98c01026d0 2024-09-10T10:37:01.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:01 smithi143 ceph-mon[31057]: Standby manager daemon y started 2024-09-10T10:37:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:01 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:37:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:01 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:37:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:01 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:37:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:01 smithi143 ceph-mon[31057]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:37:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:01 smithi178 ceph-mon[34813]: Standby manager daemon y started 2024-09-10T10:37:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:01 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:37:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:01 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:37:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:01 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:37:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:01 smithi178 ceph-mon[34813]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:37:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:01 smithi144 ceph-mon[34230]: Standby manager daemon y started 2024-09-10T10:37:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:01 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:37:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:01 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:37:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:01 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:37:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:01 smithi144 ceph-mon[34230]: from='mgr.? 172.21.15.144:0/1749154259' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:37:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:02 smithi143 ceph-mon[31057]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:02 smithi143 ceph-mon[31057]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-10T10:37:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:37:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:02 smithi178 ceph-mon[34813]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:02 smithi178 ceph-mon[34813]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-10T10:37:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:37:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:02 smithi144 ceph-mon[34230]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:02 smithi144 ceph-mon[34230]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-10T10:37:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:37:04.697 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:04 smithi143 ceph-mon[31057]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:04.698 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:04 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:04.698 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:04 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:04.698 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:04 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:04 smithi178 ceph-mon[34813]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:04.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:04 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:04.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:04 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:04.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:04 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:04 smithi144 ceph-mon[34230]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:04 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:04 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:04 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:05.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:05.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:05.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:05.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:05.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:05.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:06.757 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:06 smithi143 ceph-mon[31057]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:06.757 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T10:37:06.757 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:06 smithi178 ceph-mon[34813]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T10:37:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:06 smithi144 ceph-mon[34230]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T10:37:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:07.762 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:07 smithi143 ceph-mon[31057]: Deploying daemon osd.0 on smithi143 2024-09-10T10:37:07.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:07 smithi178 ceph-mon[34813]: Deploying daemon osd.0 on smithi143 2024-09-10T10:37:07.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:07 smithi144 ceph-mon[34230]: Deploying daemon osd.0 on smithi143 2024-09-10T10:37:08.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:08 smithi143 ceph-mon[31057]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:08.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:08 smithi144 ceph-mon[34230]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:08 smithi178 ceph-mon[34813]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:09.776 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:09 smithi143 ceph-mon[31057]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:09 smithi178 ceph-mon[34813]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:09 smithi144 ceph-mon[34230]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:12.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:12 smithi178 ceph-mon[34813]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:12 smithi144 ceph-mon[34230]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:12 smithi143 ceph-mon[31057]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:13.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:13.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:13.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:13.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:13.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:13.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:14.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:14 smithi143 ceph-mon[31057]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:14.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:14 smithi178 ceph-mon[34813]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:14.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:14 smithi144 ceph-mon[34230]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:16.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.131+0000 7f98af7fe640 1 -- 172.21.15.143:0/1692922691 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f98c006af50 con 0x7f98a80392c0 2024-09-10T10:37:16.132 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 0 on host 'smithi143' 2024-09-10T10:37:16.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98a80392c0 msgr2=0x7f98a803b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98a80392c0 0x7f98a803b780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b000b390 comp rx=0 tx=0).stop 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 msgr2=0x7f98c019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 0x7f98c019f490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f98b800b970 tx=0x7f98b800be30 comp rx=0 tx=0).stop 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 shutdown_connections 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98a80392c0 0x7f98a803b780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c01026d0 0x7f98c019f490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98c0101d10 0x7f98c0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 --2- 172.21.15.143:0/1692922691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c0100b10 0x7f98c0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 >> 172.21.15.143:0/1692922691 conn(0x7f98c00fc2a0 msgr2=0x7f98c006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:16.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 shutdown_connections 2024-09-10T10:37:16.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:37:16.133+0000 7f98c4838640 1 -- 172.21.15.143:0/1692922691 wait complete. 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:16 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:16.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:16.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:16 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:16 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:16.817 DEBUG:teuthology.orchestra.run.smithi143:osd.0> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T10:37:16.819 INFO:tasks.cephadm:Deploying osd.1 on smithi144 with /dev/vg_nvme/lv_4... 2024-09-10T10:37:16.819 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T10:37:17.071 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:37:18.492 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:18 smithi143 ceph-mon[31057]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:18.542 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:18 smithi144 ceph-mon[34230]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:18.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:18 smithi178 ceph-mon[34813]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:18.801 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:37:18 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0[45426]: 2024-09-10T10:37:18.664+0000 7f01a28bc640 -1 osd.0 0 log_to_monitors true 2024-09-10T10:37:19.145 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:37:19.170 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-09-10T10:37:19.412 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:37:19.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:19 smithi143 ceph-mon[31057]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T10:37:19.658 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:19 smithi144 ceph-mon[34230]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T10:37:19.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:19 smithi178 ceph-mon[34813]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 -- 172.21.15.144:0/4199733006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604101d10 msgr2=0x7f3604102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 --2- 172.21.15.144:0/4199733006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604101d10 0x7f3604102190 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009900 tx=0x7f35f402a410 comp rx=0 tx=0).stop 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 -- 172.21.15.144:0/4199733006 shutdown_connections 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 --2- 172.21.15.144:0/4199733006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36041026d0 0x7f3604108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 --2- 172.21.15.144:0/4199733006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604101d10 0x7f3604102190 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 --2- 172.21.15.144:0/4199733006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3604100b10 0x7f3604100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 -- 172.21.15.144:0/4199733006 >> 172.21.15.144:0/4199733006 conn(0x7f36040fc2a0 msgr2=0x7f36040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 -- 172.21.15.144:0/4199733006 shutdown_connections 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.851+0000 7f360baf8640 1 -- 172.21.15.144:0/4199733006 wait complete. 2024-09-10T10:37:19.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 Processor -- start 2024-09-10T10:37:19.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 -- start start 2024-09-10T10:37:19.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 0x7f3604197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:19.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3604101d10 0x7f36041983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:19.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36041026d0 0x7f360419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f360410d770 con 0x7f3604101d10 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f360410d5f0 con 0x7f3604100b10 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360baf8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f360410d8f0 con 0x7f36041026d0 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 0x7f3604197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 0x7f3604197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59238/0 (socket says 172.21.15.144:59238) 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 -- 172.21.15.144:0/234771780 learned_addr learned my addr 172.21.15.144:0/234771780 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609093640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3604101d10 0x7f36041983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f360a095640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36041026d0 0x7f360419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 -- 172.21.15.144:0/234771780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36041026d0 msgr2=0x7f360419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36041026d0 0x7f360419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 -- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3604101d10 msgr2=0x7f36041983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.852+0000 7f3609894640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3604101d10 0x7f36041983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:19.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f3609894640 1 -- 172.21.15.144:0/234771780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360419fa90 con 0x7f3604100b10 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f3609894640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 0x7f3604197e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f360000e9c0 tx=0x7f360000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360000cd80 con 0x7f3604100b10 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36000035f0 con 0x7f3604100b10 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600010690 con 0x7f3604100b10 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360419fd80 con 0x7f3604100b10 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.853+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041a0180 con 0x7f3604100b10 2024-09-10T10:37:19.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.854+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3604108000 con 0x7f3604100b10 2024-09-10T10:37:19.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.855+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f36000037a0 con 0x7f3604100b10 2024-09-10T10:37:19.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.856+0000 7f35faffd640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35f00a2ba0 0x7f35f00a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:19.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.856+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7f36000cd790 con 0x7f3604100b10 2024-09-10T10:37:19.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.856+0000 7f3609093640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35f00a2ba0 0x7f35f00a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:19.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.856+0000 7f3609093640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35f00a2ba0 0x7f35f00a5060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f35f4006010 tx=0x7f35f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:19.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:19.860+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f360009c2f0 con 0x7f3604100b10 2024-09-10T10:37:20.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:20.033+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3604062630 con 0x7f35f00a2ba0 2024-09-10T10:37:20.732 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:20.732 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T10:37:20.732 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T10:37:20.732 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:37:20.732 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:37:20.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:20.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:37:20.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:37:20.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:20.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:20.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:20.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:37:20.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:37:20.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:21.450 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0[45426]: 2024-09-10T10:37:21.101+0000 7f019e83f640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T10:37:21.450 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0[45426]: 2024-09-10T10:37:21.104+0000 7f019665f640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:37:21.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi143 2024-09-10T10:37:21.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi143 to 17088M 2024-09-10T10:37:21.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:21.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:37:21.643 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T10:37:21.644 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.644 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.644 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' 2024-09-10T10:37:21.644 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:21 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi143 2024-09-10T10:37:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi143 to 17088M 2024-09-10T10:37:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:21.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:37:21.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T10:37:21.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' 2024-09-10T10:37:21.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi143 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi143 to 17088M 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: from='osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737]' entity='osd.0' 2024-09-10T10:37:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:21 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] boot 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/3386547290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]: dispatch 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]: dispatch 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]': finished 2024-09-10T10:37:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: osdmap e9: 2 total, 1 up, 2 in 2024-09-10T10:37:22.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:22.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:37:22.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:37:22.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] boot 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/3386547290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]: dispatch 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]: dispatch 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]': finished 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: osdmap e9: 2 total, 1 up, 2 in 2024-09-10T10:37:22.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:37:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:37:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: osd.0 [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] boot 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/3386547290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]: dispatch 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]: dispatch 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1c81ecf-0a81-4ef2-8abd-cc38f88f5482"}]': finished 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: osdmap e9: 2 total, 1 up, 2 in 2024-09-10T10:37:22.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:23.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:23 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/289299290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:37:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:23 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/289299290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:23 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/289299290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:37:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:24 smithi143 ceph-mon[31057]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:24.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:24 smithi143 ceph-mon[31057]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T10:37:24.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:24 smithi178 ceph-mon[34813]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:24 smithi178 ceph-mon[34813]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T10:37:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:24.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:24 smithi144 ceph-mon[34230]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:24 smithi144 ceph-mon[34230]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T10:37:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:26 smithi143 ceph-mon[31057]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:26 smithi178 ceph-mon[34813]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:26 smithi144 ceph-mon[34230]: pgmap v43: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:28.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:28 smithi143 ceph-mon[31057]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:28.804 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:28 smithi144 ceph-mon[34230]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:28 smithi178 ceph-mon[34813]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:30 smithi144 ceph-mon[34230]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:30 smithi143 ceph-mon[31057]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:30 smithi178 ceph-mon[34813]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:31.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T10:37:31.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:31.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T10:37:31.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:31.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T10:37:31.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:32.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:32 smithi143 ceph-mon[31057]: Deploying daemon osd.1 on smithi144 2024-09-10T10:37:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:32 smithi143 ceph-mon[31057]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:32 smithi178 ceph-mon[34813]: Deploying daemon osd.1 on smithi144 2024-09-10T10:37:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:32 smithi178 ceph-mon[34813]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:32.925 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:32 smithi144 ceph-mon[34230]: Deploying daemon osd.1 on smithi144 2024-09-10T10:37:32.925 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:32 smithi144 ceph-mon[34230]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:34.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:34 smithi143 ceph-mon[31057]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:34 smithi178 ceph-mon[34813]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:34 smithi144 ceph-mon[34230]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:36.576 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:36 smithi144 ceph-mon[34230]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:36.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:36 smithi143 ceph-mon[31057]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:36 smithi178 ceph-mon[34813]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:37.698 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:37 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:37.698 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:37 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:37.698 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:37 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:37.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:37 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:37.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:37 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:37.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:37 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:37.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:37 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:37.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:37 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:37.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:37 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:38.689 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:38 smithi144 ceph-mon[34230]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:38 smithi143 ceph-mon[31057]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:38 smithi178 ceph-mon[34813]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:39 smithi143 ceph-mon[31057]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:40.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:39 smithi178 ceph-mon[34813]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:40.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:40.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:40.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:39 smithi144 ceph-mon[34230]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:41.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.236+0000 7f35faffd640 1 -- 172.21.15.144:0/234771780 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3604062630 con 0x7f35f00a2ba0 2024-09-10T10:37:41.238 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 1 on host 'smithi144' 2024-09-10T10:37:41.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.238+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35f00a2ba0 msgr2=0x7f35f00a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:41.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35f00a2ba0 0x7f35f00a5060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f35f4006010 tx=0x7f35f4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 msgr2=0x7f3604197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:41.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 0x7f3604197e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f360000e9c0 tx=0x7f360000ee80 comp rx=0 tx=0).stop 2024-09-10T10:37:41.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 shutdown_connections 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35f00a2ba0 0x7f35f00a5060 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36041026d0 0x7f360419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3604101d10 0x7f36041983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 --2- 172.21.15.144:0/234771780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3604100b10 0x7f3604197e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 >> 172.21.15.144:0/234771780 conn(0x7f36040fc2a0 msgr2=0x7f36041051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 shutdown_connections 2024-09-10T10:37:41.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:37:41.239+0000 7f360baf8640 1 -- 172.21.15.144:0/234771780 wait complete. 2024-09-10T10:37:41.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:41 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:41.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:41 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:41.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:41 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:41.556 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:41 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:41.556 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:41 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:41.556 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:41 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:41.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:41 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:41.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:41 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:41.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:41 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:41.883 DEBUG:teuthology.orchestra.run.smithi144:osd.1> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T10:37:41.886 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_4... 2024-09-10T10:37:41.886 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T10:37:42.137 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:37:42.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:42 smithi143 ceph-mon[31057]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:42.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:42 smithi178 ceph-mon[34813]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:42.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:42 smithi144 ceph-mon[34230]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:43.185 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:37:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1[38581]: 2024-09-10T10:37:42.883+0000 7f3959a72640 -1 osd.1 0 log_to_monitors true 2024-09-10T10:37:43.420 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:43 smithi178 ceph-mon[34813]: from='osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:37:43.420 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:43 smithi178 ceph-mon[34813]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:37:43.435 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:43 smithi144 ceph-mon[34230]: from='osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:37:43.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:43 smithi144 ceph-mon[34230]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:37:43.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:43 smithi143 ceph-mon[31057]: from='osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:37:43.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:43 smithi143 ceph-mon[31057]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:37:44.529 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:44 smithi178 ceph-mon[34813]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T10:37:44.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:44 smithi178 ceph-mon[34813]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T10:37:44.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:44.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:44 smithi178 ceph-mon[34813]: from='osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:37:44.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:44 smithi178 ceph-mon[34813]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:37:44.530 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:44 smithi178 ceph-mon[34813]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:37:44.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:44 smithi143 ceph-mon[31057]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T10:37:44.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:44 smithi143 ceph-mon[31057]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T10:37:44.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:44.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:44 smithi143 ceph-mon[31057]: from='osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:37:44.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:44 smithi143 ceph-mon[31057]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:37:44.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:44 smithi143 ceph-mon[31057]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:44.557 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-09-10T10:37:44.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:44 smithi144 ceph-mon[34230]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T10:37:44.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:44 smithi144 ceph-mon[34230]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T10:37:44.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:44.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:44 smithi144 ceph-mon[34230]: from='osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:37:44.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:44 smithi144 ceph-mon[34230]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:37:44.648 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:44 smithi144 ceph-mon[34230]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:44.804 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:37:45.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 -- 172.21.15.178:0/1421604484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4107980 msgr2=0x7f50a4109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:45.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 --2- 172.21.15.178:0/1421604484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4107980 0x7f50a4109d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f509c009ca0 tx=0x7f509c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 -- 172.21.15.178:0/1421604484 shutdown_connections 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 --2- 172.21.15.178:0/1421604484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4107980 0x7f50a4109d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 --2- 172.21.15.178:0/1421604484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4105050 0x7f50a4107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 --2- 172.21.15.178:0/1421604484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4102720 0x7f50a4104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 -- 172.21.15.178:0/1421604484 >> 172.21.15.178:0/1421604484 conn(0x7f50a40fc280 msgr2=0x7f50a40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 -- 172.21.15.178:0/1421604484 shutdown_connections 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.249+0000 7f50a968e640 1 -- 172.21.15.178:0/1421604484 wait complete. 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 Processor -- start 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 -- start start 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4102720 0x7f50a4102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 0x7f50a41006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:45.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4107980 0x7f50a4100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410d740 con 0x7f50a4107980 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410d5c0 con 0x7f50a4102720 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a968e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410d8c0 con 0x7f50a4105050 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 0x7f50a41006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 0x7f50a41006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49280/0 (socket says 172.21.15.178:49280) 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.250+0000 7f50a27fc640 1 -- 172.21.15.178:0/2934055503 learned_addr learned my addr 172.21.15.178:0/2934055503 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a2ffd640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4102720 0x7f50a4102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a37fe640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4107980 0x7f50a4100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a27fc640 1 -- 172.21.15.178:0/2934055503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4102720 msgr2=0x7f50a4102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a27fc640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4102720 0x7f50a4102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a27fc640 1 -- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4107980 msgr2=0x7f50a4100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a27fc640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4107980 0x7f50a4100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a27fc640 1 -- 172.21.15.178:0/2934055503 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a41014d0 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a37fe640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4107980 0x7f50a4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a27fc640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 0x7f50a41006d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9f0 tx=0x7f509800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509800ccc0 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a41a1f80 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50980054d0 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098010430 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.251+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a41a23e0 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.252+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a40fdd70 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.254+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f5098005000 con 0x7f50a4105050 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.255+0000 7f508bfff640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f508c0a2b70 0x7f508c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:45.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.255+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f50980cdd10 con 0x7f50a4105050 2024-09-10T10:37:45.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.255+0000 7f50a2ffd640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f508c0a2b70 0x7f508c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:45.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.256+0000 7f50a2ffd640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f508c0a2b70 0x7f508c0a5030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5094005d00 tx=0x7f509400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:45.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.258+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50980541b0 con 0x7f50a4105050 2024-09-10T10:37:45.299 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1[38581]: 2024-09-10T10:37:45.017+0000 7f39559f5640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T10:37:45.299 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1[38581]: 2024-09-10T10:37:45.020+0000 7f394d815640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:37:45.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:45 smithi178 ceph-mon[34813]: from='osd.1 ' entity='osd.1' 2024-09-10T10:37:45.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:37:45.442+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f50a4062630 con 0x7f508c0a2b70 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:45 smithi143 ceph-mon[31057]: from='osd.1 ' entity='osd.1' 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:37:45.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:45 smithi144 ceph-mon[34230]: from='osd.1 ' entity='osd.1' 2024-09-10T10:37:46.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi144 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi144 to 17088M 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] boot 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T10:37:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:37:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:37:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi144 2024-09-10T10:37:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi144 to 17088M 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] boot 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T10:37:46.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi144 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi144 to 17088M 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:37:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:46.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: osd.1 [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] boot 2024-09-10T10:37:46.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T10:37:46.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:37:48.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: pgmap v58: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/1362846110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]: dispatch 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]: dispatch 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]': finished 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:37:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:48 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/2319783912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: pgmap v58: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/1362846110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]: dispatch 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]: dispatch 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]': finished 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:37:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:48 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/2319783912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: pgmap v58: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/1362846110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]: dispatch 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]: dispatch 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5d7e27a-4fb6-4951-86f4-b0f40050cdd2"}]': finished 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:37:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:48 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/2319783912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:37:50.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:50 smithi178 ceph-mon[34813]: pgmap v60: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:50.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:50 smithi144 ceph-mon[34230]: pgmap v60: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:50.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:50 smithi143 ceph-mon[31057]: pgmap v60: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:52.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:52 smithi178 ceph-mon[34813]: pgmap v61: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:52.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:52 smithi144 ceph-mon[34230]: pgmap v61: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:52.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:52 smithi143 ceph-mon[31057]: pgmap v61: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:54.564 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:54 smithi178 ceph-mon[34813]: pgmap v62: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:54 smithi144 ceph-mon[34230]: pgmap v62: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:54.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:54 smithi143 ceph-mon[31057]: pgmap v62: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:56.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:56 smithi178 ceph-mon[34813]: pgmap v63: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T10:37:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:56 smithi144 ceph-mon[34230]: pgmap v63: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T10:37:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:56 smithi143 ceph-mon[31057]: pgmap v63: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T10:37:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:57.604 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:57 smithi178 ceph-mon[34813]: Deploying daemon osd.2 on smithi178 2024-09-10T10:37:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:57 smithi144 ceph-mon[34230]: Deploying daemon osd.2 on smithi178 2024-09-10T10:37:57.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:57 smithi143 ceph-mon[31057]: Deploying daemon osd.2 on smithi178 2024-09-10T10:37:58.689 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:37:58 smithi144 ceph-mon[34230]: pgmap v64: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:58.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:37:58 smithi143 ceph-mon[31057]: pgmap v64: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:37:58.785 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:37:58 smithi178 ceph-mon[34813]: pgmap v64: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:00 smithi144 ceph-mon[34230]: pgmap v65: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:00.779 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:00 smithi178 ceph-mon[34813]: pgmap v65: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:00.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:00 smithi143 ceph-mon[31057]: pgmap v65: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:02 smithi144 ceph-mon[34230]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:02.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:02 smithi178 ceph-mon[34813]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:02.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:02.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:02.708 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:02 smithi143 ceph-mon[31057]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:04.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:04 smithi178 ceph-mon[34813]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:04.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:04 smithi143 ceph-mon[31057]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:04 smithi144 ceph-mon[34230]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:06.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:06.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:06.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:06 smithi143 ceph-mon[31057]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:06.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:06.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:06.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.305 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:06 smithi178 ceph-mon[34813]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:06.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:06.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:06.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:06.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:06 smithi144 ceph-mon[34230]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:06.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.595+0000 7f508bfff640 1 -- 172.21.15.178:0/2934055503 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f50a4062630 con 0x7f508c0a2b70 2024-09-10T10:38:06.596 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-09-10T10:38:06.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f508c0a2b70 msgr2=0x7f508c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:06.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f508c0a2b70 0x7f508c0a5030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5094005d00 tx=0x7f509400b410 comp rx=0 tx=0).stop 2024-09-10T10:38:06.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 msgr2=0x7f50a41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:06.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 0x7f50a41006d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9f0 tx=0x7f509800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:38:06.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 shutdown_connections 2024-09-10T10:38:06.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f508c0a2b70 0x7f508c0a5030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:06.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50a4107980 0x7f50a4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:06.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50a4105050 0x7f50a41006d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:06.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 --2- 172.21.15.178:0/2934055503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50a4102720 0x7f50a4102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:06.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 >> 172.21.15.178:0/2934055503 conn(0x7f50a40fc280 msgr2=0x7f50a41030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:06.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 shutdown_connections 2024-09-10T10:38:06.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:38:06.597+0000 7f50a968e640 1 -- 172.21.15.178:0/2934055503 wait complete. 2024-09-10T10:38:07.221 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:07.221 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:07.221 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:07.223 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T10:38:07.226 INFO:tasks.cephadm:Deploying osd.3 on smithi143 with /dev/vg_nvme/lv_3... 2024-09-10T10:38:07.226 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T10:38:07.338 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:07.338 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:07.338 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:07.467 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:38:07.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:07.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:07.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:08.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:08 smithi143 ceph-mon[31057]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:08.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:08 smithi143 ceph-mon[31057]: from='osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:38:08.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:08 smithi143 ceph-mon[31057]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:38:08.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:08 smithi178 ceph-mon[34813]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:08.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:08 smithi178 ceph-mon[34813]: from='osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:38:08.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:08 smithi178 ceph-mon[34813]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:38:08.328 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2[39764]: 2024-09-10T10:38:07.988+0000 7fd654050640 -1 osd.2 0 log_to_monitors true 2024-09-10T10:38:08.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:08 smithi144 ceph-mon[34230]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:08.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:08 smithi144 ceph-mon[34230]: from='osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:38:08.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:08 smithi144 ceph-mon[34230]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:38:09.196 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:09 smithi143 ceph-mon[31057]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T10:38:09.197 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:09 smithi143 ceph-mon[31057]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T10:38:09.197 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:09 smithi143 ceph-mon[31057]: from='osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:38:09.197 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:09.197 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:09 smithi143 ceph-mon[31057]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:38:09.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-mon[34813]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T10:38:09.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-mon[34813]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T10:38:09.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-mon[34813]: from='osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:38:09.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:09.202 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-mon[34813]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:38:09.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:09 smithi144 ceph-mon[34230]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T10:38:09.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:09 smithi144 ceph-mon[34230]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T10:38:09.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:09 smithi144 ceph-mon[34230]: from='osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:38:09.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:09.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:09 smithi144 ceph-mon[34230]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:38:09.838 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2[39764]: 2024-09-10T10:38:09.641+0000 7fd64ffd3640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T10:38:09.838 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2[39764]: 2024-09-10T10:38:09.646+0000 7fd647df3640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:38:09.906 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:38:09.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_3 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='osd.2 ' entity='osd.2' 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:10.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:10.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:10.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='osd.2 ' entity='osd.2' 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:10.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:10.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:38:10.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T10:38:10.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:10.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='osd.2 ' entity='osd.2' 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:10.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 -- 172.21.15.143:0/711766842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90381037c0 msgr2=0x7f903810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 --2- 172.21.15.143:0/711766842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90381037c0 0x7f903810a050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9030009900 tx=0x7f903002a410 comp rx=0 tx=0).stop 2024-09-10T10:38:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 -- 172.21.15.143:0/711766842 shutdown_connections 2024-09-10T10:38:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 --2- 172.21.15.143:0/711766842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90381037c0 0x7f903810a050 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 --2- 172.21.15.143:0/711766842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9038102e00 0x7f9038103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 --2- 172.21.15.143:0/711766842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9038101c00 0x7f9038102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 -- 172.21.15.143:0/711766842 >> 172.21.15.143:0/711766842 conn(0x7f90380fd390 msgr2=0x7f90380ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.648+0000 7f903dba8640 1 -- 172.21.15.143:0/711766842 shutdown_connections 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 -- 172.21.15.143:0/711766842 wait complete. 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 Processor -- start 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 -- start start 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 0x7f9038197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9038102e00 0x7f9038198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90381037c0 0x7f903819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f903810d730 con 0x7f9038101c00 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f903810d5b0 con 0x7f90381037c0 2024-09-10T10:38:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.649+0000 7f903dba8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f903810d8b0 con 0x7f9038102e00 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 0x7f9038197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 0x7f9038197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33290/0 (socket says 172.21.15.143:33290) 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 -- 172.21.15.143:0/3099934258 learned_addr learned my addr 172.21.15.143:0/3099934258 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f9036ffd640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9038102e00 0x7f9038198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f9037fff640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90381037c0 0x7f903819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 -- 172.21.15.143:0/3099934258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9038102e00 msgr2=0x7f9038198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9038102e00 0x7f9038198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 -- 172.21.15.143:0/3099934258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90381037c0 msgr2=0x7f903819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90381037c0 0x7f903819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 -- 172.21.15.143:0/3099934258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903819fa10 con 0x7f9038101c00 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f9037fff640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90381037c0 0x7f903819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:38:10.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f90377fe640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 0x7f9038197de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9028009fd0 tx=0x7f902800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028011470 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9028011ac0 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903819fd00 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902801aab0 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.650+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90381a01d0 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.651+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9038109050 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.652+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f9028011c70 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.653+0000 7f9034ff9640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f90240a2c20 0x7f90240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.654+0000 7f9036ffd640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f90240a2c20 0x7f90240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.654+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f90280cdf80 con 0x7f9038101c00 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.654+0000 7f9036ffd640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f90240a2c20 0x7f90240a50e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9038199300 tx=0x7f9020005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:10.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.657+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9028016030 con 0x7f9038101c00 2024-09-10T10:38:10.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:10.834+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9038062630 con 0x7f90240a2c20 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi178 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi178 to 17087M 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] boot 2024-09-10T10:38:11.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T10:38:11.240 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:11.240 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:11.240 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:11.240 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi178 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi178 to 17087M 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] boot 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:11.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:11.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:11.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi178 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi178 to 17087M 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: osd.2 [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] boot 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:11.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:12.189 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:12 smithi143 ceph-mon[31057]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:12.189 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:12 smithi143 ceph-mon[31057]: pgmap v74: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:12.190 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:38:12.190 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T10:38:12.190 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:12 smithi143 ceph-mon[31057]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T10:38:12.190 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:38:12.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:12 smithi178 ceph-mon[34813]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:12.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:12 smithi178 ceph-mon[34813]: pgmap v74: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:12.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:38:12.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T10:38:12.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:12 smithi178 ceph-mon[34813]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T10:38:12.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:38:12.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:12 smithi144 ceph-mon[34230]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:12.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:12 smithi144 ceph-mon[34230]: pgmap v74: 0 pgs: ; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:12.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:38:12.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T10:38:12.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:12 smithi144 ceph-mon[34230]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T10:38:12.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:38:13.051 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:12 smithi143 sudo[49823]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:38:13.051 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:12 smithi143 sudo[49823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:13.051 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:12 smithi143 sudo[49823]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:13.051 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:13 smithi143 sudo[49823]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:13.550 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:13 smithi143 sudo[49849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:38:13.550 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:13 smithi143 sudo[49849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:13.551 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:13 smithi143 sudo[49849]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:13.551 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:38:13 smithi143 sudo[49849]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:13.794 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:38:13.795 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:13.795 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42633]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:13.795 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42633]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3700121141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a60f699-6beb-43e3-a94b-e5544e39fec5"}]: dispatch 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3700121141' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a60f699-6beb-43e3-a94b-e5544e39fec5"}]': finished 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:13 smithi144 ceph-mon[34230]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:13.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3700121141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a60f699-6beb-43e3-a94b-e5544e39fec5"}]: dispatch 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3700121141' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a60f699-6beb-43e3-a94b-e5544e39fec5"}]': finished 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:13.804 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:13 smithi143 ceph-mon[31057]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:13.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3700121141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a60f699-6beb-43e3-a94b-e5544e39fec5"}]: dispatch 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3700121141' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a60f699-6beb-43e3-a94b-e5544e39fec5"}]': finished 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:13 smithi178 ceph-mon[34813]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:14.186 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42636]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:38:14.186 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:14.186 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42636]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:14.186 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:38:13 smithi144 sudo[42636]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:14.304+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f90280983e0 con 0x7f9038101c00 2024-09-10T10:38:14.329 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:38:14.330 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:14.330 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43786]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:14.330 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43786]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:14.674 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43789]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:38:14.674 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43789]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:14.675 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43789]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:14.675 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:38:14 smithi178 sudo[43789]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:14.856 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:38:14.856 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:14.856 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49926]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:14.856 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49926]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:15.118 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:14 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2426076616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:15.118 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:14 smithi178 ceph-mon[34813]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:14 smithi178 ceph-mon[34813]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43792]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43792]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43792]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:15.119 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43792]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:15.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:14 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2426076616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:15.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:15.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:14 smithi144 ceph-mon[34230]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T10:38:15.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:15.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:14 smithi144 ceph-mon[34230]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:15.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49931]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:38:15.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49931]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:15.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49931]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:15.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 sudo[49931]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:15.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2426076616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:15.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:15.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 ceph-mon[31057]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T10:38:15.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:15.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:14 smithi143 ceph-mon[31057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:15.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43795]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:38:15.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43795]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:15.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43795]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:15.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 sudo[43795]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:15.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:38:15.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:15.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42639]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:15.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42639]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:16.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:16.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:15 smithi178 ceph-mon[34813]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42642]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42642]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42642]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 sudo[42642]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:16.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:15 smithi144 ceph-mon[34230]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:38:16.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:16.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:16.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:16.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 104 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:16.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:15 smithi143 ceph-mon[31057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:38:18.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:18 smithi178 ceph-mon[34813]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:18.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:18 smithi144 ceph-mon[34230]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:18.716 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:18 smithi143 ceph-mon[31057]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:20.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:20 smithi178 ceph-mon[34813]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:20.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:20 smithi144 ceph-mon[34230]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:20.700 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:20 smithi143 ceph-mon[31057]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:22.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:22 smithi143 ceph-mon[31057]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T10:38:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:22.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:22 smithi178 ceph-mon[34813]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:22.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T10:38:22.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:22.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:22 smithi144 ceph-mon[34230]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T10:38:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:23.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:23 smithi143 ceph-mon[31057]: Deploying daemon osd.3 on smithi143 2024-09-10T10:38:23.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:23 smithi144 ceph-mon[34230]: Deploying daemon osd.3 on smithi143 2024-09-10T10:38:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:23 smithi178 ceph-mon[34813]: Deploying daemon osd.3 on smithi143 2024-09-10T10:38:24.523 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:24 smithi143 ceph-mon[31057]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:24.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:24 smithi144 ceph-mon[34230]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:24 smithi178 ceph-mon[34813]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:26 smithi144 ceph-mon[34230]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:26 smithi143 ceph-mon[31057]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:26 smithi178 ceph-mon[34813]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:28.598 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:28 smithi143 ceph-mon[31057]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:28.599 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:28 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:28.599 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:28 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:28.599 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:28 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:28 smithi144 ceph-mon[34230]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:28 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:28 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:28 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:28 smithi178 ceph-mon[34813]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:28 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:28 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:28 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:30.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:30 smithi143 ceph-mon[31057]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:30 smithi178 ceph-mon[34813]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:30.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:30 smithi144 ceph-mon[34230]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:31 smithi178 ceph-mon[34813]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:31 smithi144 ceph-mon[34230]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:32.224 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.224 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.224 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:32.224 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:32.224 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:32.224 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:31 smithi143 ceph-mon[31057]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:32.262 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 3 on host 'smithi143' 2024-09-10T10:38:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.260+0000 7f9034ff9640 1 -- 172.21.15.143:0/3099934258 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9038062630 con 0x7f90240a2c20 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f90240a2c20 msgr2=0x7f90240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f90240a2c20 0x7f90240a50e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9038199300 tx=0x7f9020005f90 comp rx=0 tx=0).stop 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 msgr2=0x7f9038197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 0x7f9038197de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9028009fd0 tx=0x7f902800c270 comp rx=0 tx=0).stop 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 shutdown_connections 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f90240a2c20 0x7f90240a50e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90381037c0 0x7f903819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9038102e00 0x7f9038198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:32.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 --2- 172.21.15.143:0/3099934258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9038101c00 0x7f9038197de0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.263+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 >> 172.21.15.143:0/3099934258 conn(0x7f90380fd390 msgr2=0x7f9038106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.264+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 shutdown_connections 2024-09-10T10:38:32.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:38:32.264+0000 7f903dba8640 1 -- 172.21.15.143:0/3099934258 wait complete. 2024-09-10T10:38:32.934 DEBUG:teuthology.orchestra.run.smithi143:osd.3> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T10:38:32.937 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_3... 2024-09-10T10:38:32.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T10:38:33.124 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:33.124 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:33.124 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:33.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:33.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:33.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:33.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:33.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:33.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:33.193 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:38:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:33 smithi144 ceph-mon[34230]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:34.124 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:33 smithi143 ceph-mon[31057]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:34.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:33 smithi178 ceph-mon[34813]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:34.801 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:38:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:38:34.461+0000 7f23ca78d640 -1 osd.3 0 log_to_monitors true 2024-09-10T10:38:35.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:34 smithi178 ceph-mon[34813]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T10:38:35.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:34 smithi144 ceph-mon[34230]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T10:38:35.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:34 smithi143 ceph-mon[31057]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T10:38:35.711 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:38:35.738 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-09-10T10:38:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:38:36.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:35 smithi144 ceph-mon[34230]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T10:38:36.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:35 smithi144 ceph-mon[34230]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T10:38:36.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:35 smithi144 ceph-mon[34230]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:38:36.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:36.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:35 smithi144 ceph-mon[34230]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:36.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:35 smithi178 ceph-mon[34813]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T10:38:36.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:35 smithi178 ceph-mon[34813]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T10:38:36.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:35 smithi178 ceph-mon[34813]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:38:36.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:36.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:35 smithi178 ceph-mon[34813]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:36.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-mon[31057]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T10:38:36.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-mon[31057]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T10:38:36.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-mon[31057]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:38:36.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:36.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-mon[31057]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:36.301 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:38:35.876+0000 7f23c6710640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T10:38:36.301 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:38:35 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:38:35.880+0000 7f23c2538640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:38:36.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 -- 172.21.15.144:0/3530291195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63901037c0 msgr2=0x7f639010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:36.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 --2- 172.21.15.144:0/3530291195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63901037c0 0x7f639010a050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6388009ca0 tx=0x7f638802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:38:36.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 -- 172.21.15.144:0/3530291195 shutdown_connections 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 --2- 172.21.15.144:0/3530291195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63901037c0 0x7f639010a050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 --2- 172.21.15.144:0/3530291195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390102e00 0x7f6390103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 --2- 172.21.15.144:0/3530291195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6390101c00 0x7f6390102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.501+0000 7f63954f0640 1 -- 172.21.15.144:0/3530291195 >> 172.21.15.144:0/3530291195 conn(0x7f63900fd390 msgr2=0x7f63900ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 -- 172.21.15.144:0/3530291195 shutdown_connections 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 -- 172.21.15.144:0/3530291195 wait complete. 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 Processor -- start 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 -- start start 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390101c00 0x7f6390197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 0x7f6390198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63901037c0 0x7f639019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:36.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f639010d730 con 0x7f63901037c0 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f639010d5b0 con 0x7f6390102e00 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.502+0000 7f63954f0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f639010d8b0 con 0x7f6390101c00 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 0x7f6390198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 0x7f6390198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51786/0 (socket says 172.21.15.144:51786) 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 -- 172.21.15.144:0/1660004888 learned_addr learned my addr 172.21.15.144:0/1660004888 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f638f7fe640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63901037c0 0x7f639019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f638effd640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390101c00 0x7f6390197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 -- 172.21.15.144:0/1660004888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390101c00 msgr2=0x7f6390197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390101c00 0x7f6390197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 -- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63901037c0 msgr2=0x7f639019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63901037c0 0x7f639019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:36.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 -- 172.21.15.144:0/1660004888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639019f9f0 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f638effd640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390101c00 0x7f6390197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63867fc640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 0x7f6390198300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f637800d770 tx=0x7f637800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378017070 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639019fce0 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63780054d0 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378004cf0 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.503+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63901a01c0 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.504+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6390109020 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.505+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6378004e90 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.506+0000 7f638cff9640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f637c0a2bb0 0x7f637c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.506+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f63780cd7f0 con 0x7f6390102e00 2024-09-10T10:38:36.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.506+0000 7f638effd640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f637c0a2bb0 0x7f637c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:36.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.507+0000 7f638effd640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f637c0a2bb0 0x7f637c0a5070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6390102c60 tx=0x7f6380005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:36.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.511+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f637809b9f0 con 0x7f6390102e00 2024-09-10T10:38:36.687 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:36.686+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6390062630 con 0x7f637c0a2bb0 2024-09-10T10:38:37.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:38:37.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T10:38:37.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:37.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:37.095 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi143 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi143 to 8544M 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi143 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi143 to 8544M 2024-09-10T10:38:37.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:37.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:37.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:37.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:37.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi143 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:37.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi143 to 8544M 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:37.302 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] boot 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:38.085 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:37 smithi144 ceph-mon[34230]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T10:38:38.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] boot 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:37 smithi178 ceph-mon[34813]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: osd.3 [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] boot 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:38:38.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:37 smithi143 ceph-mon[31057]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T10:38:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T10:38:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/1404945492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]: dispatch 2024-09-10T10:38:39.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]: dispatch 2024-09-10T10:38:39.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]': finished 2024-09-10T10:38:39.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T10:38:39.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:39.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:39.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:39 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/579729189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:40.087 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/1404945492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]: dispatch 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]: dispatch 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]': finished 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:40.088 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:39 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/579729189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:40.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/1404945492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]: dispatch 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]: dispatch 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d383b3ca-df71-4034-ac1f-bcb431e38265"}]': finished 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:39 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/579729189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:41.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:41 smithi178 ceph-mon[34813]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T10:38:41.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:41 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:41.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:41 smithi144 ceph-mon[34230]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T10:38:41.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:41 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:41.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:41 smithi143 ceph-mon[31057]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T10:38:41.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:41 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:42.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:42 smithi178 ceph-mon[34813]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T10:38:42.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:42 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:42.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:42 smithi178 ceph-mon[34813]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-09-10T10:38:42.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:42 smithi144 ceph-mon[34230]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T10:38:42.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:42 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:42.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:42 smithi144 ceph-mon[34230]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-09-10T10:38:42.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:42 smithi143 ceph-mon[31057]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T10:38:42.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:42 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:38:42.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:42 smithi143 ceph-mon[31057]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-09-10T10:38:44.534 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:44 smithi144 ceph-mon[34230]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-10T10:38:44.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:44 smithi178 ceph-mon[34813]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-10T10:38:44.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:44 smithi143 ceph-mon[31057]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-10T10:38:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:46 smithi144 ceph-mon[34230]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T10:38:46.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:46 smithi143 ceph-mon[31057]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T10:38:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:46 smithi178 ceph-mon[34813]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T10:38:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T10:38:47.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:47.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T10:38:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:47.811 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T10:38:47.811 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:48.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:48 smithi178 ceph-mon[34813]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:38:48.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:48 smithi178 ceph-mon[34813]: Deploying daemon osd.4 on smithi144 2024-09-10T10:38:48.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:48 smithi143 ceph-mon[31057]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:38:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:48 smithi143 ceph-mon[31057]: Deploying daemon osd.4 on smithi144 2024-09-10T10:38:48.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:48 smithi144 ceph-mon[34230]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:38:48.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:48 smithi144 ceph-mon[34230]: Deploying daemon osd.4 on smithi144 2024-09-10T10:38:50.553 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:50 smithi144 ceph-mon[34230]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T10:38:50.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:50 smithi143 ceph-mon[31057]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T10:38:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:50 smithi178 ceph-mon[34813]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T10:38:52.569 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:52 smithi144 ceph-mon[34230]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T10:38:52.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:52 smithi143 ceph-mon[31057]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T10:38:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:52 smithi178 ceph-mon[34813]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T10:38:54.452 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:54 smithi144 ceph-mon[34230]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:54.452 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:54.452 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:54.452 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:54.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:54 smithi143 ceph-mon[31057]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:54 smithi178 ceph-mon[34813]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:56 smithi144 ceph-mon[34230]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:56 smithi143 ceph-mon[31057]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:56 smithi178 ceph-mon[34813]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:57.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.105+0000 7f638cff9640 1 -- 172.21.15.144:0/1660004888 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6390062630 con 0x7f637c0a2bb0 2024-09-10T10:38:57.107 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-09-10T10:38:57.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f637c0a2bb0 msgr2=0x7f637c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:57.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f637c0a2bb0 0x7f637c0a5070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6390102c60 tx=0x7f6380005f90 comp rx=0 tx=0).stop 2024-09-10T10:38:57.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 msgr2=0x7f6390198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:57.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 0x7f6390198300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f637800d770 tx=0x7f637800dc30 comp rx=0 tx=0).stop 2024-09-10T10:38:57.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 shutdown_connections 2024-09-10T10:38:57.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f637c0a2bb0 0x7f637c0a5070 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:57.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63901037c0 0x7f639019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:57.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6390102e00 0x7f6390198300 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:57.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 --2- 172.21.15.144:0/1660004888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6390101c00 0x7f6390197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:57.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 >> 172.21.15.144:0/1660004888 conn(0x7f63900fd390 msgr2=0x7f63901061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:57.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 shutdown_connections 2024-09-10T10:38:57.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:38:57.108+0000 7f63954f0640 1 -- 172.21.15.144:0/1660004888 wait complete. 2024-09-10T10:38:57.696 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T10:38:57.699 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-09-10T10:38:57.699 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T10:38:57.720 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.720 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.720 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.720 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:38:57.953 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:38:58.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:38:58 smithi178 ceph-mon[34813]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:58.715 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:38:58 smithi143 ceph-mon[31057]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:38:58.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:38:58 smithi144 ceph-mon[34230]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:00.186 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:39:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4[45271]: 2024-09-10T10:39:00.072+0000 7fda6327e640 -1 osd.4 0 log_to_monitors true 2024-09-10T10:39:00.513 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:00 smithi144 ceph-mon[34230]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:00.513 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:00 smithi144 ceph-mon[34230]: from='osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:39:00.513 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:00 smithi144 ceph-mon[34230]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:39:00.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:00 smithi178 ceph-mon[34813]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:00 smithi178 ceph-mon[34813]: from='osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:39:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:00 smithi178 ceph-mon[34813]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:39:00.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:39:00.723 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-09-10T10:39:00.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:00 smithi143 ceph-mon[31057]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:00 smithi143 ceph-mon[31057]: from='osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:39:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:00 smithi143 ceph-mon[31057]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:39:01.009 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:39:01.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 -- 172.21.15.178:0/2269155548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30107900 msgr2=0x7fba30109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:01.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 --2- 172.21.15.178:0/2269155548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30107900 0x7fba30109d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fba28009ca0 tx=0x7fba2802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 -- 172.21.15.178:0/2269155548 shutdown_connections 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 --2- 172.21.15.178:0/2269155548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba30107900 0x7fba30109d50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 --2- 172.21.15.178:0/2269155548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba30104fd0 0x7fba301073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 --2- 172.21.15.178:0/2269155548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba301026a0 0x7fba30104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 -- 172.21.15.178:0/2269155548 >> 172.21.15.178:0/2269155548 conn(0x7fba300fc2a0 msgr2=0x7fba300fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 -- 172.21.15.178:0/2269155548 shutdown_connections 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.528+0000 7fba35813640 1 -- 172.21.15.178:0/2269155548 wait complete. 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 Processor -- start 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 -- start start 2024-09-10T10:39:01.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 0x7fba30102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba30104fd0 0x7fba30100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30107900 0x7fba30100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010d770 con 0x7fba30104fd0 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010d5f0 con 0x7fba30107900 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba35813640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010d8f0 con 0x7fba301026a0 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba2effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 0x7fba30102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba2effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 0x7fba30102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:46946/0 (socket says 172.21.15.178:46946) 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba2effd640 1 -- 172.21.15.178:0/3533745851 learned_addr learned my addr 172.21.15.178:0/3533745851 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.529+0000 7fba2e7fc640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba30104fd0 0x7fba30100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2f7fe640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30107900 0x7fba30100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2effd640 1 -- 172.21.15.178:0/3533745851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30107900 msgr2=0x7fba30100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:01.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2effd640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30107900 0x7fba30100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2effd640 1 -- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba30104fd0 msgr2=0x7fba30100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2effd640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba30104fd0 0x7fba30100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2effd640 1 -- 172.21.15.178:0/3533745851 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba30101450 con 0x7fba301026a0 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2f7fe640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30107900 0x7fba30100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba2effd640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 0x7fba30102000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fba2000d770 tx=0x7fba2000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20017070 con 0x7fba301026a0 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba200054d0 con 0x7fba301026a0 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba301a1fb0 con 0x7fba301026a0 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20004cf0 con 0x7fba301026a0 2024-09-10T10:39:01.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.530+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba301a2410 con 0x7fba301026a0 2024-09-10T10:39:01.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.531+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba300fdd90 con 0x7fba301026a0 2024-09-10T10:39:01.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.532+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fba20004e90 con 0x7fba301026a0 2024-09-10T10:39:01.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.533+0000 7fba34811640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fba180a2b80 0x7fba180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:01.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.533+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7fba200cd920 con 0x7fba301026a0 2024-09-10T10:39:01.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.533+0000 7fba2e7fc640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fba180a2b80 0x7fba180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:01.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.534+0000 7fba2e7fc640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fba180a2b80 0x7fba180a5040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba30101d80 tx=0x7fba2400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:01.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.538+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba2009ba10 con 0x7fba301026a0 2024-09-10T10:39:01.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:01.716+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fba30062630 con 0x7fba180a2b80 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:01.719 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:01.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.936 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4[45271]: 2024-09-10T10:39:01.472+0000 7fda5f201640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T10:39:01.936 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4[45271]: 2024-09-10T10:39:01.480+0000 7fda5b029640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:39:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T10:39:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:39:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:01.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:01.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi144 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi144 to 8544M 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:02.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:02.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:02.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi144 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi144 to 8544M 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:02.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:02.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:02.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi144 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi144 to 8544M 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:03.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:03 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:39:03.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:03 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:39:03.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:03 smithi178 ceph-mon[34813]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:03.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:03 smithi178 ceph-mon[34813]: osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] boot 2024-09-10T10:39:03.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:03 smithi178 ceph-mon[34813]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T10:39:03.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:03 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:03 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:39:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:03 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:39:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:03 smithi143 ceph-mon[31057]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:03 smithi143 ceph-mon[31057]: osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] boot 2024-09-10T10:39:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:03 smithi143 ceph-mon[31057]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T10:39:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:03 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:03 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:39:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:03 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:39:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:03 smithi144 ceph-mon[34230]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:03 smithi144 ceph-mon[34230]: osd.4 [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] boot 2024-09-10T10:39:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:03 smithi144 ceph-mon[34230]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T10:39:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:03 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:39:04.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:04.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T10:39:04.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/1550823146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]: dispatch 2024-09-10T10:39:04.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]: dispatch 2024-09-10T10:39:04.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]': finished 2024-09-10T10:39:04.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T10:39:04.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:04 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:04.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T10:39:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/1550823146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]: dispatch 2024-09-10T10:39:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]: dispatch 2024-09-10T10:39:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]': finished 2024-09-10T10:39:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T10:39:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:04 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/1550823146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]: dispatch 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]: dispatch 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb5db187-d7f1-4f74-a4ec-ff8e4e686270"}]': finished 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T10:39:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:04 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:05.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:05 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/3700901344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:05.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:05 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/3700901344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:05.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:05 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/3700901344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:06 smithi143 ceph-mon[31057]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:06 smithi178 ceph-mon[34813]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:06.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:06 smithi144 ceph-mon[34230]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:08.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:08 smithi143 ceph-mon[31057]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:08.910 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:08 smithi144 ceph-mon[34230]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:08.910 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:08 smithi178 ceph-mon[34813]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:10.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:10 smithi178 ceph-mon[34813]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:10 smithi143 ceph-mon[31057]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:10 smithi144 ceph-mon[34230]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:12.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:12 smithi178 ceph-mon[34813]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T10:39:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:12 smithi143 ceph-mon[31057]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T10:39:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:12 smithi144 ceph-mon[34230]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T10:39:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:13 smithi143 ceph-mon[31057]: Deploying daemon osd.5 on smithi178 2024-09-10T10:39:13.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:13 smithi178 ceph-mon[34813]: Deploying daemon osd.5 on smithi178 2024-09-10T10:39:13.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:13 smithi144 ceph-mon[34230]: Deploying daemon osd.5 on smithi178 2024-09-10T10:39:14.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:14 smithi178 ceph-mon[34813]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:14 smithi143 ceph-mon[31057]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:14 smithi144 ceph-mon[34230]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:16.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:16 smithi143 ceph-mon[31057]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:16 smithi178 ceph-mon[34813]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:16.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:16 smithi144 ceph-mon[34230]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:18.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:18 smithi178 ceph-mon[34813]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:18.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:18 smithi144 ceph-mon[34230]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:18 smithi143 ceph-mon[31057]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:19 smithi143 ceph-mon[31057]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:19 smithi178 ceph-mon[34813]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:19 smithi144 ceph-mon[34230]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:22 smithi143 ceph-mon[31057]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:22.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.566 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:22.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:22.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.567 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:22 smithi178 ceph-mon[34813]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:22.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:22 smithi144 ceph-mon[34230]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:22.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.890+0000 7fba34811640 1 -- 172.21.15.178:0/3533745851 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fba30062630 con 0x7fba180a2b80 2024-09-10T10:39:22.892 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-09-10T10:39:22.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.892+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fba180a2b80 msgr2=0x7fba180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:22.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.892+0000 7fba35813640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fba180a2b80 0x7fba180a5040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba30101d80 tx=0x7fba2400b410 comp rx=0 tx=0).stop 2024-09-10T10:39:22.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.892+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 msgr2=0x7fba30102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.892+0000 7fba35813640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 0x7fba30102000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fba2000d770 tx=0x7fba2000dc30 comp rx=0 tx=0).stop 2024-09-10T10:39:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 shutdown_connections 2024-09-10T10:39:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fba180a2b80 0x7fba180a5040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30107900 0x7fba30100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba30104fd0 0x7fba30100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 --2- 172.21.15.178:0/3533745851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba301026a0 0x7fba30102000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:22.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 >> 172.21.15.178:0/3533745851 conn(0x7fba300fc2a0 msgr2=0x7fba30103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:22.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 shutdown_connections 2024-09-10T10:39:22.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:39:22.893+0000 7fba35813640 1 -- 172.21.15.178:0/3533745851 wait complete. 2024-09-10T10:39:23.549 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T10:39:23.551 INFO:tasks.cephadm:Deploying osd.6 on smithi143 with /dev/vg_nvme/lv_2... 2024-09-10T10:39:23.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T10:39:23.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:23 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:23.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:23 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:23.572 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:23 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:23.579 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:23 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:23.579 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:23 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:23.579 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:23 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:23.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:23 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:23.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:23 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:23.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:23 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:23.806 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:39:24.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:39:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:39:23.992+0000 7f586bb86640 -1 osd.5 0 log_to_monitors true 2024-09-10T10:39:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:24 smithi143 ceph-mon[31057]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:24 smithi143 ceph-mon[31057]: from='osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:39:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:24 smithi143 ceph-mon[31057]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:39:24.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:24 smithi178 ceph-mon[34813]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:24.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:24 smithi178 ceph-mon[34813]: from='osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:39:24.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:24 smithi178 ceph-mon[34813]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:39:24.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:24 smithi144 ceph-mon[34230]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:24.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:24 smithi144 ceph-mon[34230]: from='osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:39:24.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:24 smithi144 ceph-mon[34230]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:39:25.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:25 smithi143 ceph-mon[31057]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T10:39:25.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:25 smithi143 ceph-mon[31057]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T10:39:25.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:25 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:25.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:25 smithi143 ceph-mon[31057]: from='osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:39:25.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:25 smithi143 ceph-mon[31057]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:39:25.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-mon[34813]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T10:39:25.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-mon[34813]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T10:39:25.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:25.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-mon[34813]: from='osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:39:25.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-mon[34813]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:39:25.675 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:39:25.334+0000 7f5867b09640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T10:39:25.675 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:39:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:39:25.339+0000 7f5863931640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:39:25.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:25 smithi144 ceph-mon[34230]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T10:39:25.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:25 smithi144 ceph-mon[34230]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T10:39:25.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:25 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:25.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:25 smithi144 ceph-mon[34230]: from='osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:39:25.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:25 smithi144 ceph-mon[34230]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:39:26.410 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:26 smithi143 ceph-mon[31057]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:39:26.410 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:26 smithi143 ceph-mon[31057]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T10:39:26.410 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.410 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.410 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:26 smithi143 ceph-mon[31057]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:26.410 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:26 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.413 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:39:26.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_2 2024-09-10T10:39:26.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:26 smithi178 ceph-mon[34813]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:39:26.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:26 smithi178 ceph-mon[34813]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T10:39:26.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:26 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:26 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:26 smithi178 ceph-mon[34813]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:26.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:26 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:26 smithi144 ceph-mon[34230]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:39:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:26 smithi144 ceph-mon[34230]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T10:39:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:26 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:26 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:26 smithi144 ceph-mon[34230]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:39:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:26 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:26.729 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:39:27.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 -- 172.21.15.143:0/1203743961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100d90 msgr2=0x7f8e8c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1203743961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100d90 0x7f8e8c1011f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a410 comp rx=0 tx=0).stop 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 -- 172.21.15.143:0/1203743961 shutdown_connections 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1203743961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c101730 0x7f8e8c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1203743961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100d90 0x7f8e8c1011f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1203743961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c100450 0x7f8e8c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 -- 172.21.15.143:0/1203743961 >> 172.21.15.143:0/1203743961 conn(0x7f8e8c0fc280 msgr2=0x7f8e8c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 -- 172.21.15.143:0/1203743961 shutdown_connections 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 -- 172.21.15.143:0/1203743961 wait complete. 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 Processor -- start 2024-09-10T10:39:27.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.288+0000 7f8e9264e640 1 -- start start 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e9264e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 0x7f8e8c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e9264e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c100d90 0x7f8e8c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e9264e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101730 0x7f8e8c1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e9264e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10f8d0 con 0x7f8e8c100450 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e9264e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10f750 con 0x7f8e8c101730 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e9264e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10fa50 con 0x7f8e8c100d90 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 0x7f8e8c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 0x7f8e8c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57010/0 (socket says 172.21.15.143:57010) 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 -- 172.21.15.143:0/1984664250 learned_addr learned my addr 172.21.15.143:0/1984664250 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:39:27.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8b7fe640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c100d90 0x7f8e8c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e90beb640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101730 0x7f8e8c1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 -- 172.21.15.143:0/1984664250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c100d90 msgr2=0x7f8e8c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c100d90 0x7f8e8c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 -- 172.21.15.143:0/1984664250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101730 msgr2=0x7f8e8c1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101730 0x7f8e8c1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.289+0000 7f8e8bfff640 1 -- 172.21.15.143:0/1984664250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1a1c40 con 0x7f8e8c100450 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.290+0000 7f8e8bfff640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 0x7f8e8c19a010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c890 tx=0x7f8e7c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.290+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004220 con 0x7f8e8c100450 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.290+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a1f30 con 0x7f8e8c100450 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.290+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c0043c0 con 0x7f8e8c100450 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.290+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004a00 con 0x7f8e8c100450 2024-09-10T10:39:27.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.290+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a2390 con 0x7f8e8c100450 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.291+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c10b370 con 0x7f8e8c100450 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.292+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8e7c004ba0 con 0x7f8e8c100450 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.292+0000 7f8e897fa640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8e740a2bb0 0x7f8e740a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.293+0000 7f8e8b7fe640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8e740a2bb0 0x7f8e740a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.293+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f8e7c0cde40 con 0x7f8e8c100450 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.293+0000 7f8e8b7fe640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8e740a2bb0 0x7f8e740a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006010 tx=0x7f8e80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:27.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.297+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e7c014030 con 0x7f8e8c100450 2024-09-10T10:39:27.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:27.483+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8e8c062630 con 0x7f8e740a2bb0 2024-09-10T10:39:27.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:39:27.552 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] boot 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi178 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi178 to 8543M 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:27.553 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] boot 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi178 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:27.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:27.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi178 to 8543M 2024-09-10T10:39:27.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:27.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:27.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: osd.5 [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] boot 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi178 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:27.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:27.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi178 to 8543M 2024-09-10T10:39:27.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:27.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:27.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:27.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:28.637 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:28 smithi143 ceph-mon[31057]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T10:39:28.637 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:28 smithi143 ceph-mon[31057]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:28.637 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:28 smithi143 ceph-mon[31057]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:28.637 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:28 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:28.637 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:28 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:28.638 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:28 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:28.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:28 smithi178 ceph-mon[34813]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T10:39:28.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:28 smithi178 ceph-mon[34813]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:28.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:28 smithi178 ceph-mon[34813]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:28.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:28 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:28.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:28 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:28.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:28 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:28 smithi144 ceph-mon[34230]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T10:39:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:28 smithi144 ceph-mon[34230]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:28 smithi144 ceph-mon[34230]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:28 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:28 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:28 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:29.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:29 smithi143 ceph-mon[31057]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T10:39:29.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:29 smithi178 ceph-mon[34813]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T10:39:29.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:29 smithi144 ceph-mon[34230]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T10:39:30.516 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:30 smithi143 ceph-mon[31057]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T10:39:30.517 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:30 smithi143 ceph-mon[31057]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:30.517 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:30 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4051424847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b57aa64-3a10-4155-a2c6-984a15789ab4"}]: dispatch 2024-09-10T10:39:30.517 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:30 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4051424847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b57aa64-3a10-4155-a2c6-984a15789ab4"}]': finished 2024-09-10T10:39:30.517 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:30 smithi143 ceph-mon[31057]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T10:39:30.517 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:30.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:30 smithi178 ceph-mon[34813]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T10:39:30.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:30 smithi178 ceph-mon[34813]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:30.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:30 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4051424847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b57aa64-3a10-4155-a2c6-984a15789ab4"}]: dispatch 2024-09-10T10:39:30.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:30 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4051424847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b57aa64-3a10-4155-a2c6-984a15789ab4"}]': finished 2024-09-10T10:39:30.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:30 smithi178 ceph-mon[34813]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T10:39:30.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:30 smithi144 ceph-mon[34230]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T10:39:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:30 smithi144 ceph-mon[34230]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:30 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4051424847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b57aa64-3a10-4155-a2c6-984a15789ab4"}]: dispatch 2024-09-10T10:39:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:30 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4051424847' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b57aa64-3a10-4155-a2c6-984a15789ab4"}]': finished 2024-09-10T10:39:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:30 smithi144 ceph-mon[34230]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T10:39:30.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:31.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:31 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3239144432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:31.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:31 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3239144432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:31.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:31 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3239144432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:32.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:32 smithi178 ceph-mon[34813]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-09-10T10:39:32.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:32 smithi144 ceph-mon[34230]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-09-10T10:39:32.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:32 smithi143 ceph-mon[31057]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-09-10T10:39:34.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:34 smithi144 ceph-mon[34230]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:39:34.746 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:34 smithi143 ceph-mon[31057]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:39:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:34 smithi178 ceph-mon[34813]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:39:36.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:36 smithi143 ceph-mon[31057]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T10:39:36.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:36 smithi144 ceph-mon[34230]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T10:39:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:36 smithi178 ceph-mon[34813]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T10:39:38.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:38 smithi178 ceph-mon[34813]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:39:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:38 smithi143 ceph-mon[31057]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:39:38.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:38 smithi144 ceph-mon[34230]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:39:39.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T10:39:39.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T10:39:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:39.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T10:39:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:40.661 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:40 smithi143 ceph-mon[31057]: Deploying daemon osd.6 on smithi143 2024-09-10T10:39:40.661 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:40 smithi143 ceph-mon[31057]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:39:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:40 smithi178 ceph-mon[34813]: Deploying daemon osd.6 on smithi143 2024-09-10T10:39:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:40 smithi178 ceph-mon[34813]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:39:40.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:40 smithi144 ceph-mon[34230]: Deploying daemon osd.6 on smithi143 2024-09-10T10:39:40.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:40 smithi144 ceph-mon[34230]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:39:42.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:42 smithi143 ceph-mon[31057]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:39:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:42 smithi178 ceph-mon[34813]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:39:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:42 smithi144 ceph-mon[34230]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:39:44.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:44 smithi143 ceph-mon[31057]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:44 smithi178 ceph-mon[34813]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:44.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:44 smithi144 ceph-mon[34230]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:45.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:45.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:45.498 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:45.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:45.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:45.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:45.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:45.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:45.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:46.608 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:46 smithi143 ceph-mon[31057]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:46 smithi178 ceph-mon[34813]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:46.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:46 smithi144 ceph-mon[34230]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:48.733 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:48 smithi143 ceph-mon[31057]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:48.733 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:48.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:48.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.773+0000 7f8e897fa640 1 -- 172.21.15.143:0/1984664250 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8e8c062630 con 0x7f8e740a2bb0 2024-09-10T10:39:48.775 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 6 on host 'smithi143' 2024-09-10T10:39:48.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.776+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8e740a2bb0 msgr2=0x7f8e740a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:48.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.776+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8e740a2bb0 0x7f8e740a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006010 tx=0x7f8e80000bc0 comp rx=0 tx=0).stop 2024-09-10T10:39:48.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.776+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 msgr2=0x7f8e8c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:48.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.776+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 0x7f8e8c19a010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c890 tx=0x7f8e7c00cd50 comp rx=0 tx=0).stop 2024-09-10T10:39:48.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 shutdown_connections 2024-09-10T10:39:48.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8e740a2bb0 0x7f8e740a5070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:48.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e8c101730 0x7f8e8c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:48.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e8c100d90 0x7f8e8c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:48.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 --2- 172.21.15.143:0/1984664250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e8c100450 0x7f8e8c19a010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:48.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 >> 172.21.15.143:0/1984664250 conn(0x7f8e8c0fc280 msgr2=0x7f8e8c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:48.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 shutdown_connections 2024-09-10T10:39:48.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:39:48.777+0000 7f8e9264e640 1 -- 172.21.15.143:0/1984664250 wait complete. 2024-09-10T10:39:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:48 smithi178 ceph-mon[34813]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:48.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:48.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:48.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:48 smithi144 ceph-mon[34230]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:49.479 DEBUG:teuthology.orchestra.run.smithi143:osd.6> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T10:39:49.482 INFO:tasks.cephadm:Deploying osd.7 on smithi144 with /dev/vg_nvme/lv_2... 2024-09-10T10:39:49.482 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T10:39:49.616 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:49.617 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:49.617 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:49.742 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:39:49.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:49.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:49.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:49.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:49.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:50.594 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:50 smithi143 ceph-mon[31057]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:50 smithi178 ceph-mon[34813]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:50 smithi144 ceph-mon[34230]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:51.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:39:50 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:39:50.875+0000 7fd112b82640 -1 osd.6 0 log_to_monitors true 2024-09-10T10:39:51.753 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:51 smithi144 ceph-mon[34230]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T10:39:51.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:51 smithi143 ceph-mon[31057]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T10:39:51.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:51 smithi178 ceph-mon[34813]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.470 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:52.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:52.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.667 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.668 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.668 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:52.668 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:52.668 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.669 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:39:52.696 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-09-10T10:39:52.800 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:39:52.467+0000 7fd10eb05640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T10:39:52.800 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:39:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:39:52.480+0000 7fd10a92d640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:52.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:52.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:39:52.950 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:39:53.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 -- 172.21.15.144:0/2588240374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede41026d0 msgr2=0x7fede4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:53.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2588240374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede41026d0 0x7fede4108f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feddc009ca0 tx=0x7feddc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:39:53.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 -- 172.21.15.144:0/2588240374 shutdown_connections 2024-09-10T10:39:53.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2588240374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede41026d0 0x7fede4108f60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:53.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2588240374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede4101d10 0x7fede4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:53.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2588240374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fede4100b10 0x7fede4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 -- 172.21.15.144:0/2588240374 >> 172.21.15.144:0/2588240374 conn(0x7fede40fc2a0 msgr2=0x7fede40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 -- 172.21.15.144:0/2588240374 shutdown_connections 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 -- 172.21.15.144:0/2588240374 wait complete. 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 Processor -- start 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.514+0000 7fedeb59f640 1 -- start start 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fede4100b10 0x7fede4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 0x7fede4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede41026d0 0x7fede419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fede410c6c0 con 0x7fede41026d0 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fede410c540 con 0x7fede4101d10 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fede410c840 con 0x7fede4100b10 2024-09-10T10:39:53.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 0x7fede4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 0x7fede4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42036/0 (socket says 172.21.15.144:42036) 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 -- 172.21.15.144:0/2964350934 learned_addr learned my addr 172.21.15.144:0/2964350934 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede933b640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fede4100b10 0x7fede4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede9b3c640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede41026d0 0x7fede419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 -- 172.21.15.144:0/2964350934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fede4100b10 msgr2=0x7fede4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fede4100b10 0x7fede4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 -- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede41026d0 msgr2=0x7fede419f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede41026d0 0x7fede419f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 -- 172.21.15.144:0/2964350934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede419fa30 con 0x7fede4101d10 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede9b3c640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede41026d0 0x7fede419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fede8b3a640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 0x7fede4198340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fede000c920 tx=0x7fede000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0007c70 con 0x7fede4101d10 2024-09-10T10:39:53.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede419fd20 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fede0007e10 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0018820 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.515+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede41a0200 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.517+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede4108a50 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.517+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fede00189c0 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.518+0000 7fedd67fc640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fedd00a2bb0 0x7fedd00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.518+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fede00ce500 con 0x7fede4101d10 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.518+0000 7fede933b640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fedd00a2bb0 0x7fedd00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.519+0000 7fede933b640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fedd00a2bb0 0x7fedd00a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fede4101b70 tx=0x7fedd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.522+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fede0014030 con 0x7fede4101d10 2024-09-10T10:39:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:53 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi143 2024-09-10T10:39:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:53 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi143 to 5696M 2024-09-10T10:39:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:53 smithi144 ceph-mon[34230]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:39:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:53 smithi144 ceph-mon[34230]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T10:39:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:53.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:39:53.694+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fede4062630 con 0x7fedd00a2bb0 2024-09-10T10:39:53.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:53 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi143 2024-09-10T10:39:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:53 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi143 to 5696M 2024-09-10T10:39:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:53 smithi143 ceph-mon[31057]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:39:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:53 smithi143 ceph-mon[31057]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T10:39:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:53.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:53 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi143 2024-09-10T10:39:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:53 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi143 to 5696M 2024-09-10T10:39:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:53 smithi178 ceph-mon[34813]: from='osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:39:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:53 smithi178 ceph-mon[34813]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T10:39:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:39:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:39:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] boot 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:54.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:54 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] boot 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:54.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:54 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: osd.6 [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] boot 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:54 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:55.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:55 smithi144 ceph-mon[34230]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:55.624 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:55 smithi144 ceph-mon[34230]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T10:39:55.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:55 smithi143 ceph-mon[31057]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:55.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:55 smithi143 ceph-mon[31057]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T10:39:55.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:55 smithi178 ceph-mon[34813]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:55.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:55 smithi178 ceph-mon[34813]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T10:39:56.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:56 smithi144 ceph-mon[34230]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:39:56.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:56 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/3341767386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]: dispatch 2024-09-10T10:39:56.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:56 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]: dispatch 2024-09-10T10:39:56.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:56 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]': finished 2024-09-10T10:39:56.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:56 smithi144 ceph-mon[34230]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T10:39:56.645 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:39:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:56 smithi143 ceph-mon[31057]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:39:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:56 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/3341767386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]: dispatch 2024-09-10T10:39:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:56 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]: dispatch 2024-09-10T10:39:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:56 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]': finished 2024-09-10T10:39:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:56 smithi143 ceph-mon[31057]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T10:39:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:39:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:56 smithi178 ceph-mon[34813]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:39:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:56 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/3341767386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]: dispatch 2024-09-10T10:39:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:56 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]: dispatch 2024-09-10T10:39:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:56 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680d5255-bade-46c7-a4be-00bfb073eafb"}]': finished 2024-09-10T10:39:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:56 smithi178 ceph-mon[34813]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T10:39:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:39:57.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:57 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/620750278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:57 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/620750278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:57.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:57 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/620750278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:58.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:39:58 smithi143 ceph-mon[31057]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:39:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:39:58 smithi178 ceph-mon[34813]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:39:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:39:58 smithi144 ceph-mon[34230]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:00 smithi143 ceph-mon[31057]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:00 smithi143 ceph-mon[31057]: overall HEALTH_OK 2024-09-10T10:40:00.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:00 smithi144 ceph-mon[34230]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:00.863 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:00 smithi144 ceph-mon[34230]: overall HEALTH_OK 2024-09-10T10:40:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:00 smithi178 ceph-mon[34813]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:00 smithi178 ceph-mon[34813]: overall HEALTH_OK 2024-09-10T10:40:02.633 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:02 smithi144 ceph-mon[34230]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:02 smithi143 ceph-mon[31057]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:02 smithi178 ceph-mon[34813]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:04.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:04 smithi143 ceph-mon[31057]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:04 smithi178 ceph-mon[34813]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:04.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:04 smithi144 ceph-mon[34230]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:05.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T10:40:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:05.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T10:40:05.811 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:05.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T10:40:05.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:06.663 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:06 smithi144 ceph-mon[34230]: Deploying daemon osd.7 on smithi144 2024-09-10T10:40:06.663 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:06 smithi144 ceph-mon[34230]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:06.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:06 smithi143 ceph-mon[31057]: Deploying daemon osd.7 on smithi144 2024-09-10T10:40:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:06 smithi143 ceph-mon[31057]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:06 smithi178 ceph-mon[34813]: Deploying daemon osd.7 on smithi144 2024-09-10T10:40:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:06 smithi178 ceph-mon[34813]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:08.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:08 smithi143 ceph-mon[31057]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:08 smithi178 ceph-mon[34813]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:08 smithi144 ceph-mon[34230]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:10.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:10 smithi144 ceph-mon[34230]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:10.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:10 smithi143 ceph-mon[31057]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:10 smithi178 ceph-mon[34813]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:11.798 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:11.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:11.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:11.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:11.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:11.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:11.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:11.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:12.693 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:12 smithi144 ceph-mon[34230]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:12 smithi143 ceph-mon[31057]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:12 smithi178 ceph-mon[34813]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:14.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.394+0000 7fedd67fc640 1 -- 172.21.15.144:0/2964350934 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fede4062630 con 0x7fedd00a2bb0 2024-09-10T10:40:14.396 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 7 on host 'smithi144' 2024-09-10T10:40:14.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fedd00a2bb0 msgr2=0x7fedd00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:14.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fedd00a2bb0 0x7fedd00a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fede4101b70 tx=0x7fedd8008040 comp rx=0 tx=0).stop 2024-09-10T10:40:14.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 msgr2=0x7fede4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:14.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 0x7fede4198340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fede000c920 tx=0x7fede000cde0 comp rx=0 tx=0).stop 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 shutdown_connections 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fedd00a2bb0 0x7fedd00a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fede41026d0 0x7fede419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fede4101d10 0x7fede4198340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 --2- 172.21.15.144:0/2964350934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fede4100b10 0x7fede4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 >> 172.21.15.144:0/2964350934 conn(0x7fede40fc2a0 msgr2=0x7fede40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 shutdown_connections 2024-09-10T10:40:14.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:40:14.397+0000 7fedeb59f640 1 -- 172.21.15.144:0/2964350934 wait complete. 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:14.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:15.051 DEBUG:teuthology.orchestra.run.smithi144:osd.7> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T10:40:15.055 INFO:tasks.cephadm:Deploying osd.8 on smithi178 with /dev/vg_nvme/lv_2... 2024-09-10T10:40:15.056 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T10:40:15.310 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:40:16.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:16 smithi144 ceph-mon[34230]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:16.500 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:40:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:40:16.473+0000 7f8311ddf640 -1 osd.7 0 log_to_monitors true 2024-09-10T10:40:16.664 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:16 smithi178 ceph-mon[34813]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:16.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:16 smithi143 ceph-mon[31057]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:17.464 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:17 smithi178 ceph-mon[34813]: from='osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:40:17.465 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:17 smithi178 ceph-mon[34813]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:40:17.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:17 smithi144 ceph-mon[34230]: from='osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:40:17.640 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:17 smithi144 ceph-mon[34230]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:40:17.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:17 smithi143 ceph-mon[31057]: from='osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:40:17.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:17 smithi143 ceph-mon[31057]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:40:18.142 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:40:18.167 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-09-10T10:40:18.411 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:40:18.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:18.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:18.778 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 -- 172.21.15.178:0/3812934021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce4101c00 msgr2=0x7f8ce4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/3812934021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce4101c00 0x7f8ce4102000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009900 tx=0x7f8cd802a410 comp rx=0 tx=0).stop 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 -- 172.21.15.178:0/3812934021 shutdown_connections 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/3812934021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ce41037c0 0x7f8ce410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/3812934021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4102e00 0x7f8ce4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/3812934021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce4101c00 0x7f8ce4102000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 -- 172.21.15.178:0/3812934021 >> 172.21.15.178:0/3812934021 conn(0x7f8ce40fd3b0 msgr2=0x7f8ce40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:18.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 -- 172.21.15.178:0/3812934021 shutdown_connections 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.857+0000 7f8ceac8d640 1 -- 172.21.15.178:0/3812934021 wait complete. 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 Processor -- start 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 -- start start 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4101c00 0x7f8ce4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ce4102e00 0x7f8ce4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 0x7f8ce419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410d710 con 0x7f8ce4101c00 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410d590 con 0x7f8ce4102e00 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ceac8d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce410d890 con 0x7f8ce41037c0 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 0x7f8ce419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 0x7f8ce419f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53870/0 (socket says 172.21.15.178:53870) 2024-09-10T10:40:18.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 -- 172.21.15.178:0/240579935 learned_addr learned my addr 172.21.15.178:0/240579935 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce3fff640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ce4102e00 0x7f8ce4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce8a29640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4101c00 0x7f8ce4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 -- 172.21.15.178:0/240579935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ce4102e00 msgr2=0x7f8ce4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ce4102e00 0x7f8ce4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 -- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4101c00 msgr2=0x7f8ce4197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4101c00 0x7f8ce4197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce922a640 1 -- 172.21.15.178:0/240579935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce419fa10 con 0x7f8ce41037c0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.858+0000 7f8ce8a29640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4101c00 0x7f8ce4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.859+0000 7f8ce922a640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 0x7f8ce419f3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eaa0 tx=0x7f8cdc00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.859+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00cdb0 con 0x7f8ce41037c0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.859+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce419fd00 con 0x7f8ce41037c0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.859+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cdc005020 con 0x7f8ce41037c0 2024-09-10T10:40:18.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.859+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0116d0 con 0x7f8ce41037c0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.859+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a0190 con 0x7f8ce41037c0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.860+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce4108ff0 con 0x7f8ce41037c0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.861+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8cdc0051d0 con 0x7f8ce41037c0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.861+0000 7f8ce1ffb640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cd40a2bb0 0x7f8cd40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.862+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f8cdc0ce540 con 0x7f8ce41037c0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.862+0000 7f8ce8a29640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cd40a2bb0 0x7f8cd40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.862+0000 7f8ce8a29640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cd40a2bb0 0x7f8cd40a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006010 tx=0x7f8cd8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:18.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:18.866+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cdc016030 con 0x7f8ce41037c0 2024-09-10T10:40:18.936 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:40:18.445+0000 7f830dd62640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T10:40:18.936 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:40:18.452+0000 7f8309b8a640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:18.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:18.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:18.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:18.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:19.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:19.041+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8ce4062630 con 0x7f8cd40a2bb0 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi144 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi144 to 5696M 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:19.788 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:19 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:19.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi144 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi144 to 5696M 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:19 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi144 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi144 to 5696M 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:19.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:19 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:20.776 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:40:20.776 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:40:20.776 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:20.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:20.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:20.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] boot 2024-09-10T10:40:20.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T10:40:20.777 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:20 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:20.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:40:20.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:40:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] boot 2024-09-10T10:40:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T10:40:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:20 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: osd.7 [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] boot 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T10:40:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:20 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:40:21.768 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:21 smithi178 ceph-mon[34813]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T10:40:21.769 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:21 smithi178 ceph-mon[34813]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T10:40:21.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:21 smithi143 ceph-mon[31057]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T10:40:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:21 smithi143 ceph-mon[31057]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T10:40:21.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:21 smithi144 ceph-mon[34230]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T10:40:21.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:21 smithi144 ceph-mon[34230]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T10:40:22.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/3593385557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]: dispatch 2024-09-10T10:40:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]: dispatch 2024-09-10T10:40:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]': finished 2024-09-10T10:40:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T10:40:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:22 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/3766954835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/3593385557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]: dispatch 2024-09-10T10:40:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]: dispatch 2024-09-10T10:40:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]': finished 2024-09-10T10:40:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T10:40:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:22 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/3766954835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/3593385557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]: dispatch 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]: dispatch 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f707e74d-dfb8-4966-802f-5df28f3ac0e9"}]': finished 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:22 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/3766954835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:23.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:23 smithi143 ceph-mon[31057]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T10:40:23.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:23 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:23 smithi178 ceph-mon[34813]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T10:40:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:23 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:23.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:23 smithi144 ceph-mon[34230]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T10:40:23.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:23 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:24.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:24 smithi143 ceph-mon[31057]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:24 smithi178 ceph-mon[34813]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:24 smithi144 ceph-mon[34230]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:26.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:26 smithi178 ceph-mon[34813]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:26.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:26 smithi143 ceph-mon[31057]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:26 smithi144 ceph-mon[34230]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:28.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:28 smithi143 ceph-mon[31057]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:28.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:28 smithi178 ceph-mon[34813]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:28 smithi144 ceph-mon[34230]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:30.644 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:30 smithi178 ceph-mon[34813]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:30.644 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T10:40:30.644 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:30 smithi143 ceph-mon[31057]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T10:40:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:30 smithi144 ceph-mon[34230]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T10:40:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:31.632 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:31 smithi178 ceph-mon[34813]: Deploying daemon osd.8 on smithi178 2024-09-10T10:40:31.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:31 smithi143 ceph-mon[31057]: Deploying daemon osd.8 on smithi178 2024-09-10T10:40:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:31 smithi144 ceph-mon[34230]: Deploying daemon osd.8 on smithi178 2024-09-10T10:40:32.555 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:32 smithi178 ceph-mon[34813]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:32 smithi143 ceph-mon[31057]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:32 smithi144 ceph-mon[34230]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:34.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:34 smithi178 ceph-mon[34813]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:34.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:34 smithi143 ceph-mon[31057]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:34 smithi144 ceph-mon[34230]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:36.671 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:36 smithi178 ceph-mon[34813]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:36.671 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:36.672 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:36.672 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:36.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:36 smithi143 ceph-mon[31057]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:36 smithi144 ceph-mon[34230]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:38.709 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:38 smithi178 ceph-mon[34813]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:38 smithi143 ceph-mon[31057]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:38.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:38 smithi144 ceph-mon[34230]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:39.807 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 8 on host 'smithi178' 2024-09-10T10:40:39.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.804+0000 7f8ce1ffb640 1 -- 172.21.15.178:0/240579935 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ce4062630 con 0x7f8cd40a2bb0 2024-09-10T10:40:39.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cd40a2bb0 msgr2=0x7f8cd40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:39.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cd40a2bb0 0x7f8cd40a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006010 tx=0x7f8cd8000bc0 comp rx=0 tx=0).stop 2024-09-10T10:40:39.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 msgr2=0x7f8ce419f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:39.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 0x7f8ce419f3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00eaa0 tx=0x7f8cdc00ef60 comp rx=0 tx=0).stop 2024-09-10T10:40:39.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 shutdown_connections 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cd40a2bb0 0x7f8cd40a5070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ce41037c0 0x7f8ce419f3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ce4102e00 0x7f8ce4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 --2- 172.21.15.178:0/240579935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ce4101c00 0x7f8ce4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 >> 172.21.15.178:0/240579935 conn(0x7f8ce40fd3b0 msgr2=0x7f8ce41061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 shutdown_connections 2024-09-10T10:40:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:40:39.807+0000 7f8ceac8d640 1 -- 172.21.15.178:0/240579935 wait complete. 2024-09-10T10:40:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:39 smithi143 ceph-mon[31057]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:39 smithi178 ceph-mon[34813]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:39 smithi144 ceph-mon[34230]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:40.521 DEBUG:teuthology.orchestra.run.smithi178:osd.8> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T10:40:40.523 INFO:tasks.cephadm:Deploying osd.9 on smithi143 with /dev/vg_nvme/lv_1... 2024-09-10T10:40:40.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:40:40.778 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:40:40.899 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:40.900 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:40.900 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:41.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:41.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:41.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:41.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:41.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:41.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:41.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:41 smithi178 ceph-mon[34813]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:42.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:41 smithi143 ceph-mon[31057]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:42.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:41 smithi144 ceph-mon[34230]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:42.425 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:40:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:40:42.065+0000 7fa89d441640 -1 osd.8 0 log_to_monitors true 2024-09-10T10:40:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:42 smithi178 ceph-mon[34813]: from='osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:40:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:42 smithi178 ceph-mon[34813]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:40:43.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:42 smithi143 ceph-mon[31057]: from='osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:40:43.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:42 smithi143 ceph-mon[31057]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:40:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:42 smithi144 ceph-mon[34230]: from='osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:40:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:42 smithi144 ceph-mon[34230]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:40:43.693 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:40:43.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_1 2024-09-10T10:40:43.984 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi178 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi178 to 5695M 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:43 smithi143 ceph-mon[31057]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:40:44.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T10:40:44.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T10:40:44.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:40:44.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:44.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:40:44.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi178 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi178 to 5695M 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:44.177 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:43 smithi178 ceph-mon[34813]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi178 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi178 to 5695M 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:44.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:43 smithi144 ceph-mon[34230]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:40:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 -- 172.21.15.143:0/2980506573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94100ad0 msgr2=0x7fbc94100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 --2- 172.21.15.143:0/2980506573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94100ad0 0x7fbc94100ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009900 tx=0x7fbc8802a410 comp rx=0 tx=0).stop 2024-09-10T10:40:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 -- 172.21.15.143:0/2980506573 shutdown_connections 2024-09-10T10:40:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 --2- 172.21.15.143:0/2980506573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc94102690 0x7fbc94108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 --2- 172.21.15.143:0/2980506573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94101cd0 0x7fbc94102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 --2- 172.21.15.143:0/2980506573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94100ad0 0x7fbc94100ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 -- 172.21.15.143:0/2980506573 >> 172.21.15.143:0/2980506573 conn(0x7fbc940fc280 msgr2=0x7fbc940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 -- 172.21.15.143:0/2980506573 shutdown_connections 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 -- 172.21.15.143:0/2980506573 wait complete. 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.469+0000 7fbc99952640 1 Processor -- start 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 -- start start 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94100ad0 0x7fbc9406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc94101cd0 0x7fbc9419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 0x7fbc9419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410c600 con 0x7fbc94102690 2024-09-10T10:40:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410c480 con 0x7fbc94101cd0 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc99952640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410c780 con 0x7fbc94100ad0 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 0x7fbc9419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 0x7fbc9419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60536/0 (socket says 172.21.15.143:60536) 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 -- 172.21.15.143:0/2853609463 learned_addr learned my addr 172.21.15.143:0/2853609463 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc92ffd640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94100ad0 0x7fbc9406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc927fc640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc94101cd0 0x7fbc9419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 -- 172.21.15.143:0/2853609463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94100ad0 msgr2=0x7fbc9406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94100ad0 0x7fbc9406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 -- 172.21.15.143:0/2853609463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc94101cd0 msgr2=0x7fbc9419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc94101cd0 0x7fbc9419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.470+0000 7fbc937fe640 1 -- 172.21.15.143:0/2853609463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9419fb00 con 0x7fbc94102690 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.471+0000 7fbc937fe640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 0x7fbc9419f1f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cad0 tx=0x7fbc8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.471+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00ae20 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.471+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9419fd90 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.471+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc8c005020 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.471+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00f6b0 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.471+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc941a0280 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.472+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc94107f60 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.473+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbc8c0051d0 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.474+0000 7fbc98950640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc7c0a2b80 0x7fbc7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.474+0000 7fbc92ffd640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc7c0a2b80 0x7fbc7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.474+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7fbc8c0ce5a0 con 0x7fbc94102690 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.474+0000 7fbc92ffd640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc7c0a2b80 0x7fbc7c0a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc88004770 tx=0x7fbc88000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.478+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc8c014030 con 0x7fbc94102690 2024-09-10T10:40:44.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:40:44.660+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbc94062630 con 0x7fbc7c0a2b80 2024-09-10T10:40:44.674 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:40:44.267+0000 7fa899bd8640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T10:40:44.674 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:40:44.276+0000 7fa8911e4640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: from='osd.8 ' entity='osd.8' 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:45.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:45.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T10:40:45.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:45.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:45.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: from='osd.8 ' entity='osd.8' 2024-09-10T10:40:45.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:45.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:45.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: from='osd.8 ' entity='osd.8' 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:45.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] boot 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:46.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:45 smithi143 ceph-mon[31057]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:46.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:40:46.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:40:46.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:46.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:46.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] boot 2024-09-10T10:40:46.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T10:40:46.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:46.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:45 smithi178 ceph-mon[34813]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: osd.8 [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] boot 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:40:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:45 smithi144 ceph-mon[34230]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:40:47.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:46 smithi143 ceph-mon[31057]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T10:40:47.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:46 smithi178 ceph-mon[34813]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T10:40:47.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:46 smithi144 ceph-mon[34230]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/884263391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3005ede3-bfda-4cfd-a562-8c1540c829af"}]: dispatch 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/884263391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3005ede3-bfda-4cfd-a562-8c1540c829af"}]': finished 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:40:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:47 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3565188882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:48.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T10:40:48.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/884263391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3005ede3-bfda-4cfd-a562-8c1540c829af"}]: dispatch 2024-09-10T10:40:48.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/884263391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3005ede3-bfda-4cfd-a562-8c1540c829af"}]': finished 2024-09-10T10:40:48.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T10:40:48.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:40:48.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:40:48.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:47 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3565188882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:48.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T10:40:48.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/884263391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3005ede3-bfda-4cfd-a562-8c1540c829af"}]: dispatch 2024-09-10T10:40:48.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/884263391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3005ede3-bfda-4cfd-a562-8c1540c829af"}]': finished 2024-09-10T10:40:48.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T10:40:48.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:40:48.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:40:48.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:47 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3565188882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:49.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:49 smithi178 ceph-mon[34813]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T10:40:49.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:40:49.435 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:49 smithi144 ceph-mon[34230]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T10:40:49.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:40:49.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:49 smithi143 ceph-mon[31057]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T10:40:49.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:40:50.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:50 smithi178 ceph-mon[34813]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:40:50.435 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:50 smithi144 ceph-mon[34230]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:40:50.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:50 smithi143 ceph-mon[31057]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:40:52.752 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:52 smithi143 ceph-mon[31057]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T10:40:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:52 smithi178 ceph-mon[34813]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T10:40:52.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:52 smithi144 ceph-mon[34230]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T10:40:54.592 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:54 smithi143 ceph-mon[31057]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T10:40:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:54 smithi178 ceph-mon[34813]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T10:40:54.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:54 smithi144 ceph-mon[34230]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T10:40:56.547 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:56 smithi143 ceph-mon[31057]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T10:40:56.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T10:40:56.548 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:56 smithi178 ceph-mon[34813]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T10:40:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T10:40:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:56 smithi144 ceph-mon[34230]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T10:40:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T10:40:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:57.708 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:57 smithi143 ceph-mon[31057]: Deploying daemon osd.9 on smithi143 2024-09-10T10:40:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:57 smithi178 ceph-mon[34813]: Deploying daemon osd.9 on smithi143 2024-09-10T10:40:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:57 smithi144 ceph-mon[34230]: Deploying daemon osd.9 on smithi143 2024-09-10T10:40:58.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:40:58 smithi143 ceph-mon[31057]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:40:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:40:58 smithi178 ceph-mon[34813]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:40:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:40:58 smithi144 ceph-mon[34230]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:41:00.785 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:00 smithi143 ceph-mon[31057]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T10:41:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:00 smithi178 ceph-mon[34813]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T10:41:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:00 smithi144 ceph-mon[34230]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:02 smithi143 ceph-mon[31057]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:02.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:02.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:02 smithi178 ceph-mon[34813]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:02.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:02 smithi144 ceph-mon[34230]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:04.749 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:04 smithi143 ceph-mon[31057]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:04 smithi178 ceph-mon[34813]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:04 smithi144 ceph-mon[34230]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:05.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.528+0000 7fbc98950640 1 -- 172.21.15.143:0/2853609463 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc94062630 con 0x7fbc7c0a2b80 2024-09-10T10:41:05.530 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 9 on host 'smithi143' 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc7c0a2b80 msgr2=0x7fbc7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc7c0a2b80 0x7fbc7c0a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc88004770 tx=0x7fbc88000bc0 comp rx=0 tx=0).stop 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 msgr2=0x7fbc9419f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 0x7fbc9419f1f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cad0 tx=0x7fbc8c00cf90 comp rx=0 tx=0).stop 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 shutdown_connections 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc7c0a2b80 0x7fbc7c0a5040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc94102690 0x7fbc9419f1f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc94101cd0 0x7fbc9419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.530+0000 7fbc99952640 1 --2- 172.21.15.143:0/2853609463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94100ad0 0x7fbc9406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:05.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.531+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 >> 172.21.15.143:0/2853609463 conn(0x7fbc940fc280 msgr2=0x7fbc94105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:05.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.531+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 shutdown_connections 2024-09-10T10:41:05.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:05.531+0000 7fbc99952640 1 -- 172.21.15.143:0/2853609463 wait complete. 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:05 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:05 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:05 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:06.444 DEBUG:teuthology.orchestra.run.smithi143:osd.9> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T10:41:06.446 INFO:tasks.cephadm:Deploying osd.10 on smithi144 with /dev/vg_nvme/lv_1... 2024-09-10T10:41:06.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:41:06.704 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:41:07.800 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:41:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:41:07.589+0000 7f88eb934640 -1 osd.9 0 log_to_monitors true 2024-09-10T10:41:08.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:08 smithi143 ceph-mon[31057]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:08.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:08 smithi143 ceph-mon[31057]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T10:41:08.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:08 smithi178 ceph-mon[34813]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:08.878 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:08 smithi178 ceph-mon[34813]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T10:41:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:08 smithi144 ceph-mon[34230]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:08 smithi144 ceph-mon[34230]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T10:41:09.258 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T10:41:09.285 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-09-10T10:41:09.539 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:41:09.570 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:09.571 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.800 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:41:09.458+0000 7f88e78b7640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T10:41:09.801 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:41:09.468+0000 7f88e36df640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:09.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T10:41:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T10:41:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-09-10T10:41:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:09.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:09.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:09.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:09.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:09.927 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 -- 172.21.15.144:0/3725668744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c601026c0 msgr2=0x7f0c60108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 --2- 172.21.15.144:0/3725668744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c601026c0 0x7f0c60108f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009ca0 tx=0x7f0c5802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 -- 172.21.15.144:0/3725668744 shutdown_connections 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 --2- 172.21.15.144:0/3725668744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c601026c0 0x7f0c60108f50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 --2- 172.21.15.144:0/3725668744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c60101d00 0x7f0c60102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 --2- 172.21.15.144:0/3725668744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 0x7f0c60100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.985+0000 7f0c6790a640 1 -- 172.21.15.144:0/3725668744 >> 172.21.15.144:0/3725668744 conn(0x7f0c600fc270 msgr2=0x7f0c600fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 -- 172.21.15.144:0/3725668744 shutdown_connections 2024-09-10T10:41:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 -- 172.21.15.144:0/3725668744 wait complete. 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 Processor -- start 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 -- start start 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 0x7f0c6019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 0x7f0c6019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c601026c0 0x7f0c601a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6010c630 con 0x7f0c601026c0 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6010c4b0 con 0x7f0c60101d00 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c6790a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6010c7b0 con 0x7f0c60100b00 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c64ea5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 0x7f0c6019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c64ea5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 0x7f0c6019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58904/0 (socket says 172.21.15.144:58904) 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.986+0000 7f0c64ea5640 1 -- 172.21.15.144:0/934732645 learned_addr learned my addr 172.21.15.144:0/934732645 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 0x7f0c6019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c65ea7640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c601026c0 0x7f0c601a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 -- 172.21.15.144:0/934732645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 msgr2=0x7f0c6019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 0x7f0c6019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 -- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c601026c0 msgr2=0x7f0c601a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c601026c0 0x7f0c601a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 -- 172.21.15.144:0/934732645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601a1c60 con 0x7f0c60100b00 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c64ea5640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 0x7f0c6019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c656a6640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 0x7f0c6019a030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00e9c0 tx=0x7f0c5c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00ccc0 con 0x7f0c60100b00 2024-09-10T10:41:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c0035f0 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601a1f50 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00ce60 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.987+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c601a23a0 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.988+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c601a2630 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.989+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0c5c0037a0 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.990+0000 7f0c567fc640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0c4c0a2c30 0x7f0c4c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.990+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f0c5c0ceab0 con 0x7f0c60100b00 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.990+0000 7f0c64ea5640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0c4c0a2c30 0x7f0c4c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.991+0000 7f0c64ea5640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0c4c0a2c30 0x7f0c4c0a50f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c6019b550 tx=0x7f0c5000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:10.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:09.993+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c5c09c290 con 0x7f0c60100b00 2024-09-10T10:41:10.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:10.176+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0c6006aef0 con 0x7f0c4c0a2c30 2024-09-10T10:41:10.737 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi143 2024-09-10T10:41:10.737 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi143 to 4272M 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:10.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:10 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi143 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi143 to 4272M 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:10.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:10.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:10 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi143 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi143 to 4272M 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:10 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:11.782 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:11 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:41:11.782 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:11 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:41:11.782 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:11 smithi144 ceph-mon[34230]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:11.782 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:11 smithi144 ceph-mon[34230]: osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] boot 2024-09-10T10:41:11.783 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:11 smithi144 ceph-mon[34230]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T10:41:11.783 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:11 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:11 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:41:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:11 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:41:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:11 smithi143 ceph-mon[31057]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:11 smithi143 ceph-mon[31057]: osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] boot 2024-09-10T10:41:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:11 smithi143 ceph-mon[31057]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T10:41:11.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:11 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:11 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:41:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:11 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:41:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:11 smithi178 ceph-mon[34813]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:11 smithi178 ceph-mon[34813]: osd.9 [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] boot 2024-09-10T10:41:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:11 smithi178 ceph-mon[34813]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T10:41:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:11 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:41:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:12 smithi144 ceph-mon[34230]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:12 smithi144 ceph-mon[34230]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T10:41:12.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:12 smithi143 ceph-mon[31057]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:12 smithi143 ceph-mon[31057]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T10:41:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:12 smithi178 ceph-mon[34813]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:41:12.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:12 smithi178 ceph-mon[34813]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T10:41:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:13 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/1677591542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]: dispatch 2024-09-10T10:41:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:13 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]: dispatch 2024-09-10T10:41:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:13 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]': finished 2024-09-10T10:41:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:13 smithi144 ceph-mon[34230]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T10:41:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:13 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:13.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:13 smithi144 ceph-mon[34230]: from='client.? 172.21.15.144:0/787394498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:13.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:13 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/1677591542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]: dispatch 2024-09-10T10:41:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:13 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]: dispatch 2024-09-10T10:41:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:13 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]': finished 2024-09-10T10:41:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:13 smithi143 ceph-mon[31057]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T10:41:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:13 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:13.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:13 smithi143 ceph-mon[31057]: from='client.? 172.21.15.144:0/787394498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:13 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/1677591542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]: dispatch 2024-09-10T10:41:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:13 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]: dispatch 2024-09-10T10:41:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:13 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f83408d-f6e1-4760-8b33-81493fb716f9"}]': finished 2024-09-10T10:41:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:13 smithi178 ceph-mon[34813]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T10:41:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:13 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:13.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:13 smithi178 ceph-mon[34813]: from='client.? 172.21.15.144:0/787394498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:14.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:14 smithi143 ceph-mon[31057]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:14 smithi178 ceph-mon[34813]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:14.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:14 smithi144 ceph-mon[34230]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:16.782 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:16 smithi144 ceph-mon[34230]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:16.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:16 smithi143 ceph-mon[31057]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:16 smithi178 ceph-mon[34813]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:18.629 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:18 smithi144 ceph-mon[34230]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:18.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:18 smithi143 ceph-mon[31057]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:18 smithi178 ceph-mon[34813]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:20 smithi143 ceph-mon[31057]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:20.813 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:20 smithi144 ceph-mon[34230]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:20 smithi178 ceph-mon[34813]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:21.516 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:21 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:41:21.516 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:21 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:21.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:41:21.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:21 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:21.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:21 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:41:21.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:21 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:22.584 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:22 smithi144 ceph-mon[34230]: Deploying daemon osd.10 on smithi144 2024-09-10T10:41:22.584 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:22 smithi144 ceph-mon[34230]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:22.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:22 smithi143 ceph-mon[31057]: Deploying daemon osd.10 on smithi144 2024-09-10T10:41:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:22 smithi143 ceph-mon[31057]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:22 smithi178 ceph-mon[34813]: Deploying daemon osd.10 on smithi144 2024-09-10T10:41:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:22 smithi178 ceph-mon[34813]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:24.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:24 smithi144 ceph-mon[34230]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:24.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:24 smithi143 ceph-mon[31057]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:24 smithi178 ceph-mon[34813]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:26.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:26 smithi144 ceph-mon[34230]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:26 smithi143 ceph-mon[31057]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:26 smithi178 ceph-mon[34813]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:27.681 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:27.681 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:27.681 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:27.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:27.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:27.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:28.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:28 smithi143 ceph-mon[31057]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:28.862 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:28 smithi144 ceph-mon[34230]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:28 smithi178 ceph-mon[34813]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:30.564 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:30 smithi144 ceph-mon[34230]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:30.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:30.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:30.565 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:30 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.600+0000 7f0c567fc640 1 -- 172.21.15.144:0/934732645 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0c6006aef0 con 0x7f0c4c0a2c30 2024-09-10T10:41:30.601 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 10 on host 'smithi144' 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0c4c0a2c30 msgr2=0x7f0c4c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0c4c0a2c30 0x7f0c4c0a50f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c6019b550 tx=0x7f0c5000a380 comp rx=0 tx=0).stop 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 msgr2=0x7f0c6019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 0x7f0c6019a030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00e9c0 tx=0x7f0c5c00ee80 comp rx=0 tx=0).stop 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 shutdown_connections 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0c4c0a2c30 0x7f0c4c0a50f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c601026c0 0x7f0c601a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c60101d00 0x7f0c6019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 --2- 172.21.15.144:0/934732645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c60100b00 0x7f0c6019a030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:30.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 >> 172.21.15.144:0/934732645 conn(0x7f0c600fc270 msgr2=0x7f0c6006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:30.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 shutdown_connections 2024-09-10T10:41:30.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T10:41:30.602+0000 7f0c6790a640 1 -- 172.21.15.144:0/934732645 wait complete. 2024-09-10T10:41:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:30 smithi143 ceph-mon[31057]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:30 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:30 smithi178 ceph-mon[34813]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:30 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:31.297 DEBUG:teuthology.orchestra.run.smithi144:osd.10> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T10:41:31.299 INFO:tasks.cephadm:Deploying osd.11 on smithi178 with /dev/vg_nvme/lv_1... 2024-09-10T10:41:31.299 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:41:31.555 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:41:31.584 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:31.584 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:31.584 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:31 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:31.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:31.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:31.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:31 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:31.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:31.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:31.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:31 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:32.755 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:32 smithi144 ceph-mon[34230]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:32.799 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:32 smithi178 ceph-mon[34813]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:32.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:32 smithi143 ceph-mon[31057]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:33.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:41:33 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:41:33.017+0000 7fdcae84a640 -1 osd.10 0 log_to_monitors true 2024-09-10T10:41:33.707 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:33 smithi178 ceph-mon[34813]: from='osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:41:33.707 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:33 smithi178 ceph-mon[34813]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:41:33.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:33 smithi143 ceph-mon[31057]: from='osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:41:33.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:33 smithi143 ceph-mon[31057]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:41:33.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:33 smithi144 ceph-mon[34230]: from='osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:41:33.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:33 smithi144 ceph-mon[34230]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:41:34.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T10:41:34.526 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-09-10T10:41:34.781 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:41:34.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:34.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:34.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.871 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:34.871 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:41:34.871 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:34.872 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:34.873 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.936 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:41:34.538+0000 7fdcaa7cd640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T10:41:34.936 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:41:34.547+0000 7fdca65f5640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:41:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:41:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 -- 172.21.15.178:0/871064070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 msgr2=0x7f174c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 --2- 172.21.15.178:0/871064070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c109d50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1744009ca0 tx=0x7f174402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 -- 172.21.15.178:0/871064070 shutdown_connections 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 --2- 172.21.15.178:0/871064070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c109d50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 --2- 172.21.15.178:0/871064070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f174c105030 0x7f174c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 --2- 172.21.15.178:0/871064070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174c102700 0x7f174c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 -- 172.21.15.178:0/871064070 >> 172.21.15.178:0/871064070 conn(0x7f174c0fc280 msgr2=0x7f174c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 -- 172.21.15.178:0/871064070 shutdown_connections 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 -- 172.21.15.178:0/871064070 wait complete. 2024-09-10T10:41:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.266+0000 7f1753ee2640 1 Processor -- start 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 -- start start 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174c102700 0x7f174c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f174c105030 0x7f174c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10d790 con 0x7f174c105030 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10d610 con 0x7f174c102700 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1753ee2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10d910 con 0x7f174c107960 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c1a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41386/0 (socket says 172.21.15.178:41386) 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 -- 172.21.15.178:0/1045481734 learned_addr learned my addr 172.21.15.178:0/1045481734 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f1751c7e640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174c102700 0x7f174c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175147d640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f174c105030 0x7f174c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 -- 172.21.15.178:0/1045481734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174c102700 msgr2=0x7f174c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174c102700 0x7f174c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 -- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f174c105030 msgr2=0x7f174c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f174c105030 0x7f174c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 -- 172.21.15.178:0/1045481734 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c1a1c50 con 0x7f174c107960 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.267+0000 7f175247f640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c1a15e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1744009dc0 tx=0x7f1744003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.268+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744004040 con 0x7f174c107960 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.268+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1744004690 con 0x7f174c107960 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.268+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c1a1ee0 con 0x7f174c107960 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.268+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174402f4c0 con 0x7f174c107960 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.268+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c1a2390 con 0x7f174c107960 2024-09-10T10:41:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.269+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c109450 con 0x7f174c107960 2024-09-10T10:41:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.270+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1744030650 con 0x7f174c107960 2024-09-10T10:41:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.270+0000 7f1742ffd640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17380a2bb0 0x7f17380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.271+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7f17440f0370 con 0x7f174c107960 2024-09-10T10:41:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.271+0000 7f1751c7e640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17380a2bb0 0x7f17380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.271+0000 7f1751c7e640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17380a2bb0 0x7f17380a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f173c009890 tx=0x7f173c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:35.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.278+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17440bdad0 con 0x7f174c107960 2024-09-10T10:41:35.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:35.458+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f174c062630 con 0x7f17380a2bb0 2024-09-10T10:41:35.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi144 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi144 to 4272M 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:35.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi144 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi144 to 4272M 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:35.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi144 2024-09-10T10:41:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi144 to 4272M 2024-09-10T10:41:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T10:41:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T10:41:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:35.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:35.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:35.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] boot 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T10:41:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:36 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:36.900 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:41:36.900 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:41:36.900 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:36.901 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:36.901 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:36.901 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] boot 2024-09-10T10:41:36.901 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T10:41:36.901 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:36 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: osd.10 [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] boot 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T10:41:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:36 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:41:37.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:37 smithi143 ceph-mon[31057]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T10:41:37.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:37 smithi178 ceph-mon[34813]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T10:41:37.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:37 smithi144 ceph-mon[34230]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/3076711955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]: dispatch 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]: dispatch 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]': finished 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:41:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:38 smithi143 ceph-mon[31057]: from='client.? 172.21.15.178:0/63326464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T10:41:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T10:41:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/3076711955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]: dispatch 2024-09-10T10:41:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]: dispatch 2024-09-10T10:41:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]': finished 2024-09-10T10:41:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:41:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:41:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:38 smithi144 ceph-mon[34230]: from='client.? 172.21.15.178:0/63326464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/3076711955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]: dispatch 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]: dispatch 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38c6f977-35b1-4373-95af-491a20581f8f"}]': finished 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:41:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:38 smithi178 ceph-mon[34813]: from='client.? 172.21.15.178:0/63326464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:39.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:39 smithi143 ceph-mon[31057]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:39 smithi178 ceph-mon[34813]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:39 smithi144 ceph-mon[34230]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:42.711 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:42 smithi178 ceph-mon[34813]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-10T10:41:42.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:42 smithi143 ceph-mon[31057]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-10T10:41:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:42 smithi144 ceph-mon[34230]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-10T10:41:44.545 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:44 smithi178 ceph-mon[34813]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:41:44.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:44 smithi143 ceph-mon[31057]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:41:44.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:44 smithi144 ceph-mon[34230]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:41:46.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:46 smithi178 ceph-mon[34813]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T10:41:46.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:46 smithi143 ceph-mon[31057]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T10:41:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:46 smithi144 ceph-mon[34230]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T10:41:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:41:47.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:47 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:47.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:41:47.858 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:47 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:47.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:41:47.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:47 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:48.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:48 smithi178 ceph-mon[34813]: Deploying daemon osd.11 on smithi178 2024-09-10T10:41:48.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:48 smithi178 ceph-mon[34813]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T10:41:48.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:48 smithi143 ceph-mon[31057]: Deploying daemon osd.11 on smithi178 2024-09-10T10:41:48.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:48 smithi143 ceph-mon[31057]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T10:41:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:48 smithi144 ceph-mon[34230]: Deploying daemon osd.11 on smithi178 2024-09-10T10:41:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:48 smithi144 ceph-mon[34230]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T10:41:50.573 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:50 smithi178 ceph-mon[34813]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:41:50.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:50 smithi143 ceph-mon[31057]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:41:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:50 smithi144 ceph-mon[34230]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:41:52.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:52 smithi143 ceph-mon[31057]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:52 smithi178 ceph-mon[34813]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:52 smithi144 ceph-mon[34230]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:53.752 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:53.752 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:53.752 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:53.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:54.645 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:54 smithi178 ceph-mon[34813]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:54.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:54 smithi143 ceph-mon[31057]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:54.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:54 smithi144 ceph-mon[34230]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:56.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.499+0000 7f1742ffd640 1 -- 172.21.15.178:0/1045481734 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f174c062630 con 0x7f17380a2bb0 2024-09-10T10:41:56.500 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 11 on host 'smithi178' 2024-09-10T10:41:56.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17380a2bb0 msgr2=0x7f17380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:56.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17380a2bb0 0x7f17380a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f173c009890 tx=0x7f173c008040 comp rx=0 tx=0).stop 2024-09-10T10:41:56.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 msgr2=0x7f174c1a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c1a15e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1744009dc0 tx=0x7f1744003ba0 comp rx=0 tx=0).stop 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 shutdown_connections 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f17380a2bb0 0x7f17380a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f174c107960 0x7f174c1a15e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f174c105030 0x7f174c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 --2- 172.21.15.178:0/1045481734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174c102700 0x7f174c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 >> 172.21.15.178:0/1045481734 conn(0x7f174c0fc280 msgr2=0x7f174c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 shutdown_connections 2024-09-10T10:41:56.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:41:56.501+0000 7f1753ee2640 1 -- 172.21.15.178:0/1045481734 wait complete. 2024-09-10T10:41:56.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:56 smithi143 ceph-mon[31057]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:56.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:56 smithi178 ceph-mon[34813]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:56 smithi144 ceph-mon[34230]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:57.188 DEBUG:teuthology.orchestra.run.smithi178:osd.11> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T10:41:57.191 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T10:41:57.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:41:57.466 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:41:57.751 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.751 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.751 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.751 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.808 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:41:57.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.960+0000 7f18b5821640 1 -- 172.21.15.143:0/992227360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b0101d00 msgr2=0x7f18b0102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.960+0000 7f18b5821640 1 --2- 172.21.15.143:0/992227360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b0101d00 0x7f18b0102180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1898009900 tx=0x7f189802a410 comp rx=0 tx=0).stop 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 -- 172.21.15.143:0/992227360 shutdown_connections 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 --2- 172.21.15.143:0/992227360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b01026c0 0x7f18b0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 --2- 172.21.15.143:0/992227360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b0101d00 0x7f18b0102180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 --2- 172.21.15.143:0/992227360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0100b00 0x7f18b0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 -- 172.21.15.143:0/992227360 >> 172.21.15.143:0/992227360 conn(0x7f18b00fc270 msgr2=0x7f18b00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 -- 172.21.15.143:0/992227360 shutdown_connections 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 -- 172.21.15.143:0/992227360 wait complete. 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 Processor -- start 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 -- start start 2024-09-10T10:41:57.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.961+0000 7f18b5821640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b0100b00 0x7f18b006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0101d00 0x7f18b019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 0x7f18b019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f18b010c6b0 con 0x7f18b01026c0 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18b010c530 con 0x7f18b0101d00 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f18b010c830 con 0x7f18b0100b00 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 0x7f18b019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b0100b00 0x7f18b006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 0x7f18b019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57468/0 (socket says 172.21.15.143:57468) 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b0100b00 0x7f18b006b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:51606/0 (socket says 172.21.15.143:51606) 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 -- 172.21.15.143:0/2765791233 learned_addr learned my addr 172.21.15.143:0/2765791233 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:41:57.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18a67fc640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0101d00 0x7f18b019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 -- 172.21.15.143:0/2765791233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b0100b00 msgr2=0x7f18b006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b0100b00 0x7f18b006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 -- 172.21.15.143:0/2765791233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0101d00 msgr2=0x7f18b019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0101d00 0x7f18b019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 -- 172.21.15.143:0/2765791233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019fbc0 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18af7fe640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 0x7f18b019f2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009fd0 tx=0x7f18a800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8017070 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18a8003e20 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8010460 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019fe50 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.962+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a0310 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.963+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b0108090 con 0x7f18b01026c0 2024-09-10T10:41:57.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.965+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f18a8003970 con 0x7f18b01026c0 2024-09-10T10:41:57.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.965+0000 7f18b481f640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f189c0a2b80 0x7f189c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:57.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.966+0000 7f18aeffd640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f189c0a2b80 0x7f189c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:57.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.966+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7f18a80ce7c0 con 0x7f18b01026c0 2024-09-10T10:41:57.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.966+0000 7f18aeffd640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f189c0a2b80 0x7f189c0a5040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18b0101b60 tx=0x7f18a0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:57.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:57.969+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18a809bd00 con 0x7f18b01026c0 2024-09-10T10:41:58.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.137+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f18b0062e20 con 0x7f18b01026c0 2024-09-10T10:41:58.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.138+0000 7f18b481f640 1 -- 172.21.15.143:0/2765791233 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f18a809b600 con 0x7f18b01026c0 2024-09-10T10:41:58.139 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.139+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f189c0a2b80 msgr2=0x7f189c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.139+0000 7f18b5821640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f189c0a2b80 0x7f189c0a5040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18b0101b60 tx=0x7f18a0009110 comp rx=0 tx=0).stop 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 msgr2=0x7f18b019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 0x7f18b019f2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009fd0 tx=0x7f18a800def0 comp rx=0 tx=0).stop 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 shutdown_connections 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f189c0a2b80 0x7f189c0a5040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18b01026c0 0x7f18b019f2b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:58.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b0101d00 0x7f18b019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:58.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 --2- 172.21.15.143:0/2765791233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f18b0100b00 0x7f18b006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:58.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 >> 172.21.15.143:0/2765791233 conn(0x7f18b00fc270 msgr2=0x7f18b0105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:58.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 shutdown_connections 2024-09-10T10:41:58.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:41:58.140+0000 7f18b5821640 1 -- 172.21.15.143:0/2765791233 wait complete. 2024-09-10T10:41:58.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:58 smithi143 ceph-mon[31057]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:58.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:58 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2765791233' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:41:58.821 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:58 smithi178 ceph-mon[34813]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:58.821 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:58 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2765791233' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:41:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:58 smithi144 ceph-mon[34230]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:41:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:58 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2765791233' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:41:58.998 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1725964895,"num_in_osds":12,"osd_in_since":1725964897,"num_remapped_pgs":0} 2024-09-10T10:41:59.094 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:41:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:41:58.818+0000 7f78f3e15640 -1 osd.11 0 log_to_monitors true 2024-09-10T10:41:59.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:59 smithi143 ceph-mon[31057]: from='osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:41:59.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:41:59 smithi143 ceph-mon[31057]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:41:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:59 smithi178 ceph-mon[34813]: from='osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:41:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:41:59 smithi178 ceph-mon[34813]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:41:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:59 smithi144 ceph-mon[34230]: from='osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:41:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:41:59 smithi144 ceph-mon[34230]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:41:59.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:42:00.245 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:00.632 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:00.634 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.674 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:42:00.507+0000 7f78efd98640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T10:42:00.675 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:42:00.519+0000 7f78ebbc0640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:00.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 -- 172.21.15.143:0/3372939075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40100b10 msgr2=0x7fbf40100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3372939075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40100b10 0x7fbf40100f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009900 tx=0x7fbf3402a410 comp rx=0 tx=0).stop 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 -- 172.21.15.143:0/3372939075 shutdown_connections 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3372939075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf401026d0 0x7fbf40108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3372939075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf40101d10 0x7fbf40102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3372939075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40100b10 0x7fbf40100f10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 -- 172.21.15.143:0/3372939075 >> 172.21.15.143:0/3372939075 conn(0x7fbf400fc2a0 msgr2=0x7fbf400fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 -- 172.21.15.143:0/3372939075 shutdown_connections 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.711+0000 7fbf457dd640 1 -- 172.21.15.143:0/3372939075 wait complete. 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 Processor -- start 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 -- start start 2024-09-10T10:42:00.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf40100b10 0x7fbf4006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 0x7fbf4019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf401026d0 0x7fbf4019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010c720 con 0x7fbf40101d10 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010c5a0 con 0x7fbf401026d0 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf457dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010c8a0 con 0x7fbf40100b10 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf3e7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 0x7fbf4019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf3e7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 0x7fbf4019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57488/0 (socket says 172.21.15.143:57488) 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf3e7fc640 1 -- 172.21.15.143:0/3145526171 learned_addr learned my addr 172.21.15.143:0/3145526171 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.712+0000 7fbf3effd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf40100b10 0x7fbf4006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:00.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3f7fe640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf401026d0 0x7fbf4019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3e7fc640 1 -- 172.21.15.143:0/3145526171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf40100b10 msgr2=0x7fbf4006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3e7fc640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf40100b10 0x7fbf4006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3e7fc640 1 -- 172.21.15.143:0/3145526171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf401026d0 msgr2=0x7fbf4019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3e7fc640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf401026d0 0x7fbf4019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3e7fc640 1 -- 172.21.15.143:0/3145526171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4019fc40 con 0x7fbf40101d10 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3f7fe640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf401026d0 0x7fbf4019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf3e7fc640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 0x7fbf4019cd10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000b4a0 tx=0x7fbf3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:00.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30014070 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf30010ac0 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4019fed0 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30017c30 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.713+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401a0400 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.714+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40108040 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.715+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbf30017070 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.716+0000 7fbf27fff640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbf280a2b80 0x7fbf280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.716+0000 7fbf3effd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbf280a2b80 0x7fbf280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.716+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7fbf300ce150 con 0x7fbf40101d10 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.716+0000 7fbf3effd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbf280a2b80 0x7fbf280a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf34000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:00.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.720+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf3009b670 con 0x7fbf40101d10 2024-09-10T10:42:00.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.885+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbf4002d970 con 0x7fbf40101d10 2024-09-10T10:42:00.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.886+0000 7fbf27fff640 1 -- 172.21.15.143:0/3145526171 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fbf3009af70 con 0x7fbf40101d10 2024-09-10T10:42:00.887 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbf280a2b80 msgr2=0x7fbf280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbf280a2b80 0x7fbf280a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf34000bc0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 msgr2=0x7fbf4019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 0x7fbf4019cd10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000b4a0 tx=0x7fbf3000b960 comp rx=0 tx=0).stop 2024-09-10T10:42:00.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 shutdown_connections 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbf280a2b80 0x7fbf280a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf401026d0 0x7fbf4019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf40101d10 0x7fbf4019cd10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 --2- 172.21.15.143:0/3145526171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf40100b10 0x7fbf4006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 >> 172.21.15.143:0/3145526171 conn(0x7fbf400fc2a0 msgr2=0x7fbf40105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 shutdown_connections 2024-09-10T10:42:00.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:00.889+0000 7fbf457dd640 1 -- 172.21.15.143:0/3145526171 wait complete. 2024-09-10T10:42:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:42:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:42:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T10:42:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:42:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:00.938 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:01.547 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1725964895,"num_in_osds":12,"osd_in_since":1725964897,"num_remapped_pgs":0} 2024-09-10T10:42:01.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: Detected new or changed devices on smithi178 2024-09-10T10:42:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi178 to 4271M 2024-09-10T10:42:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:42:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T10:42:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:01 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3145526171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:42:01.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: Detected new or changed devices on smithi178 2024-09-10T10:42:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi178 to 4271M 2024-09-10T10:42:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:42:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T10:42:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:01 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3145526171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: Detected new or changed devices on smithi178 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi178 to 4271M 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:01 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3145526171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:42:02.548 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:42:02.799 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: purged_snaps scrub starts 2024-09-10T10:42:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: purged_snaps scrub ok 2024-09-10T10:42:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:42:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] boot 2024-09-10T10:42:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:42:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: purged_snaps scrub starts 2024-09-10T10:42:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: purged_snaps scrub ok 2024-09-10T10:42:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:42:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] boot 2024-09-10T10:42:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:42:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: purged_snaps scrub starts 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: purged_snaps scrub ok 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: osd.11 [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] boot 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:42:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:42:03.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 -- 172.21.15.143:0/3314719475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87981026d0 msgr2=0x7f8798108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 --2- 172.21.15.143:0/3314719475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87981026d0 0x7f8798108f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8790009ca0 tx=0x7f879002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 -- 172.21.15.143:0/3314719475 shutdown_connections 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 --2- 172.21.15.143:0/3314719475 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87981026d0 0x7f8798108f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 --2- 172.21.15.143:0/3314719475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8798101d10 0x7f8798102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 --2- 172.21.15.143:0/3314719475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8798100b10 0x7f8798100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 -- 172.21.15.143:0/3314719475 >> 172.21.15.143:0/3314719475 conn(0x7f87980fc2a0 msgr2=0x7f87980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 -- 172.21.15.143:0/3314719475 shutdown_connections 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.295+0000 7f879e3f9640 1 -- 172.21.15.143:0/3314719475 wait complete. 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 Processor -- start 2024-09-10T10:42:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 -- start start 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8798100b10 0x7f879810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 0x7f8798109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87981026d0 0x7f879810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f879810d770 con 0x7f8798101d10 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f879810d5f0 con 0x7f8798100b10 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879e3f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f879810d8f0 con 0x7f87981026d0 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 0x7f8798109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 0x7f8798109ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57500/0 (socket says 172.21.15.143:57500) 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 -- 172.21.15.143:0/2359781887 learned_addr learned my addr 172.21.15.143:0/2359781887 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f879c996640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87981026d0 0x7f879810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f8797fff640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8798100b10 0x7f879810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 -- 172.21.15.143:0/2359781887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87981026d0 msgr2=0x7f879810a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87981026d0 0x7f879810a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 -- 172.21.15.143:0/2359781887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8798100b10 msgr2=0x7f879810b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8798100b10 0x7f879810b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 -- 172.21.15.143:0/2359781887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879810a8e0 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.296+0000 7f878ffff640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 0x7f8798109ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f878000d770 tx=0x7f878000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.297+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780017070 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.297+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87800054d0 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.297+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780004cf0 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.297+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87981a1ed0 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.297+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87981a2330 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.298+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8798107f60 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.299+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8780004e90 con 0x7f8798101d10 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.299+0000 7f8795ffb640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87840a2c00 0x7f87840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:03.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.299+0000 7f8797fff640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87840a2c00 0x7f87840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:03.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.299+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f87800ce640 con 0x7f8798101d10 2024-09-10T10:42:03.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.300+0000 7f8797fff640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87840a2c00 0x7f87840a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87880058c0 tx=0x7f8788005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:03.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.303+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f878009ba40 con 0x7f8798101d10 2024-09-10T10:42:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.472+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8798062df0 con 0x7f8798101d10 2024-09-10T10:42:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.472+0000 7f8795ffb640 1 -- 172.21.15.143:0/2359781887 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f878009b340 con 0x7f8798101d10 2024-09-10T10:42:03.473 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:03.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87840a2c00 msgr2=0x7f87840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87840a2c00 0x7f87840a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87880058c0 tx=0x7f8788005850 comp rx=0 tx=0).stop 2024-09-10T10:42:03.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 msgr2=0x7f8798109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 0x7f8798109ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f878000d770 tx=0x7f878000dc30 comp rx=0 tx=0).stop 2024-09-10T10:42:03.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 shutdown_connections 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87840a2c00 0x7f87840a50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87981026d0 0x7f879810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8798101d10 0x7f8798109ae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 --2- 172.21.15.143:0/2359781887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8798100b10 0x7f879810b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 >> 172.21.15.143:0/2359781887 conn(0x7f87980fc2a0 msgr2=0x7f8798105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 shutdown_connections 2024-09-10T10:42:03.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:03.474+0000 7f879e3f9640 1 -- 172.21.15.143:0/2359781887 wait complete. 2024-09-10T10:42:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:03 smithi143 ceph-mon[31057]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:42:03.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2359781887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:42:03.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:03 smithi178 ceph-mon[34813]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:42:03.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2359781887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:42:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:03 smithi144 ceph-mon[34230]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:42:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2359781887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:42:04.255 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1725964921,"num_in_osds":12,"osd_in_since":1725964897,"num_remapped_pgs":0} 2024-09-10T10:42:04.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:42:04.499 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:04.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:04 smithi143 ceph-mon[31057]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:04 smithi178 ceph-mon[34813]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:04 smithi144 ceph-mon[34230]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:04.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/1402062927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04100a80 msgr2=0x7f6b04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:04.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/1402062927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04100a80 0x7f6b04100ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009930 tx=0x7f6aec02a420 comp rx=0 tx=0).stop 2024-09-10T10:42:04.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/1402062927 shutdown_connections 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/1402062927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04101420 0x7f6b0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/1402062927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04100a80 0x7f6b04100ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/1402062927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b04106a80 0x7f6b04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/1402062927 >> 172.21.15.143:0/1402062927 conn(0x7f6b040fc7a0 msgr2=0x7f6b040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/1402062927 shutdown_connections 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.969+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/1402062927 wait complete. 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 Processor -- start 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 -- start start 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b04100a80 0x7f6b0410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04101420 0x7f6b0410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 0x7f6b0410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:04.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04111e50 con 0x7f6b04106a80 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04111cd0 con 0x7f6b04101420 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b0a7b2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04111fd0 con 0x7f6b04100a80 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 0x7f6b0410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 0x7f6b0410ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57518/0 (socket says 172.21.15.143:57518) 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 -- 172.21.15.143:0/2394598553 learned_addr learned my addr 172.21.15.143:0/2394598553 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b03fff640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b04100a80 0x7f6b0410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6afbfff640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04101420 0x7f6b0410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 -- 172.21.15.143:0/2394598553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b04100a80 msgr2=0x7f6b0410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b04100a80 0x7f6b0410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 -- 172.21.15.143:0/2394598553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04101420 msgr2=0x7f6b0410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04101420 0x7f6b0410e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.970+0000 7f6b08d4f640 1 -- 172.21.15.143:0/2394598553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b041a6890 con 0x7f6b04106a80 2024-09-10T10:42:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.971+0000 7f6b08d4f640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 0x7f6b0410ee60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00eaa0 tx=0x7f6afc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.971+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00cdb0 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.971+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6afc005020 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.971+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b041a6b80 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.971+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0116d0 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.971+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b041a7040 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.972+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b04100ee0 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.973+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6afc0051d0 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.973+0000 7f6b01ffb640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6af00a2b80 0x7f6af00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.974+0000 7f6b03fff640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6af00a2b80 0x7f6af00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.974+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f6afc0cecf0 con 0x7f6b04106a80 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.974+0000 7f6b03fff640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6af00a2b80 0x7f6af00a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6af400bb60 tx=0x7f6af400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:04.978+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6afc0d3070 con 0x7f6b04106a80 2024-09-10T10:42:05.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.143+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b04108ff0 con 0x7f6b04106a80 2024-09-10T10:42:05.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.144+0000 7f6b01ffb640 1 -- 172.21.15.143:0/2394598553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18563 (secure 0 0 0) 0x7f6afc016030 con 0x7f6b04106a80 2024-09-10T10:42:05.146 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:05.146 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":76,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:42:02.518306+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6af00a2b80 msgr2=0x7f6af00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6af00a2b80 0x7f6af00a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6af400bb60 tx=0x7f6af400b390 comp rx=0 tx=0).stop 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 msgr2=0x7f6b0410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 0x7f6b0410ee60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00eaa0 tx=0x7f6afc00ef60 comp rx=0 tx=0).stop 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 shutdown_connections 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6af00a2b80 0x7f6af00a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b04106a80 0x7f6b0410ee60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04101420 0x7f6b0410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 --2- 172.21.15.143:0/2394598553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b04100a80 0x7f6b0410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 >> 172.21.15.143:0/2394598553 conn(0x7f6b040fc7a0 msgr2=0x7f6b040fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:05.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 shutdown_connections 2024-09-10T10:42:05.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:05.147+0000 7f6b0a7b2640 1 -- 172.21.15.143:0/2394598553 wait complete. 2024-09-10T10:42:05.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:05 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2394598553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:42:05.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:05 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2394598553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:42:05.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:05 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2394598553' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:42:05.975 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:38:11.637482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:42:05.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T10:42:06.217 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:06.592 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:06 smithi143 ceph-mon[31057]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 -- 172.21.15.143:0/329951891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18810c180 msgr2=0x7fb18810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 --2- 172.21.15.143:0/329951891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18810c180 0x7fb18810e600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb180009ca0 tx=0x7fb18002a950 comp rx=0 tx=0).stop 2024-09-10T10:42:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 -- 172.21.15.143:0/329951891 shutdown_connections 2024-09-10T10:42:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 --2- 172.21.15.143:0/329951891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb18810c180 0x7fb18810e600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 --2- 172.21.15.143:0/329951891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1881036c0 0x7fb18810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 --2- 172.21.15.143:0/329951891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb188102da0 0x7fb188103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 -- 172.21.15.143:0/329951891 >> 172.21.15.143:0/329951891 conn(0x7fb1880fc7a0 msgr2=0x7fb1880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 -- 172.21.15.143:0/329951891 shutdown_connections 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.672+0000 7fb18e031640 1 -- 172.21.15.143:0/329951891 wait complete. 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 Processor -- start 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 -- start start 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb188102da0 0x7fb188198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 0x7fb188198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18810c180 0x7fb18819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb188111e10 con 0x7fb1881036c0 2024-09-10T10:42:06.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb188111c90 con 0x7fb188102da0 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb18e031640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb188111f90 con 0x7fb18810c180 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 0x7fb188198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 0x7fb188198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36106/0 (socket says 172.21.15.143:36106) 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.673+0000 7fb17ffff640 1 -- 172.21.15.143:0/2685187265 learned_addr learned my addr 172.21.15.143:0/2685187265 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb187fff640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18810c180 0x7fb18819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb1877fe640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb188102da0 0x7fb188198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb17ffff640 1 -- 172.21.15.143:0/2685187265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18810c180 msgr2=0x7fb18819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb17ffff640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18810c180 0x7fb18819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb17ffff640 1 -- 172.21.15.143:0/2685187265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb188102da0 msgr2=0x7fb188198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb17ffff640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb188102da0 0x7fb188198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb17ffff640 1 -- 172.21.15.143:0/2685187265 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18819d2c0 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb1877fe640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb188102da0 0x7fb188198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb17ffff640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 0x7fb188198850 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c890 tx=0x7fb17000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170004220 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18819d5b0 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1700043c0 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170004a00 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.674+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1881a4e90 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.675+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb188103180 con 0x7fb1881036c0 2024-09-10T10:42:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.676+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb170004ba0 con 0x7fb1881036c0 2024-09-10T10:42:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.677+0000 7fb1857fa640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb1740a2b80 0x7fb1740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.677+0000 7fb1877fe640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb1740a2b80 0x7fb1740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.677+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb1700ce840 con 0x7fb1881036c0 2024-09-10T10:42:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.678+0000 7fb1877fe640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb1740a2b80 0x7fb1740a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb1780058c0 tx=0x7fb178005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:06.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.681+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb170014030 con 0x7fb1881036c0 2024-09-10T10:42:06.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.847+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb18802d7c0 con 0x7fb1881036c0 2024-09-10T10:42:06.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.847+0000 7fb1857fa640 1 -- 172.21.15.143:0/2685187265 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fb17009bc40 con 0x7fb1881036c0 2024-09-10T10:42:06.848 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-09-10T10:42:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb1740a2b80 msgr2=0x7fb1740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb1740a2b80 0x7fb1740a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb1780058c0 tx=0x7fb178005850 comp rx=0 tx=0).stop 2024-09-10T10:42:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 msgr2=0x7fb188198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 0x7fb188198850 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c890 tx=0x7fb17000cd50 comp rx=0 tx=0).stop 2024-09-10T10:42:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 shutdown_connections 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb1740a2b80 0x7fb1740a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18810c180 0x7fb18819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1881036c0 0x7fb188198850 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 --2- 172.21.15.143:0/2685187265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb188102da0 0x7fb188198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 >> 172.21.15.143:0/2685187265 conn(0x7fb1880fc7a0 msgr2=0x7fb1880fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 shutdown_connections 2024-09-10T10:42:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:06.849+0000 7fb18e031640 1 -- 172.21.15.143:0/2685187265 wait complete. 2024-09-10T10:42:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:06 smithi178 ceph-mon[34813]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:06 smithi144 ceph-mon[34230]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.a on smithi143 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.d on smithi143 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.g on smithi143 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.j on smithi143 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.b on smithi144 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.e on smithi144 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.h on smithi144 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.k on smithi144 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.c on smithi178 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.f on smithi178 2024-09-10T10:42:07.645 INFO:tasks.cephadm:Adding mds.i on smithi178 2024-09-10T10:42:07.646 INFO:tasks.cephadm:Adding mds.l on smithi178 2024-09-10T10:42:07.646 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l' 2024-09-10T10:42:07.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:07 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2685187265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:42:07.905 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:42:07.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:07 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2685187265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:42:07.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:07 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2685187265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 -- 172.21.15.178:0/589926100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4102690 msgr2=0x7fd0d4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 --2- 172.21.15.178:0/589926100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4102690 0x7fd0d4108f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009ca0 tx=0x7fd0cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 -- 172.21.15.178:0/589926100 shutdown_connections 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 --2- 172.21.15.178:0/589926100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4102690 0x7fd0d4108f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 --2- 172.21.15.178:0/589926100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4101cd0 0x7fd0d4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 --2- 172.21.15.178:0/589926100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d4100ad0 0x7fd0d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 -- 172.21.15.178:0/589926100 >> 172.21.15.178:0/589926100 conn(0x7fd0d40fc280 msgr2=0x7fd0d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 -- 172.21.15.178:0/589926100 shutdown_connections 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.362+0000 7fd0da911640 1 -- 172.21.15.178:0/589926100 wait complete. 2024-09-10T10:42:08.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 Processor -- start 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 -- start start 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d4100ad0 0x7fd0d406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 0x7fd0d419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4102690 0x7fd0d419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410c5e0 con 0x7fd0d4102690 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410c460 con 0x7fd0d4100ad0 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0da911640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410c760 con 0x7fd0d4101cd0 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 0x7fd0d419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 0x7fd0d419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52912/0 (socket says 172.21.15.178:52912) 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0d37fe640 1 -- 172.21.15.178:0/3328816222 learned_addr learned my addr 172.21.15.178:0/3328816222 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0d8eae640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4102690 0x7fd0d419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.363+0000 7fd0d3fff640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d4100ad0 0x7fd0d406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d37fe640 1 -- 172.21.15.178:0/3328816222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d4100ad0 msgr2=0x7fd0d406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:08.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d37fe640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d4100ad0 0x7fd0d406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d37fe640 1 -- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4102690 msgr2=0x7fd0d419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d37fe640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4102690 0x7fd0d419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d37fe640 1 -- 172.21.15.178:0/3328816222 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d419f9b0 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d8eae640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4102690 0x7fd0d419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d37fe640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 0x7fd0d419ca80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800d950 tx=0x7fd0c800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8017070 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d419fc40 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0c8009c30 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800f620 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.364+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d41a0180 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.365+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d4107e70 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.366+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd0c80109a0 con 0x7fd0d4101cd0 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.367+0000 7fd0d17fa640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd0bc0a2b80 0x7fd0bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:08.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.367+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd0c80ce820 con 0x7fd0d4101cd0 2024-09-10T10:42:08.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.367+0000 7fd0d3fff640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd0bc0a2b80 0x7fd0bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:08.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.368+0000 7fd0d3fff640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd0bc0a2b80 0x7fd0bc0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40058c0 tx=0x7fd0c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:08.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.370+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0c8009de0 con 0x7fd0d4101cd0 2024-09-10T10:42:08.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.550+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}) -- 0x7fd0d40627f0 con 0x7fd0bc0a2b80 2024-09-10T10:42:08.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.557+0000 7fd0d17fa640 1 -- 172.21.15.178:0/3328816222 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd0d40627f0 con 0x7fd0bc0a2b80 2024-09-10T10:42:08.558 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mds.all update... 2024-09-10T10:42:08.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd0bc0a2b80 msgr2=0x7fd0bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd0bc0a2b80 0x7fd0bc0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40058c0 tx=0x7fd0c4005850 comp rx=0 tx=0).stop 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 msgr2=0x7fd0d419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 0x7fd0d419ca80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800d950 tx=0x7fd0c800de10 comp rx=0 tx=0).stop 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 shutdown_connections 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd0bc0a2b80 0x7fd0bc0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d4102690 0x7fd0d419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0d4101cd0 0x7fd0d419ca80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 --2- 172.21.15.178:0/3328816222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d4100ad0 0x7fd0d406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 >> 172.21.15.178:0/3328816222 conn(0x7fd0d40fc280 msgr2=0x7fd0d4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.559+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 shutdown_connections 2024-09-10T10:42:08.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T10:42:08.560+0000 7fd0da911640 1 -- 172.21.15.178:0/3328816222 wait complete. 2024-09-10T10:42:08.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:08 smithi143 ceph-mon[31057]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:08.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:08 smithi144 ceph-mon[34230]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:08 smithi178 ceph-mon[34813]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:09.296 DEBUG:teuthology.orchestra.run.smithi143:mds.a> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T10:42:09.299 DEBUG:teuthology.orchestra.run.smithi143:mds.d> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T10:42:09.342 DEBUG:teuthology.orchestra.run.smithi143:mds.g> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T10:42:09.385 DEBUG:teuthology.orchestra.run.smithi143:mds.j> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T10:42:09.429 DEBUG:teuthology.orchestra.run.smithi144:mds.b> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T10:42:09.433 DEBUG:teuthology.orchestra.run.smithi144:mds.e> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T10:42:09.476 DEBUG:teuthology.orchestra.run.smithi144:mds.h> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T10:42:09.520 DEBUG:teuthology.orchestra.run.smithi144:mds.k> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T10:42:09.564 DEBUG:teuthology.orchestra.run.smithi178:mds.c> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T10:42:09.566 DEBUG:teuthology.orchestra.run.smithi178:mds.f> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T10:42:09.609 DEBUG:teuthology.orchestra.run.smithi178:mds.i> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T10:42:09.652 DEBUG:teuthology.orchestra.run.smithi178:mds.l> sudo journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T10:42:09.695 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T10:42:09.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:09.713 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:09.713 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: Saving service mds.all spec with placement smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-10T10:42:09.713 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: Deploying daemon mds.l on smithi178 2024-09-10T10:42:09.714 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:09 smithi178 ceph-mon[34813]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:09.718 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:09.718 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: Saving service mds.all spec with placement smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-10T10:42:09.718 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: Deploying daemon mds.l on smithi178 2024-09-10T10:42:09.719 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:09 smithi143 ceph-mon[31057]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: Saving service mds.all spec with placement smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:09.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:09.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:09.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: Deploying daemon mds.l on smithi178 2024-09-10T10:42:09.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:09 smithi144 ceph-mon[34230]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:09.941 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:10.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 -- 172.21.15.143:0/3638325361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c102790 msgr2=0x7f635c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 --2- 172.21.15.143:0/3638325361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c102790 0x7f635c1067e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f634c009ca0 tx=0x7f634c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 -- 172.21.15.143:0/3638325361 shutdown_connections 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 --2- 172.21.15.143:0/3638325361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f635c106f10 0x7f635c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 --2- 172.21.15.143:0/3638325361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c102790 0x7f635c1067e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 --2- 172.21.15.143:0/3638325361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c101de0 0x7f635c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 -- 172.21.15.143:0/3638325361 >> 172.21.15.143:0/3638325361 conn(0x7f635c0fd8c0 msgr2=0x7f635c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 -- 172.21.15.143:0/3638325361 shutdown_connections 2024-09-10T10:42:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 -- 172.21.15.143:0/3638325361 wait complete. 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.397+0000 7f6360807640 1 Processor -- start 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 -- start start 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 0x7f635c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c102790 0x7f635c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f635c106f10 0x7f635c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f635c111e40 con 0x7f635c101de0 2024-09-10T10:42:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f635c111cc0 con 0x7f635c102790 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6360807640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f635c111fc0 con 0x7f635c106f10 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 0x7f635c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 0x7f635c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36126/0 (socket says 172.21.15.143:36126) 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 -- 172.21.15.143:0/772081205 learned_addr learned my addr 172.21.15.143:0/772081205 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635ad9d640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f635c106f10 0x7f635c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6359d9b640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c102790 0x7f635c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 -- 172.21.15.143:0/772081205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f635c106f10 msgr2=0x7f635c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f635c106f10 0x7f635c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 -- 172.21.15.143:0/772081205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c102790 msgr2=0x7f635c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c102790 0x7f635c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 -- 172.21.15.143:0/772081205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c19d2d0 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f6359d9b640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c102790 0x7f635c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f635a59c640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 0x7f635c198320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f635000c920 tx=0x7f635000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.398+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350007c70 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.399+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c19d5c0 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.399+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6350007e10 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.399+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350018820 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.399+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c1a4ea0 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.399+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c1021c0 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.401+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f63500189c0 con 0x7f635c101de0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.401+0000 7f634b7fe640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f63440a2b80 0x7f63440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.401+0000 7f6359d9b640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f63440a2b80 0x7f63440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.401+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f63500cecd0 con 0x7f635c101de0 2024-09-10T10:42:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.402+0000 7f6359d9b640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f63440a2b80 0x7f63440a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f635c0fdce0 tx=0x7f634c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.405+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6350014030 con 0x7f635c101de0 2024-09-10T10:42:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.606+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f635c109b60 con 0x7f635c101de0 2024-09-10T10:42:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.606+0000 7f634b7fe640 1 -- 172.21.15.143:0/772081205 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f635009c0d0 con 0x7f635c101de0 2024-09-10T10:42:10.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:10.608 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":1,"btime":"2024-09-10T10:35:07:959271+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T10:42:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 1 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f63440a2b80 msgr2=0x7f63440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f63440a2b80 0x7f63440a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f635c0fdce0 tx=0x7f634c0093f0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 msgr2=0x7f635c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 0x7f635c198320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f635000c920 tx=0x7f635000cde0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 shutdown_connections 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f63440a2b80 0x7f63440a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f635c106f10 0x7f635c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f635c102790 0x7f635c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 --2- 172.21.15.143:0/772081205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f635c101de0 0x7f635c198320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.608+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 >> 172.21.15.143:0/772081205 conn(0x7f635c0fd8c0 msgr2=0x7f635c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.609+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 shutdown_connections 2024-09-10T10:42:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:10.609+0000 7f6360807640 1 -- 172.21.15.143:0/772081205 wait complete. 2024-09-10T10:42:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:10 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/772081205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:10.925 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:10 smithi178 systemd[1]: Starting Ceph mds.l for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:10.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:10 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/772081205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:11.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:10 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/772081205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:11.356 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 podman[64365]: 2024-09-10 10:42:10.999614574 +0000 UTC m=+0.015452934 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:11.356 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 podman[64365]: 2024-09-10 10:42:11.117798443 +0000 UTC m=+0.133636803 container create 8978b9fc4ab496c79cc59133bc25e16a894a4dd973925b2da3fd5f626fd06cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T10:42:11.357 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 podman[64365]: 2024-09-10 10:42:11.329065916 +0000 UTC m=+0.344904277 container init 8978b9fc4ab496c79cc59133bc25e16a894a4dd973925b2da3fd5f626fd06cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T10:42:11.357 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 podman[64365]: 2024-09-10 10:42:11.3320506 +0000 UTC m=+0.347888968 container start 8978b9fc4ab496c79cc59133bc25e16a894a4dd973925b2da3fd5f626fd06cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T10:42:11.357 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 ceph-mds[64384]: -- 172.21.15.178:0/1462257253 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598317ed860 con 0x5598317cb000 2024-09-10T10:42:11.674 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l[64380]: starting mds.l at 2024-09-10T10:42:11.675 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 bash[64365]: 8978b9fc4ab496c79cc59133bc25e16a894a4dd973925b2da3fd5f626fd06cc0 2024-09-10T10:42:11.675 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:42:11 smithi178 systemd[1]: Started Ceph mds.l for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:12.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6832/1016684638,v1:172.21.15.178:6833/1016684638] up:boot 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: fsmap 1 up:standby 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:12.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:12 smithi178 ceph-mon[34813]: Deploying daemon mds.k on smithi144 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6832/1016684638,v1:172.21.15.178:6833/1016684638] up:boot 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: fsmap 1 up:standby 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:12.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:12.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:12.687 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:12 smithi144 ceph-mon[34230]: Deploying daemon mds.k on smithi144 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6832/1016684638,v1:172.21.15.178:6833/1016684638] up:boot 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: fsmap 1 up:standby 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:12.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:12 smithi143 ceph-mon[31057]: Deploying daemon mds.k on smithi144 2024-09-10T10:42:13.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:13.668 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:13.686 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:13 smithi144 systemd[1]: Starting Ceph mds.k for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:14.034 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:13 smithi144 podman[63206]: 2024-09-10 10:42:13.676074853 +0000 UTC m=+0.016545699 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:14.034 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:13 smithi144 podman[63206]: 2024-09-10 10:42:13.778740412 +0000 UTC m=+0.119211253 container create 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T10:42:14.034 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:14 smithi144 podman[63206]: 2024-09-10 10:42:14.006735998 +0000 UTC m=+0.347206837 container init 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:42:14.034 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:14 smithi144 podman[63206]: 2024-09-10 10:42:14.009852297 +0000 UTC m=+0.350323138 container start 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T10:42:14.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 -- 172.21.15.143:0/3835361610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac10c170 msgr2=0x7f11ac10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 --2- 172.21.15.143:0/3835361610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac10c170 0x7f11ac10e610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009ca0 tx=0x7f11a402a950 comp rx=0 tx=0).stop 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 -- 172.21.15.143:0/3835361610 shutdown_connections 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 --2- 172.21.15.143:0/3835361610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac10c170 0x7f11ac10e610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 --2- 172.21.15.143:0/3835361610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103660 0x7f11ac10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 --2- 172.21.15.143:0/3835361610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11ac102d40 0x7f11ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 -- 172.21.15.143:0/3835361610 >> 172.21.15.143:0/3835361610 conn(0x7f11ac0fc7a0 msgr2=0x7f11ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.128+0000 7f11b44a0640 1 -- 172.21.15.143:0/3835361610 shutdown_connections 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 -- 172.21.15.143:0/3835361610 wait complete. 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 Processor -- start 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 -- start start 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 0x7f11ac1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103660 0x7f11ac100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11ac10c170 0x7f11ac101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac111e20 con 0x7f11ac102d40 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac111ca0 con 0x7f11ac103660 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b44a0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac111fa0 con 0x7f11ac10c170 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b223c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 0x7f11ac1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b223c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 0x7f11ac1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36148/0 (socket says 172.21.15.143:36148) 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b223c640 1 -- 172.21.15.143:0/4228022108 learned_addr learned my addr 172.21.15.143:0/4228022108 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.129+0000 7f11b1a3b640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103660 0x7f11ac100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b2a3d640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11ac10c170 0x7f11ac101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b223c640 1 -- 172.21.15.143:0/4228022108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11ac10c170 msgr2=0x7f11ac101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b223c640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11ac10c170 0x7f11ac101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b223c640 1 -- 172.21.15.143:0/4228022108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103660 msgr2=0x7f11ac100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:14.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b223c640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103660 0x7f11ac100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b223c640 1 -- 172.21.15.143:0/4228022108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac1a6800 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b223c640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 0x7f11ac1009a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11a800d770 tx=0x7f11a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8017070 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac1a6af0 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11a80054d0 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8004cf0 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.130+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac1a7030 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.131+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac103120 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.132+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f11a8004e90 con 0x7f11ac102d40 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.133+0000 7f11a37fe640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11980a2b80 0x7f11980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.133+0000 7f11b1a3b640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11980a2b80 0x7f11980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:14.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.133+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f11a80ce560 con 0x7f11ac102d40 2024-09-10T10:42:14.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.133+0000 7f11b1a3b640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11980a2b80 0x7f11980a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11ac102430 tx=0x7f119c00b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:14.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.137+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11a809b310 con 0x7f11ac102d40 2024-09-10T10:42:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.349+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11ac107240 con 0x7f11ac102d40 2024-09-10T10:42:14.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.350+0000 7f11a37fe640 1 -- 172.21.15.143:0/4228022108 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f11a809f610 con 0x7f11ac102d40 2024-09-10T10:42:14.351 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 2 2024-09-10T10:42:14.351 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"btime":"2024-09-10T10:42:11:360404+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:42:14.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11980a2b80 msgr2=0x7f11980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11980a2b80 0x7f11980a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11ac102430 tx=0x7f119c00b360 comp rx=0 tx=0).stop 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 msgr2=0x7f11ac1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 0x7f11ac1009a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11a800d770 tx=0x7f11a800dc30 comp rx=0 tx=0).stop 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 shutdown_connections 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11980a2b80 0x7f11980a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11ac10c170 0x7f11ac101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ac103660 0x7f11ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 --2- 172.21.15.143:0/4228022108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ac102d40 0x7f11ac1009a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 >> 172.21.15.143:0/4228022108 conn(0x7f11ac0fc7a0 msgr2=0x7f11ac0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 shutdown_connections 2024-09-10T10:42:14.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:14.352+0000 7f11b44a0640 1 -- 172.21.15.143:0/4228022108 wait complete. 2024-09-10T10:42:14.436 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mds[63226]: -- 172.21.15.144:0/2334690908 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56364f869860 con 0x56364f847400 2024-09-10T10:42:14.436 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k[63222]: starting mds.k at 2024-09-10T10:42:14.436 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:14 smithi144 bash[63206]: 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 2024-09-10T10:42:14.436 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:42:14 smithi144 systemd[1]: Started Ceph mds.k for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:14.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:14 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4228022108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:14 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4228022108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:14 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4228022108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:15.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:15 smithi143 ceph-mon[31057]: Deploying daemon mds.j on smithi143 2024-09-10T10:42:15.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:15 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6832/3121359191,v1:172.21.15.144:6833/3121359191] up:boot 2024-09-10T10:42:15.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:15 smithi143 ceph-mon[31057]: fsmap 2 up:standby 2024-09-10T10:42:15.669 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:15 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:42:15.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:15 smithi178 ceph-mon[34813]: Deploying daemon mds.j on smithi143 2024-09-10T10:42:15.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:15 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6832/3121359191,v1:172.21.15.144:6833/3121359191] up:boot 2024-09-10T10:42:15.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:15 smithi178 ceph-mon[34813]: fsmap 2 up:standby 2024-09-10T10:42:15.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:15 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:42:15.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:15 smithi144 ceph-mon[34230]: Deploying daemon mds.j on smithi143 2024-09-10T10:42:15.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:15 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6832/3121359191,v1:172.21.15.144:6833/3121359191] up:boot 2024-09-10T10:42:15.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:15 smithi144 ceph-mon[34230]: fsmap 2 up:standby 2024-09-10T10:42:15.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:15 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:42:16.689 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:16 smithi143 ceph-mon[31057]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:16 smithi178 ceph-mon[34813]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:16.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:16 smithi144 ceph-mon[34230]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:17.051 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:16 smithi143 systemd[1]: Starting Ceph mds.j for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:17.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:17.332 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 podman[70833]: 2024-09-10 10:42:16.975993444 +0000 UTC m=+0.015935807 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:17.332 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 podman[70833]: 2024-09-10 10:42:17.086658896 +0000 UTC m=+0.126601244 container create 74278ecfee44cc3740eb0f292b1d8cf9a0745018a9138081bc22a114905daaa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:42:17.332 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 podman[70833]: 2024-09-10 10:42:17.331460749 +0000 UTC m=+0.371403096 container init 74278ecfee44cc3740eb0f292b1d8cf9a0745018a9138081bc22a114905daaa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T10:42:17.544 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:17.667 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 podman[70833]: 2024-09-10 10:42:17.334875462 +0000 UTC m=+0.374817813 container start 74278ecfee44cc3740eb0f292b1d8cf9a0745018a9138081bc22a114905daaa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:42:17.668 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 ceph-mds[70881]: -- 172.21.15.143:0/1321822373 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c99c2f3860 con 0x55c99c2d0800 2024-09-10T10:42:17.668 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j[70865]: starting mds.j at 2024-09-10T10:42:17.668 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 bash[70833]: 74278ecfee44cc3740eb0f292b1d8cf9a0745018a9138081bc22a114905daaa1 2024-09-10T10:42:17.668 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:42:17 smithi143 systemd[1]: Started Ceph mds.j for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 -- 172.21.15.143:0/3920441237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4106f10 msgr2=0x7f56a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/3920441237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4106f10 0x7f56a410ea90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f569c009ca0 tx=0x7f569c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 -- 172.21.15.143:0/3920441237 shutdown_connections 2024-09-10T10:42:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/3920441237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4106f10 0x7f56a410ea90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/3920441237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56a4102790 0x7f56a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/3920441237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a4101de0 0x7f56a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 -- 172.21.15.143:0/3920441237 >> 172.21.15.143:0/3920441237 conn(0x7f56a40fd8c0 msgr2=0x7f56a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 -- 172.21.15.143:0/3920441237 shutdown_connections 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.031+0000 7f56a8b1e640 1 -- 172.21.15.143:0/3920441237 wait complete. 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 Processor -- start 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 -- start start 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 0x7f56a4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a4102790 0x7f56a41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56a4106f10 0x7f56a419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4111ec0 con 0x7f56a4101de0 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4111d40 con 0x7f56a4102790 2024-09-10T10:42:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a8b1e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4112040 con 0x7f56a4106f10 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 0x7f56a4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 0x7f56a4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56538/0 (socket says 172.21.15.143:56538) 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 -- 172.21.15.143:0/1066113304 learned_addr learned my addr 172.21.15.143:0/1066113304 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 -- 172.21.15.143:0/1066113304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56a4106f10 msgr2=0x7f56a419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56a4106f10 0x7f56a419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 -- 172.21.15.143:0/1066113304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a4102790 msgr2=0x7f56a41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a4102790 0x7f56a41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.032+0000 7f56a259c640 1 -- 172.21.15.143:0/1066113304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a419d430 con 0x7f56a4101de0 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.033+0000 7f56a259c640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 0x7f56a4198480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f569400b4a0 tx=0x7f569400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.033+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694014070 con 0x7f56a4101de0 2024-09-10T10:42:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.033+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5694010ac0 con 0x7f56a4101de0 2024-09-10T10:42:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.033+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a419d720 con 0x7f56a4101de0 2024-09-10T10:42:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.033+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694017c30 con 0x7f56a4101de0 2024-09-10T10:42:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.033+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41a5000 con 0x7f56a4101de0 2024-09-10T10:42:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.034+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a41021c0 con 0x7f56a4101de0 2024-09-10T10:42:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.035+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5694017070 con 0x7f56a4101de0 2024-09-10T10:42:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.036+0000 7f569b7fe640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f56900a2b80 0x7f56900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.036+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f56940ce380 con 0x7f56a4101de0 2024-09-10T10:42:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.036+0000 7f569bfff640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f56900a2b80 0x7f56900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.036+0000 7f569bfff640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f56900a2b80 0x7f56900a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56a40ffe50 tx=0x7f568c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.040+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f569409b720 con 0x7f56a4101de0 2024-09-10T10:42:18.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.248+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56a41997e0 con 0x7f56a4101de0 2024-09-10T10:42:18.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.249+0000 7f569b7fe640 1 -- 172.21.15.143:0/1066113304 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f56940bb0e0 con 0x7f56a4101de0 2024-09-10T10:42:18.251 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:18.251 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":4,"btime":"2024-09-10T10:42:17:475384+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:42:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 4 2024-09-10T10:42:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f56900a2b80 msgr2=0x7f56900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f56900a2b80 0x7f56900a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56a40ffe50 tx=0x7f568c006c90 comp rx=0 tx=0).stop 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 msgr2=0x7f56a4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 0x7f56a4198480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f569400b4a0 tx=0x7f569400b960 comp rx=0 tx=0).stop 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 shutdown_connections 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f56900a2b80 0x7f56900a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56a4106f10 0x7f56a419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a4102790 0x7f56a41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 --2- 172.21.15.143:0/1066113304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56a4101de0 0x7f56a4198480 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 >> 172.21.15.143:0/1066113304 conn(0x7f56a40fd8c0 msgr2=0x7f56a40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 shutdown_connections 2024-09-10T10:42:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:18.251+0000 7f56a8b1e640 1 -- 172.21.15.143:0/1066113304 wait complete. 2024-09-10T10:42:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.143:6834/3176590385,v1:172.21.15.143:6835/3176590385] up:boot 2024-09-10T10:42:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: fsmap 3 up:standby 2024-09-10T10:42:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: Deploying daemon mds.d on smithi143 2024-09-10T10:42:18.803 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1066113304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.143:6834/3176590385,v1:172.21.15.143:6835/3176590385] up:boot 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: fsmap 3 up:standby 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:18.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: Deploying daemon mds.d on smithi143 2024-09-10T10:42:18.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:18 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1066113304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.143:6834/3176590385,v1:172.21.15.143:6835/3176590385] up:boot 2024-09-10T10:42:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: fsmap 3 up:standby 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: Deploying daemon mds.d on smithi143 2024-09-10T10:42:18.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:18 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1066113304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:20.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:20 smithi143 ceph-mon[31057]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:20 smithi178 ceph-mon[34813]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:20 smithi144 ceph-mon[34230]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:21.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:21.375 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 -- 172.21.15.143:0/2176544007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 msgr2=0x7f8744114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 --2- 172.21.15.143:0/2176544007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 0x7f8744114f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f873c009ca0 tx=0x7f873c02a950 comp rx=0 tx=0).stop 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 -- 172.21.15.143:0/2176544007 shutdown_connections 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 --2- 172.21.15.143:0/2176544007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 0x7f8744114f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 --2- 172.21.15.143:0/2176544007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8744074eb0 0x7f8744112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 --2- 172.21.15.143:0/2176544007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f8744074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 -- 172.21.15.143:0/2176544007 >> 172.21.15.143:0/2176544007 conn(0x7f87440fd8c0 msgr2=0x7f87440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 -- 172.21.15.143:0/2176544007 shutdown_connections 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 -- 172.21.15.143:0/2176544007 wait complete. 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.890+0000 7f874a818640 1 Processor -- start 2024-09-10T10:42:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 -- start start 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f87441a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8744074eb0 0x7f87441a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 0x7f874419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8744118420 con 0x7f8744112b90 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87441182a0 con 0x7f8744074590 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f874a818640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87441185a0 con 0x7f8744074eb0 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f87441a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f87441a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46546/0 (socket says 172.21.15.143:46546) 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8748db5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 0x7f874419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 -- 172.21.15.143:0/792947853 learned_addr learned my addr 172.21.15.143:0/792947853 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f873bfff640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8744074eb0 0x7f87441a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 -- 172.21.15.143:0/792947853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8744074eb0 msgr2=0x7f87441a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8744074eb0 0x7f87441a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 -- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 msgr2=0x7f874419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 0x7f874419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 -- 172.21.15.143:0/792947853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874419efb0 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8743fff640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f87441a4690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f873400e9f0 tx=0x7f873400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.891+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873400cd80 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.892+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874419f2a0 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.892+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87340108e0 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.892+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734018920 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.892+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441ab4f0 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.892+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8744075cd0 con 0x7f8744074590 2024-09-10T10:42:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.894+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8734010a90 con 0x7f8744074590 2024-09-10T10:42:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.894+0000 7f8741ffb640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87300a2b80 0x7f87300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.894+0000 7f873bfff640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87300a2b80 0x7f87300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.894+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f87340cf350 con 0x7f8744074590 2024-09-10T10:42:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.895+0000 7f873bfff640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87300a2b80 0x7f87300a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8744100150 tx=0x7f872c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:21.897+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f873409c100 con 0x7f8744074590 2024-09-10T10:42:22.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.096+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8744074970 con 0x7f8744074590 2024-09-10T10:42:22.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.100+0000 7f8741ffb640 1 -- 172.21.15.143:0/792947853 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f87340a0400 con 0x7f8744074590 2024-09-10T10:42:22.102 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:22.102 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"btime":"2024-09-10T10:42:21:484226+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:42:22.103 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 5 2024-09-10T10:42:22.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87300a2b80 msgr2=0x7f87300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87300a2b80 0x7f87300a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8744100150 tx=0x7f872c00b390 comp rx=0 tx=0).stop 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 msgr2=0x7f87441a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f87441a4690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f873400e9f0 tx=0x7f873400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 shutdown_connections 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f87300a2b80 0x7f87300a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8744112b90 0x7f874419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8744074eb0 0x7f87441a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:22.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 --2- 172.21.15.143:0/792947853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8744074590 0x7f87441a4690 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:22.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 >> 172.21.15.143:0/792947853 conn(0x7f87440fd8c0 msgr2=0x7f87440ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:22.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 shutdown_connections 2024-09-10T10:42:22.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:22.102+0000 7f874a818640 1 -- 172.21.15.143:0/792947853 wait complete. 2024-09-10T10:42:22.416 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.416 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.416 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.416 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: Deploying daemon mds.e on smithi144 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.143:6836/2245538131,v1:172.21.15.143:6837/2245538131] up:boot 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: fsmap 4 up:standby 2024-09-10T10:42:22.417 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:22 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:42:22.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: Deploying daemon mds.e on smithi144 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.143:6836/2245538131,v1:172.21.15.143:6837/2245538131] up:boot 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: fsmap 4 up:standby 2024-09-10T10:42:22.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:22 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: Deploying daemon mds.e on smithi144 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.143:6836/2245538131,v1:172.21.15.143:6837/2245538131] up:boot 2024-09-10T10:42:22.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: fsmap 4 up:standby 2024-09-10T10:42:22.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:22 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:42:23.410 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:23 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/792947853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:23.410 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 systemd[1]: Starting Ceph mds.e for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:23.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:23 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/792947853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:23.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:23 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/792947853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:23.663 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 podman[63511]: 2024-09-10 10:42:23.31208141 +0000 UTC m=+0.015838685 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:23.663 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 podman[63511]: 2024-09-10 10:42:23.426692755 +0000 UTC m=+0.130450016 container create a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T10:42:23.663 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 podman[63511]: 2024-09-10 10:42:23.663058931 +0000 UTC m=+0.366816192 container init a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T10:42:23.936 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 podman[63511]: 2024-09-10 10:42:23.666035455 +0000 UTC m=+0.369792717 container start a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T10:42:23.936 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 ceph-mds[63530]: -- 172.21.15.144:0/4051873561 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a654609860 con 0x55a6545e7400 2024-09-10T10:42:23.936 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e[63526]: starting mds.e at 2024-09-10T10:42:23.936 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 bash[63511]: a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 2024-09-10T10:42:23.936 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:42:23 smithi144 systemd[1]: Started Ceph mds.e for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:24.399 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:24 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:24.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:24 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:24.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:24 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:24.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:25.045 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:25.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:25 smithi178 ceph-mon[34813]: Deploying daemon mds.f on smithi178 2024-09-10T10:42:25.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:25 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6834/3190468681,v1:172.21.15.144:6835/3190468681] up:boot 2024-09-10T10:42:25.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:25 smithi178 ceph-mon[34813]: fsmap 5 up:standby 2024-09-10T10:42:25.262 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:25 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:42:25.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:25 smithi144 ceph-mon[34230]: Deploying daemon mds.f on smithi178 2024-09-10T10:42:25.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:25 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6834/3190468681,v1:172.21.15.144:6835/3190468681] up:boot 2024-09-10T10:42:25.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:25 smithi144 ceph-mon[34230]: fsmap 5 up:standby 2024-09-10T10:42:25.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:25 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:42:25.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:25 smithi143 ceph-mon[31057]: Deploying daemon mds.f on smithi178 2024-09-10T10:42:25.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:25 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6834/3190468681,v1:172.21.15.144:6835/3190468681] up:boot 2024-09-10T10:42:25.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:25 smithi143 ceph-mon[31057]: fsmap 5 up:standby 2024-09-10T10:42:25.472 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:25 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:42:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 -- 172.21.15.143:0/2971921480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 msgr2=0x7f2160100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2971921480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f2160100cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2154009930 tx=0x7f215402a420 comp rx=0 tx=0).stop 2024-09-10T10:42:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 -- 172.21.15.143:0/2971921480 shutdown_connections 2024-09-10T10:42:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2971921480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 0x7f216010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2971921480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2160101290 0x7f2160107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2971921480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f2160100cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 -- 172.21.15.143:0/2971921480 >> 172.21.15.143:0/2971921480 conn(0x7f2160077210 msgr2=0x7f2160077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 -- 172.21.15.143:0/2971921480 shutdown_connections 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 -- 172.21.15.143:0/2971921480 wait complete. 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.549+0000 7f2166bf7640 1 Processor -- start 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 -- start start 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f21601983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2160101290 0x7f21601988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 0x7f216019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2160111e80 con 0x7f21601008e0 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2160111d00 con 0x7f2160101290 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2166bf7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2160112000 con 0x7f2160108160 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f21601983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f21601983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56604/0 (socket says 172.21.15.143:56604) 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 -- 172.21.15.143:0/2934485156 learned_addr learned my addr 172.21.15.143:0/2934485156 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f215ffff640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2160101290 0x7f21601988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2165194640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 0x7f216019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 -- 172.21.15.143:0/2934485156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 msgr2=0x7f216019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 0x7f216019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 -- 172.21.15.143:0/2934485156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2160101290 msgr2=0x7f21601988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:25.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2160101290 0x7f21601988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 -- 172.21.15.143:0/2934485156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216019d360 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2165194640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 0x7f216019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f2164993640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f21601983b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2154004770 tx=0x7f21540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.550+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215403f070 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.551+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f215403b460 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.551+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216019d5f0 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.551+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154035070 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.551+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21601a4f30 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.552+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216006a110 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.553+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f215403b610 con 0x7f21601008e0 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.553+0000 7f215dffb640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f21500a2b80 0x7f21500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:25.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.554+0000 7f215ffff640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f21500a2b80 0x7f21500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:25.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.554+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f21540eef20 con 0x7f21601008e0 2024-09-10T10:42:25.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.554+0000 7f215ffff640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f21500a2b80 0x7f21500a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f21601001c0 tx=0x7f214c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:25.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.558+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21540bc320 con 0x7f21601008e0 2024-09-10T10:42:25.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.758+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f216010a170 con 0x7f21601008e0 2024-09-10T10:42:25.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.758+0000 7f215dffb640 1 -- 172.21.15.143:0/2934485156 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4750 (secure 0 0 0) 0x7f21540bbc20 con 0x7f21601008e0 2024-09-10T10:42:25.761 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:25.761 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":6,"btime":"2024-09-10T10:42:24:100088+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T10:42:25.761 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 6 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f21500a2b80 msgr2=0x7f21500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f21500a2b80 0x7f21500a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f21601001c0 tx=0x7f214c00a2d0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 msgr2=0x7f21601983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f21601983b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2154004770 tx=0x7f21540042f0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 shutdown_connections 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f21500a2b80 0x7f21500a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2160108160 0x7f216019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2160101290 0x7f21601988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 --2- 172.21.15.143:0/2934485156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21601008e0 0x7f21601983b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 >> 172.21.15.143:0/2934485156 conn(0x7f2160077210 msgr2=0x7f2160105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 shutdown_connections 2024-09-10T10:42:25.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:25.761+0000 7f2166bf7640 1 -- 172.21.15.143:0/2934485156 wait complete. 2024-09-10T10:42:25.925 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:25 smithi178 systemd[1]: Starting Ceph mds.f for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:26.209 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:26 smithi178 ceph-mon[34813]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:26.209 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:26 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2934485156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:26.210 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:25 smithi178 podman[64665]: 2024-09-10 10:42:25.84791819 +0000 UTC m=+0.015798866 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:26.210 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:25 smithi178 podman[64665]: 2024-09-10 10:42:25.981089248 +0000 UTC m=+0.148969916 container create 2380f1afd3af9c440cfe028ec34fe438dd91a2959b87d33052bcf8fdd17e9bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T10:42:26.210 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:26 smithi178 podman[64665]: 2024-09-10 10:42:26.208933454 +0000 UTC m=+0.376814114 container init 2380f1afd3af9c440cfe028ec34fe438dd91a2959b87d33052bcf8fdd17e9bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:42:26.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:26 smithi144 ceph-mon[34230]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:26.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:26 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2934485156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:26.460 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:26 smithi143 ceph-mon[31057]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:26.460 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:26 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2934485156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:26.674 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:26 smithi178 podman[64665]: 2024-09-10 10:42:26.212121595 +0000 UTC m=+0.380002257 container start 2380f1afd3af9c440cfe028ec34fe438dd91a2959b87d33052bcf8fdd17e9bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:42:26.674 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:26 smithi178 ceph-mds[64684]: -- 172.21.15.178:0/108118173 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db20c3f860 con 0x55db20c1d000 2024-09-10T10:42:26.675 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: starting mds.f at 2024-09-10T10:42:26.675 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:26 smithi178 bash[64665]: 2380f1afd3af9c440cfe028ec34fe438dd91a2959b87d33052bcf8fdd17e9bd1 2024-09-10T10:42:26.675 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:42:26 smithi178 systemd[1]: Started Ceph mds.f for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:27.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: Deploying daemon mds.c on smithi178 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] up:boot 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: fsmap 6 up:standby 2024-09-10T10:42:27.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:27 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: Deploying daemon mds.c on smithi178 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] up:boot 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: fsmap 6 up:standby 2024-09-10T10:42:27.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:27 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: Deploying daemon mds.c on smithi178 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] up:boot 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: fsmap 6 up:standby 2024-09-10T10:42:27.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:27 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:42:28.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:28 smithi178 ceph-mon[34813]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:28.488 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:28.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:28 smithi144 ceph-mon[34230]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:28.701 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:28 smithi143 ceph-mon[31057]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:29.174 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:42:28 smithi178 systemd[1]: Started Ceph mds.c for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.226+0000 7feeb57ee640 1 -- 172.21.15.143:0/2176515318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01036e0 msgr2=0x7feeb0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.226+0000 7feeb57ee640 1 --2- 172.21.15.143:0/2176515318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01036e0 0x7feeb0107b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee98009930 tx=0x7fee9802a420 comp rx=0 tx=0).stop 2024-09-10T10:42:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 -- 172.21.15.143:0/2176515318 shutdown_connections 2024-09-10T10:42:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 --2- 172.21.15.143:0/2176515318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb01082a0 0x7feeb010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 --2- 172.21.15.143:0/2176515318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01036e0 0x7feeb0107b90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 --2- 172.21.15.143:0/2176515318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb0102d30 0x7feeb0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 -- 172.21.15.143:0/2176515318 >> 172.21.15.143:0/2176515318 conn(0x7feeb00fc7d0 msgr2=0x7feeb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 -- 172.21.15.143:0/2176515318 shutdown_connections 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 -- 172.21.15.143:0/2176515318 wait complete. 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 Processor -- start 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.227+0000 7feeb57ee640 1 -- start start 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 0x7feeb01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb01036e0 0x7feeb01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 0x7feeb019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feeb0111f50 con 0x7feeb01082a0 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feeb0111dd0 con 0x7feeb0102d30 2024-09-10T10:42:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feeb01120d0 con 0x7feeb01036e0 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 0x7feeb019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 0x7feeb01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeae7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb01036e0 0x7feeb01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 0x7feeb01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:34810/0 (socket says 172.21.15.143:34810) 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 0x7feeb019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37656/0 (socket says 172.21.15.143:37656) 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaeffd640 1 -- 172.21.15.143:0/1687353870 learned_addr learned my addr 172.21.15.143:0/1687353870 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 -- 172.21.15.143:0/1687353870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb01036e0 msgr2=0x7feeb01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb01036e0 0x7feeb01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 -- 172.21.15.143:0/1687353870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 msgr2=0x7feeb01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 0x7feeb01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 -- 172.21.15.143:0/1687353870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb019d460 con 0x7feeb01082a0 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaeffd640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 0x7feeb01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:42:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeae7fc640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb01036e0 0x7feeb01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeaf7fe640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 0x7feeb019cd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feea8009fd0 tx=0x7feea800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea800ce20 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feea8005020 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb019d750 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.228+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8010690 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.229+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb01a5030 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.229+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb0103110 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.231+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7feea80051d0 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.231+0000 7feea7fff640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee9c0a2b80 0x7fee9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.232+0000 7feeaeffd640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee9c0a2b80 0x7fee9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.232+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7feea80cf1e0 con 0x7feeb01082a0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.232+0000 7feeaeffd640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee9c0a2b80 0x7fee9c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feea0005fd0 tx=0x7feea000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.237+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feea809c5e0 con 0x7feeb01082a0 2024-09-10T10:42:29.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.450+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feeb0199810 con 0x7feeb01082a0 2024-09-10T10:42:29.452 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:29.452 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"btime":"2024-09-10T10:42:29:372413+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T10:42:29.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.450+0000 7feea7fff640 1 -- 172.21.15.143:0/1687353870 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7feeb0199810 con 0x7feeb01082a0 2024-09-10T10:42:29.453 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-09-10T10:42:29.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee9c0a2b80 msgr2=0x7fee9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:29.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee9c0a2b80 0x7fee9c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feea0005fd0 tx=0x7feea000b480 comp rx=0 tx=0).stop 2024-09-10T10:42:29.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 msgr2=0x7feeb019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:29.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 0x7feeb019cd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feea8009fd0 tx=0x7feea800ef10 comp rx=0 tx=0).stop 2024-09-10T10:42:29.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 shutdown_connections 2024-09-10T10:42:29.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee9c0a2b80 0x7fee9c0a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feeb01082a0 0x7feeb019cd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeb01036e0 0x7feeb01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 --2- 172.21.15.143:0/1687353870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb0102d30 0x7feeb01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:29.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 >> 172.21.15.143:0/1687353870 conn(0x7feeb00fc7d0 msgr2=0x7feeb0105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:29.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 shutdown_connections 2024-09-10T10:42:29.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:29.453+0000 7feeb57ee640 1 -- 172.21.15.143:0/1687353870 wait complete. 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: Deploying daemon mds.h on smithi144 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] up:boot 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: fsmap 7 up:standby 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:42:30.163 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:30.164 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:29 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1687353870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:30.417 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.417 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.417 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.417 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:30.417 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: Deploying daemon mds.h on smithi144 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] up:boot 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: fsmap 7 up:standby 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:30.418 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:29 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1687353870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:30.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: Deploying daemon mds.h on smithi144 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] up:boot 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: fsmap 7 up:standby 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:30.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:29 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1687353870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:31.186 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 systemd[1]: Starting Ceph mds.h for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:31.504 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 podman[63814]: 2024-09-10 10:42:31.241298841 +0000 UTC m=+0.110550860 container create 556e5bc8207e121e85ef0c8af8ceebef72e7b178f168406820bdab690d36b2dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T10:42:31.504 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 podman[63814]: 2024-09-10 10:42:31.146591986 +0000 UTC m=+0.015844005 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:31.505 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 podman[63814]: 2024-09-10 10:42:31.477469152 +0000 UTC m=+0.346721171 container init 556e5bc8207e121e85ef0c8af8ceebef72e7b178f168406820bdab690d36b2dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T10:42:31.505 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 podman[63814]: 2024-09-10 10:42:31.480572621 +0000 UTC m=+0.349824641 container start 556e5bc8207e121e85ef0c8af8ceebef72e7b178f168406820bdab690d36b2dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:42:31.505 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 ceph-mds[63834]: -- 172.21.15.144:0/4112049989 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591a8f65860 con 0x5591a8f43000 2024-09-10T10:42:31.936 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h[63830]: starting mds.h at 2024-09-10T10:42:31.936 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 bash[63814]: 556e5bc8207e121e85ef0c8af8ceebef72e7b178f168406820bdab690d36b2dc 2024-09-10T10:42:31.936 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:42:31 smithi144 systemd[1]: Started Ceph mds.h for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:32.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:32.425 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:32.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:32 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:32.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:32.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.724 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:32.725 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:32.725 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:32 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:32 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:32.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 -- 172.21.15.143:0/2952470721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 msgr2=0x7f70b410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:32.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 --2- 172.21.15.143:0/2952470721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b410d950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009cd0 tx=0x7f70ac02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 -- 172.21.15.143:0/2952470721 shutdown_connections 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 --2- 172.21.15.143:0/2952470721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b410d950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 --2- 172.21.15.143:0/2952470721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b4100a80 0x7f70b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 --2- 172.21.15.143:0/2952470721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b4106a80 0x7f70b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 -- 172.21.15.143:0/2952470721 >> 172.21.15.143:0/2952470721 conn(0x7f70b40fc7a0 msgr2=0x7f70b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 -- 172.21.15.143:0/2952470721 shutdown_connections 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.936+0000 7f70bc46f640 1 -- 172.21.15.143:0/2952470721 wait complete. 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 Processor -- start 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 -- start start 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b4100a80 0x7f70b40fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b40fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b4106a80 0x7f70b40fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4110d30 con 0x7f70b4101420 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4110bb0 con 0x7f70b4100a80 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70bc46f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4110eb0 con 0x7f70b4106a80 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b40fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b40fd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37676/0 (socket says 172.21.15.143:37676) 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 -- 172.21.15.143:0/1791286008 learned_addr learned my addr 172.21.15.143:0/1791286008 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70ba20b640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b4100a80 0x7f70b40fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70baa0c640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b4106a80 0x7f70b40fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 -- 172.21.15.143:0/1791286008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b4106a80 msgr2=0x7f70b40fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b4106a80 0x7f70b40fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 -- 172.21.15.143:0/1791286008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b4100a80 msgr2=0x7f70b40fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b4100a80 0x7f70b40fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 -- 172.21.15.143:0/1791286008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b41a67d0 con 0x7f70b4101420 2024-09-10T10:42:32.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.937+0000 7f70b9a0a640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b40fd330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70a400e9f0 tx=0x7f70a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.938+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a400cd80 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.938+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70a40054d0 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.938+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b41a6ac0 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.938+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4010490 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.938+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41a6fb0 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.939+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b4100ee0 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.940+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f70a4005020 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.940+0000 7f70ab7fe640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70a00a2b80 0x7f70a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.940+0000 7f70ba20b640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70a00a2b80 0x7f70a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.940+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f70a40cefb0 con 0x7f70b4101420 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.941+0000 7f70ba20b640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70a00a2b80 0x7f70a00a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70b000bb60 tx=0x7f70b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:32.944+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70a409c3b0 con 0x7f70b4101420 2024-09-10T10:42:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.142+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70b410a050 con 0x7f70b4101420 2024-09-10T10:42:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.142+0000 7f70ab7fe640 1 -- 172.21.15.143:0/1791286008 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7f70a409bcb0 con 0x7f70b4101420 2024-09-10T10:42:33.144 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:33.144 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":9,"btime":"2024-09-10T10:42:32:465373+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T10:42:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 9 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70a00a2b80 msgr2=0x7f70a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70a00a2b80 0x7f70a00a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70b000bb60 tx=0x7f70b000b390 comp rx=0 tx=0).stop 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 msgr2=0x7f70b40fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b40fd330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70a400e9f0 tx=0x7f70a400edf0 comp rx=0 tx=0).stop 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 shutdown_connections 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70a00a2b80 0x7f70a00a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b4106a80 0x7f70b40fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70b4101420 0x7f70b40fd330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 --2- 172.21.15.143:0/1791286008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b4100a80 0x7f70b40fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 >> 172.21.15.143:0/1791286008 conn(0x7f70b40fc7a0 msgr2=0x7f70b406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:33.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 shutdown_connections 2024-09-10T10:42:33.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:33.145+0000 7f70bc46f640 1 -- 172.21.15.143:0/1791286008 wait complete. 2024-09-10T10:42:33.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:33 smithi178 ceph-mon[34813]: Deploying daemon mds.i on smithi178 2024-09-10T10:42:33.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:33 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] up:boot 2024-09-10T10:42:33.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:33 smithi178 ceph-mon[34813]: fsmap 8 up:standby 2024-09-10T10:42:33.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:33 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:42:33.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:33 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1791286008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:33.675 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:33 smithi178 systemd[1]: Starting Ceph mds.i for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:33.746 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:33 smithi143 ceph-mon[31057]: Deploying daemon mds.i on smithi178 2024-09-10T10:42:33.746 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:33 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] up:boot 2024-09-10T10:42:33.746 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:33 smithi143 ceph-mon[31057]: fsmap 8 up:standby 2024-09-10T10:42:33.746 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:33 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:42:33.746 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:33 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1791286008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:33 smithi144 ceph-mon[34230]: Deploying daemon mds.i on smithi178 2024-09-10T10:42:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:33 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] up:boot 2024-09-10T10:42:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:33 smithi144 ceph-mon[34230]: fsmap 8 up:standby 2024-09-10T10:42:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:33 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:42:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:33 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1791286008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:34.069 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:33 smithi178 podman[65267]: 2024-09-10 10:42:33.708371599 +0000 UTC m=+0.016417851 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:34.069 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:33 smithi178 podman[65267]: 2024-09-10 10:42:33.820927854 +0000 UTC m=+0.128974096 container create ac24ad84cf7156482dfb5af9812ddfc09b599b83f0b1f26eafa9304b20d39bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:42:34.069 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:34 smithi178 podman[65267]: 2024-09-10 10:42:34.068733559 +0000 UTC m=+0.376779801 container init ac24ad84cf7156482dfb5af9812ddfc09b599b83f0b1f26eafa9304b20d39bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T10:42:34.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:34 smithi178 podman[65267]: 2024-09-10 10:42:34.071583202 +0000 UTC m=+0.379629453 container start ac24ad84cf7156482dfb5af9812ddfc09b599b83f0b1f26eafa9304b20d39bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T10:42:34.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mds[65286]: -- 172.21.15.178:0/3679181840 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3b3cf9860 con 0x55d3b3cd7400 2024-09-10T10:42:34.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: starting mds.i at 2024-09-10T10:42:34.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:34 smithi178 bash[65267]: ac24ad84cf7156482dfb5af9812ddfc09b599b83f0b1f26eafa9304b20d39bcf 2024-09-10T10:42:34.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:34 smithi178 systemd[1]: Started Ceph mds.i for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:34.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:34 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:34 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:34 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:35.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:35 smithi143 ceph-mon[31057]: Deploying daemon mds.g on smithi143 2024-09-10T10:42:35.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:35 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] up:boot 2024-09-10T10:42:35.738 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:35 smithi143 ceph-mon[31057]: fsmap 9 up:standby 2024-09-10T10:42:35.739 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:35 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:42:35.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:35.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:35 smithi178 ceph-mon[34813]: Deploying daemon mds.g on smithi143 2024-09-10T10:42:35.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:35 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] up:boot 2024-09-10T10:42:35.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:35 smithi178 ceph-mon[34813]: fsmap 9 up:standby 2024-09-10T10:42:35.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:35 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:42:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:35 smithi144 ceph-mon[34230]: Deploying daemon mds.g on smithi143 2024-09-10T10:42:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:35 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] up:boot 2024-09-10T10:42:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:35 smithi144 ceph-mon[34230]: fsmap 9 up:standby 2024-09-10T10:42:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:35 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:42:36.041 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:36.301 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:36 smithi143 systemd[1]: Starting Ceph mds.g for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:36.573 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:36 smithi143 ceph-mon[31057]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 -- 172.21.15.143:0/1200146075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0103780 msgr2=0x7fd7c0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 --2- 172.21.15.143:0/1200146075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0103780 0x7fd7c0107bc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009930 tx=0x7fd7a802a420 comp rx=0 tx=0).stop 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 -- 172.21.15.143:0/1200146075 shutdown_connections 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 --2- 172.21.15.143:0/1200146075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0108300 0x7fd7c010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 --2- 172.21.15.143:0/1200146075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0103780 0x7fd7c0107bc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 --2- 172.21.15.143:0/1200146075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0102dd0 0x7fd7c01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 -- 172.21.15.143:0/1200146075 >> 172.21.15.143:0/1200146075 conn(0x7fd7c00fc7d0 msgr2=0x7fd7c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 -- 172.21.15.143:0/1200146075 shutdown_connections 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.563+0000 7fd7c6043640 1 -- 172.21.15.143:0/1200146075 wait complete. 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 Processor -- start 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 -- start start 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0102dd0 0x7fd7c0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0103780 0x7fd7c0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 0x7fd7c01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0111f20 con 0x7fd7c0108300 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0111da0 con 0x7fd7c0103780 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7c6043640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c01120a0 con 0x7fd7c0102dd0 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 0x7fd7c01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 0x7fd7c01014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37880/0 (socket says 172.21.15.143:37880) 2024-09-10T10:42:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 -- 172.21.15.143:0/871035805 learned_addr learned my addr 172.21.15.143:0/871035805 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bf7fe640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0102dd0 0x7fd7c0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7b6ffd640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0103780 0x7fd7c0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 -- 172.21.15.143:0/871035805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0102dd0 msgr2=0x7fd7c0100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0102dd0 0x7fd7c0100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 -- 172.21.15.143:0/871035805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0103780 msgr2=0x7fd7c0100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0103780 0x7fd7c0100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bffff640 1 -- 172.21.15.143:0/871035805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c01a69c0 con 0x7fd7c0108300 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7bf7fe640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0102dd0 0x7fd7c0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.564+0000 7fd7b6ffd640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0103780 0x7fd7c0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.565+0000 7fd7bffff640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 0x7fd7c01014b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800eaa0 tx=0x7fd7b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.565+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800cdb0 con 0x7fd7c0108300 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.565+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7b8005020 con 0x7fd7c0108300 2024-09-10T10:42:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.565+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01a6cb0 con 0x7fd7c0108300 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.565+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80116d0 con 0x7fd7c0108300 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.565+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01a7240 con 0x7fd7c0108300 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.566+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c01031b0 con 0x7fd7c0108300 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.567+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd7b80051d0 con 0x7fd7c0108300 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.568+0000 7fd7bd7fa640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7ac0a2c00 0x7fd7ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.568+0000 7fd7bf7fe640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7ac0a2c00 0x7fd7ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.568+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd7b80ced40 con 0x7fd7c0108300 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.568+0000 7fd7bf7fe640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7ac0a2c00 0x7fd7ac0a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00058c0 tx=0x7fd7b0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.572+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7b8016030 con 0x7fd7c0108300 2024-09-10T10:42:36.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.782+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7c010a290 con 0x7fd7c0108300 2024-09-10T10:42:36.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.783+0000 7fd7bd7fa640 1 -- 172.21.15.143:0/871035805 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8058 (secure 0 0 0) 0x7fd7b809c140 con 0x7fd7c0108300 2024-09-10T10:42:36.784 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:36.784 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":10,"btime":"2024-09-10T10:42:34:477867+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:42:36.784 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 10 2024-09-10T10:42:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7ac0a2c00 msgr2=0x7fd7ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7ac0a2c00 0x7fd7ac0a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00058c0 tx=0x7fd7b0005ec0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 msgr2=0x7fd7c01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 0x7fd7c01014b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800eaa0 tx=0x7fd7b800ef60 comp rx=0 tx=0).stop 2024-09-10T10:42:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 shutdown_connections 2024-09-10T10:42:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7ac0a2c00 0x7fd7ac0a50c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c0108300 0x7fd7c01014b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7c0103780 0x7fd7c0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 --2- 172.21.15.143:0/871035805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7c0102dd0 0x7fd7c0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 >> 172.21.15.143:0/871035805 conn(0x7fd7c00fc7d0 msgr2=0x7fd7c0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 shutdown_connections 2024-09-10T10:42:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:36.785+0000 7fd7c6043640 1 -- 172.21.15.143:0/871035805 wait complete. 2024-09-10T10:42:36.900 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:36 smithi143 podman[72090]: 2024-09-10 10:42:36.637746536 +0000 UTC m=+0.148907656 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:36.900 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:36 smithi143 podman[72090]: 2024-09-10 10:42:36.770663026 +0000 UTC m=+0.281824146 container create b77e0014d299027f21889096d04319d05fff19d3d745ab9d76b3edd9c4df5621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:42:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:36 smithi178 ceph-mon[34813]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:36.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:36 smithi144 ceph-mon[34230]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:37.774 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:37 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/871035805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:37.774 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:37 smithi143 podman[72090]: 2024-09-10 10:42:37.57055809 +0000 UTC m=+1.081719201 container init b77e0014d299027f21889096d04319d05fff19d3d745ab9d76b3edd9c4df5621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:42:37.774 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:37 smithi143 podman[72090]: 2024-09-10 10:42:37.573936016 +0000 UTC m=+1.085097138 container start b77e0014d299027f21889096d04319d05fff19d3d745ab9d76b3edd9c4df5621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T10:42:37.774 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:37 smithi143 ceph-mds[72144]: -- 172.21.15.143:0/807190791 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df6aa95860 con 0x55df6aa73400 2024-09-10T10:42:37.774 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:37 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g[72138]: starting mds.g at 2024-09-10T10:42:37.774 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:37 smithi143 bash[72090]: b77e0014d299027f21889096d04319d05fff19d3d745ab9d76b3edd9c4df5621 2024-09-10T10:42:37.775 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:42:37 smithi143 systemd[1]: Started Ceph mds.g for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:37.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:37 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/871035805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:37.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:37 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/871035805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:38.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:38 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:38.825 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:38.825 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.825 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:38.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:38.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:38 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:38.838 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:38 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:39.759 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:39 smithi143 ceph-mon[31057]: Deploying daemon mds.a on smithi143 2024-09-10T10:42:39.759 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:39 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.143:6838/2518540467,v1:172.21.15.143:6839/2518540467] up:boot 2024-09-10T10:42:39.759 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:39 smithi143 ceph-mon[31057]: fsmap 10 up:standby 2024-09-10T10:42:39.759 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:39 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:42:39.759 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:39 smithi143 systemd[1]: Starting Ceph mds.a for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:42:39.844 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:39 smithi178 ceph-mon[34813]: Deploying daemon mds.a on smithi143 2024-09-10T10:42:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:39 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.143:6838/2518540467,v1:172.21.15.143:6839/2518540467] up:boot 2024-09-10T10:42:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:39 smithi178 ceph-mon[34813]: fsmap 10 up:standby 2024-09-10T10:42:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:39 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:42:39.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:39 smithi144 ceph-mon[34230]: Deploying daemon mds.a on smithi143 2024-09-10T10:42:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:39 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.143:6838/2518540467,v1:172.21.15.143:6839/2518540467] up:boot 2024-09-10T10:42:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:39 smithi144 ceph-mon[34230]: fsmap 10 up:standby 2024-09-10T10:42:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:39 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:42:40.051 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:39 smithi143 podman[72423]: 2024-09-10 10:42:39.825695554 +0000 UTC m=+0.015738790 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:40.051 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:39 smithi143 podman[72423]: 2024-09-10 10:42:39.958817069 +0000 UTC m=+0.148860295 container create 0d481ce063579505fdb47f20c6a939cd50283611b856eee107522fe0c5c2c553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T10:42:40.346 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:40 smithi143 podman[72423]: 2024-09-10 10:42:40.237233459 +0000 UTC m=+0.427276685 container init 0d481ce063579505fdb47f20c6a939cd50283611b856eee107522fe0c5c2c553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T10:42:40.347 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:40 smithi143 podman[72423]: 2024-09-10 10:42:40.24040302 +0000 UTC m=+0.430446249 container start 0d481ce063579505fdb47f20c6a939cd50283611b856eee107522fe0c5c2c553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T10:42:40.347 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mds[72484]: -- 172.21.15.143:0/1988517591 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb3764f860 con 0x55bb3762d400 2024-09-10T10:42:40.347 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a[72470]: starting mds.a at 2024-09-10T10:42:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:40.654 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:40 smithi143 bash[72423]: 0d481ce063579505fdb47f20c6a939cd50283611b856eee107522fe0c5c2c553 2024-09-10T10:42:40.654 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:42:40 smithi143 systemd[1]: Started Ceph mds.a for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:40.654 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:40.654 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.654 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.655 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.655 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:40.655 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:40.655 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:40 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:40.836 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:40.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:40.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:40.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:40 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2406908251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8106a90 msgr2=0x7f76f8106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2406908251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8106a90 0x7f76f8106e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009cd0 tx=0x7f76e802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2406908251 shutdown_connections 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2406908251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76f8101430 0x7f76f810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2406908251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f8100a90 0x7f76f8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2406908251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8106a90 0x7f76f8106e70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2406908251 >> 172.21.15.143:0/2406908251 conn(0x7f76f80fc7d0 msgr2=0x7f76f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2406908251 shutdown_connections 2024-09-10T10:42:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.864+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2406908251 wait complete. 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 Processor -- start 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 -- start start 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76f8100a90 0x7f76f8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 0x7f76f8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f8106a90 0x7f76f8197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8110de0 con 0x7f76f8101430 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8110c60 con 0x7f76f8106a90 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76fe1c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76f8110f60 con 0x7f76f8100a90 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 0x7f76f8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 0x7f76f8197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37926/0 (socket says 172.21.15.143:37926) 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 -- 172.21.15.143:0/2129282164 learned_addr learned my addr 172.21.15.143:0/2129282164 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f77fe640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76f8100a90 0x7f76f8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 -- 172.21.15.143:0/2129282164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76f8100a90 msgr2=0x7f76f8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76f8100a90 0x7f76f8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 -- 172.21.15.143:0/2129282164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f8106a90 msgr2=0x7f76f8197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.865+0000 7f76f6ffd640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f8106a90 0x7f76f8197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76f6ffd640 1 -- 172.21.15.143:0/2129282164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f81a69b0 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76f6ffd640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 0x7f76f8197680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b4a0 tx=0x7f76e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0014070 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e0010ac0 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f81a6ca0 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0017c30 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.866+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f81a7190 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.867+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f8100f70 con 0x7f76f8101430 2024-09-10T10:42:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.868+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f76e0017070 con 0x7f76f8101430 2024-09-10T10:42:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.869+0000 7f76f4ff9640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f76e40a2b80 0x7f76e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.869+0000 7f76f77fe640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f76e40a2b80 0x7f76e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.869+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f76e00ce270 con 0x7f76f8101430 2024-09-10T10:42:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.870+0000 7f76f77fe640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f76e40a2b80 0x7f76e40a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76e8006010 tx=0x7f76e8005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:40.873+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76e009b670 con 0x7f76f8101430 2024-09-10T10:42:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:42:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:42:40.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:40 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:41.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.076+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f802d7c0 con 0x7f76f8101430 2024-09-10T10:42:41.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.076+0000 7f76f4ff9640 1 -- 172.21.15.143:0/2129282164 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9716 (secure 0 0 0) 0x7f76e009af70 con 0x7f76f8101430 2024-09-10T10:42:41.078 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:41.078 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":12,"btime":"2024-09-10T10:42:40:508632+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:42:41.078 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 12 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f76e40a2b80 msgr2=0x7f76e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f76e40a2b80 0x7f76e40a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76e8006010 tx=0x7f76e8005f70 comp rx=0 tx=0).stop 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 msgr2=0x7f76f8197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 0x7f76f8197680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b4a0 tx=0x7f76e000b960 comp rx=0 tx=0).stop 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 shutdown_connections 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f76e40a2b80 0x7f76e40a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f8106a90 0x7f76f8197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f8101430 0x7f76f8197680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 --2- 172.21.15.143:0/2129282164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76f8100a90 0x7f76f8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.078+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 >> 172.21.15.143:0/2129282164 conn(0x7f76f80fc7d0 msgr2=0x7f76f80fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:41.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.079+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 shutdown_connections 2024-09-10T10:42:41.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:41.079+0000 7f76fe1c8640 1 -- 172.21.15.143:0/2129282164 wait complete. 2024-09-10T10:42:41.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:41 smithi143 ceph-mon[31057]: Deploying daemon mds.b on smithi144 2024-09-10T10:42:41.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:41 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.143:6840/1083221625,v1:172.21.15.143:6841/1083221625] up:boot 2024-09-10T10:42:41.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:41 smithi143 ceph-mon[31057]: fsmap 11 up:standby 2024-09-10T10:42:41.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:41 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:42:41.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:41 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2129282164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:41.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:41 smithi144 ceph-mon[34230]: Deploying daemon mds.b on smithi144 2024-09-10T10:42:41.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:41 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.143:6840/1083221625,v1:172.21.15.143:6841/1083221625] up:boot 2024-09-10T10:42:41.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:41 smithi144 ceph-mon[34230]: fsmap 11 up:standby 2024-09-10T10:42:41.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:41 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:42:41.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:41 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2129282164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:41.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:41 smithi178 ceph-mon[34813]: Deploying daemon mds.b on smithi144 2024-09-10T10:42:41.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:41 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.143:6840/1083221625,v1:172.21.15.143:6841/1083221625] up:boot 2024-09-10T10:42:41.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:41 smithi178 ceph-mon[34813]: fsmap 11 up:standby 2024-09-10T10:42:41.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:41 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:42:41.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:41 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2129282164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:42.781 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:42 smithi144 ceph-mon[34230]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:42.782 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:42 smithi144 podman[64127]: 2024-09-10 10:42:42.669628272 +0000 UTC m=+0.016034521 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:42:42.782 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:42 smithi144 podman[64127]: 2024-09-10 10:42:42.781260848 +0000 UTC m=+0.127667094 container create e2d6791d1098e57d647f0fb295e04864742ad8af42f38a5c3391b1a21ef7b45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T10:42:42.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:42 smithi143 ceph-mon[31057]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:42 smithi178 ceph-mon[34813]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:43.035 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:43 smithi144 podman[64127]: 2024-09-10 10:42:43.009164052 +0000 UTC m=+0.355570293 container init e2d6791d1098e57d647f0fb295e04864742ad8af42f38a5c3391b1a21ef7b45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T10:42:43.036 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:43 smithi144 podman[64127]: 2024-09-10 10:42:43.011892496 +0000 UTC m=+0.358298742 container start e2d6791d1098e57d647f0fb295e04864742ad8af42f38a5c3391b1a21ef7b45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:42:43.436 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:43 smithi144 ceph-mds[64146]: -- 172.21.15.144:0/772789778 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c2bb4f860 con 0x564c2bb2d400 2024-09-10T10:42:43.436 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:43 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b[64142]: starting mds.b at 2024-09-10T10:42:43.436 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:43 smithi144 bash[64127]: e2d6791d1098e57d647f0fb295e04864742ad8af42f38a5c3391b1a21ef7b45f 2024-09-10T10:42:43.436 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:42:43 smithi144 systemd[1]: Started Ceph mds.b for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:42:43.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] up:boot 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: fsmap 12 up:standby 2024-09-10T10:42:44.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:44 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:42:44.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] up:boot 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: fsmap 12 up:standby 2024-09-10T10:42:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:44 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:42:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] up:boot 2024-09-10T10:42:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: fsmap 12 up:standby 2024-09-10T10:42:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:44 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:42:44.439 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:45.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 -- 172.21.15.143:0/235796886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c102d40 msgr2=0x7fc29c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:45.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/235796886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c102d40 0x7fc29c103120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009900 tx=0x7fc28c02a480 comp rx=0 tx=0).stop 2024-09-10T10:42:45.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 -- 172.21.15.143:0/235796886 shutdown_connections 2024-09-10T10:42:45.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/235796886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc29c10c170 0x7fc29c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/235796886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc29c103660 0x7fc29c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/235796886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c102d40 0x7fc29c103120 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 -- 172.21.15.143:0/235796886 >> 172.21.15.143:0/235796886 conn(0x7fc29c0fc7a0 msgr2=0x7fc29c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 -- 172.21.15.143:0/235796886 shutdown_connections 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.985+0000 7fc2a21d1640 1 -- 172.21.15.143:0/235796886 wait complete. 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 Processor -- start 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 -- start start 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc29c102d40 0x7fc29c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 0x7fc29c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc29c10c170 0x7fc29c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c111e20 con 0x7fc29c103660 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c111ca0 con 0x7fc29c102d40 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc2a21d1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c111fa0 con 0x7fc29c10c170 2024-09-10T10:42:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc29affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 0x7fc29c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc29affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 0x7fc29c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37944/0 (socket says 172.21.15.143:37944) 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc29affd640 1 -- 172.21.15.143:0/1041494330 learned_addr learned my addr 172.21.15.143:0/1041494330 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc29b7fe640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc29c102d40 0x7fc29c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.986+0000 7fc29bfff640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc29c10c170 0x7fc29c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc29affd640 1 -- 172.21.15.143:0/1041494330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc29c10c170 msgr2=0x7fc29c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc29affd640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc29c10c170 0x7fc29c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc29affd640 1 -- 172.21.15.143:0/1041494330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc29c102d40 msgr2=0x7fc29c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc29affd640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc29c102d40 0x7fc29c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc29affd640 1 -- 172.21.15.143:0/1041494330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c19d3b0 con 0x7fc29c103660 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc29affd640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 0x7fc29c198940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e9f0 tx=0x7fc29000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000ccc0 con 0x7fc29c103660 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2900108e0 con 0x7fc29c103660 2024-09-10T10:42:45.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900187c0 con 0x7fc29c103660 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c19d6a0 con 0x7fc29c103660 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.987+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c1a4f80 con 0x7fc29c103660 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.988+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c103120 con 0x7fc29c103660 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.989+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc290010430 con 0x7fc29c103660 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.990+0000 7fc298ff9640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2840a2b80 0x7fc2840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.990+0000 7fc29b7fe640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2840a2b80 0x7fc2840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.990+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc2900ceeb0 con 0x7fc29c103660 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.990+0000 7fc29b7fe640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2840a2b80 0x7fc2840a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc28c031440 tx=0x7fc28c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:44.994+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc29009bc60 con 0x7fc29c103660 2024-09-10T10:42:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.233+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc29c107210 con 0x7fc29c103660 2024-09-10T10:42:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.234+0000 7fc298ff9640 1 -- 172.21.15.143:0/1041494330 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10545 (secure 0 0 0) 0x7fc29009ff60 con 0x7fc29c103660 2024-09-10T10:42:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:45.236 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":13,"btime":"2024-09-10T10:42:43:520133+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:42:45.236 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 13 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2840a2b80 msgr2=0x7fc2840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2840a2b80 0x7fc2840a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc28c031440 tx=0x7fc28c005b50 comp rx=0 tx=0).stop 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 msgr2=0x7fc29c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 0x7fc29c198940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e9f0 tx=0x7fc29000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 shutdown_connections 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2840a2b80 0x7fc2840a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc29c10c170 0x7fc29c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc29c103660 0x7fc29c198940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 --2- 172.21.15.143:0/1041494330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc29c102d40 0x7fc29c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 >> 172.21.15.143:0/1041494330 conn(0x7fc29c0fc7a0 msgr2=0x7fc29c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 shutdown_connections 2024-09-10T10:42:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:45.237+0000 7fc2a21d1640 1 -- 172.21.15.143:0/1041494330 wait complete. 2024-09-10T10:42:45.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:45 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:45 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1041494330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:45.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:45 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:45 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1041494330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:45 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:45 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1041494330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:42:45.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T10:42:46.267 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:46.667 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:46.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:46.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:46.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.668 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:46 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:46.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:46.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:46.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:46.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:46 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:46.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:46.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:46 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 -- 172.21.15.143:0/448394058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 msgr2=0x7fdd8c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 --2- 172.21.15.143:0/448394058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c10e610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009ca0 tx=0x7fdd8402a950 comp rx=0 tx=0).stop 2024-09-10T10:42:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 -- 172.21.15.143:0/448394058 shutdown_connections 2024-09-10T10:42:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 --2- 172.21.15.143:0/448394058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c10e610 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 --2- 172.21.15.143:0/448394058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 0x7fdd8c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 --2- 172.21.15.143:0/448394058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c102d40 0x7fdd8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 -- 172.21.15.143:0/448394058 >> 172.21.15.143:0/448394058 conn(0x7fdd8c0fc7a0 msgr2=0x7fdd8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 -- 172.21.15.143:0/448394058 shutdown_connections 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.747+0000 7fdd92b81640 1 -- 172.21.15.143:0/448394058 wait complete. 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 Processor -- start 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 -- start start 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c102d40 0x7fdd8c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 0x7fdd8c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c111e20 con 0x7fdd8c10c170 2024-09-10T10:42:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c111ca0 con 0x7fdd8c102d40 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd92b81640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c111fa0 con 0x7fdd8c103660 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd9111e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd9111e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43254/0 (socket says 172.21.15.143:43254) 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd9111e640 1 -- 172.21.15.143:0/331180152 learned_addr learned my addr 172.21.15.143:0/331180152 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd9091d640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c102d40 0x7fdd8c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.748+0000 7fdd83fff640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 0x7fdd8c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd9111e640 1 -- 172.21.15.143:0/331180152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 msgr2=0x7fdd8c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd9111e640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 0x7fdd8c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd9111e640 1 -- 172.21.15.143:0/331180152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c102d40 msgr2=0x7fdd8c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd9111e640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c102d40 0x7fdd8c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd9111e640 1 -- 172.21.15.143:0/331180152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c19d350 con 0x7fdd8c10c170 2024-09-10T10:42:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd83fff640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 0x7fdd8c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd9111e640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c19cc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd8402ae70 tx=0x7fdd8402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84034040 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd840308f0 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c19d5e0 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840047f0 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.749+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c1a4f20 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.750+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c06b070 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.751+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdd84037020 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.752+0000 7fdd81ffb640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd740a2b80 0x7fdd740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.752+0000 7fdd9091d640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd740a2b80 0x7fdd740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.753+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fdd840efdb0 con 0x7fdd8c10c170 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.753+0000 7fdd9091d640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd740a2b80 0x7fdd740a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800bbe0 tx=0x7fdd8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.756+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd840811b0 con 0x7fdd8c10c170 2024-09-10T10:42:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.938+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdd8c1071e0 con 0x7fdd8c10c170 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.939+0000 7fdd81ffb640 1 -- 172.21.15.143:0/331180152 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7fdd840bd110 con 0x7fdd8c10c170 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:42:46.941 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-09-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd740a2b80 msgr2=0x7fdd740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd740a2b80 0x7fdd740a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800bbe0 tx=0x7fdd8800b390 comp rx=0 tx=0).stop 2024-09-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 msgr2=0x7fdd8c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c19cc70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd8402ae70 tx=0x7fdd8402f5d0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 shutdown_connections 2024-09-10T10:42:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd740a2b80 0x7fdd740a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd8c10c170 0x7fdd8c19cc70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd8c103660 0x7fdd8c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 --2- 172.21.15.143:0/331180152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c102d40 0x7fdd8c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 >> 172.21.15.143:0/331180152 conn(0x7fdd8c0fc7a0 msgr2=0x7fdd8c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 shutdown_connections 2024-09-10T10:42:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:46.941+0000 7fdd92b81640 1 -- 172.21.15.143:0/331180152 wait complete. 2024-09-10T10:42:47.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:47 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/331180152' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:42:47.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:47 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/331180152' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:42:47.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:47 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/331180152' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:42:48.259 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:42:48.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:42:48.538 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:48.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:48 smithi143 ceph-mon[31057]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:48.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:48 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:48 smithi178 ceph-mon[34813]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:48 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:48 smithi144 ceph-mon[34230]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:48.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:48 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 -- 172.21.15.143:0/1562823210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4102790 msgr2=0x7f94d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 --2- 172.21.15.143:0/1562823210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4102790 0x7f94d41067e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009900 tx=0x7f94c802a410 comp rx=0 tx=0).stop 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 -- 172.21.15.143:0/1562823210 shutdown_connections 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 --2- 172.21.15.143:0/1562823210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4106f10 0x7f94d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 --2- 172.21.15.143:0/1562823210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4102790 0x7f94d41067e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 --2- 172.21.15.143:0/1562823210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94d4101de0 0x7f94d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 -- 172.21.15.143:0/1562823210 >> 172.21.15.143:0/1562823210 conn(0x7f94d40fd8c0 msgr2=0x7f94d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 -- 172.21.15.143:0/1562823210 shutdown_connections 2024-09-10T10:42:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.057+0000 7f94d918b640 1 -- 172.21.15.143:0/1562823210 wait complete. 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 Processor -- start 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 -- start start 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 0x7f94d41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4102790 0x7f94d4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94d4106f10 0x7f94d419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4111e40 con 0x7f94d4101de0 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4111cc0 con 0x7f94d4102790 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d918b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4111fc0 con 0x7f94d4106f10 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 0x7f94d41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 0x7f94d41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43274/0 (socket says 172.21.15.143:43274) 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.058+0000 7f94d2d9d640 1 -- 172.21.15.143:0/3467574992 learned_addr learned my addr 172.21.15.143:0/3467574992 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:49.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d259c640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4102790 0x7f94d4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d359e640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94d4106f10 0x7f94d419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d2d9d640 1 -- 172.21.15.143:0/3467574992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94d4106f10 msgr2=0x7f94d419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d2d9d640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94d4106f10 0x7f94d419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d2d9d640 1 -- 172.21.15.143:0/3467574992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4102790 msgr2=0x7f94d4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d2d9d640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4102790 0x7f94d4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d2d9d640 1 -- 172.21.15.143:0/3467574992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d419d370 con 0x7f94d4101de0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d259c640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4102790 0x7f94d4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d2d9d640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 0x7f94d41983c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c890 tx=0x7f94c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4004220 con 0x7f94d4101de0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d419d660 con 0x7f94d4101de0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94c40043c0 con 0x7f94d4101de0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4004a00 con 0x7f94d4101de0 2024-09-10T10:42:49.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.059+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a4f40 con 0x7f94d4101de0 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.060+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d41021c0 con 0x7f94d4101de0 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.061+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f94c4004ba0 con 0x7f94d4101de0 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.062+0000 7f94bbfff640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f94bc0a2b80 0x7f94bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.062+0000 7f94d259c640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f94bc0a2b80 0x7f94bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.062+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f94c40ce910 con 0x7f94d4101de0 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.062+0000 7f94d259c640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f94bc0a2b80 0x7f94bc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f94c8000c00 tx=0x7f94c8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:49.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.066+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94c4014030 con 0x7f94d4101de0 2024-09-10T10:42:49.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:49.252+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f94d402d7c0 con 0x7f94d4101de0 2024-09-10T10:42:50.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.209+0000 7f94bbfff640 1 -- 172.21.15.143:0/3467574992 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f94c409bd10 con 0x7f94d4101de0 2024-09-10T10:42:50.211 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-09-10T10:42:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f94bc0a2b80 msgr2=0x7f94bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f94bc0a2b80 0x7f94bc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f94c8000c00 tx=0x7f94c8031400 comp rx=0 tx=0).stop 2024-09-10T10:42:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 msgr2=0x7f94d41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 0x7f94d41983c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c890 tx=0x7f94c400cd50 comp rx=0 tx=0).stop 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 shutdown_connections 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f94bc0a2b80 0x7f94bc0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94d4106f10 0x7f94d419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d4102790 0x7f94d4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 --2- 172.21.15.143:0/3467574992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d4101de0 0x7f94d41983c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 >> 172.21.15.143:0/3467574992 conn(0x7f94d40fd8c0 msgr2=0x7f94d40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 shutdown_connections 2024-09-10T10:42:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:50.213+0000 7f94d918b640 1 -- 172.21.15.143:0/3467574992 wait complete. 2024-09-10T10:42:50.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:50.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:50.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3467574992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:42:50.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:50 smithi178 ceph-mon[34813]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3467574992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:42:50.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:50 smithi144 ceph-mon[34230]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:50.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:50.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:50.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:42:50.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3467574992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:42:50.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:50 smithi143 ceph-mon[31057]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:50.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T10:42:51.199 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:51.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:51 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3467574992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:42:51.471 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:51 smithi143 ceph-mon[31057]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:42:51.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:51 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3467574992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:42:51.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:51 smithi178 ceph-mon[34813]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:42:51.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:51 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3467574992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:42:51.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:51 smithi144 ceph-mon[34230]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 -- 172.21.15.143:0/2538629261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 msgr2=0x7fd6341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2538629261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd6341029b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd620009900 tx=0x7fd62002a480 comp rx=0 tx=0).stop 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 -- 172.21.15.143:0/2538629261 shutdown_connections 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2538629261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 0x7fd63410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2538629261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd6341029b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2538629261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd634100e20 0x7fd634101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 -- 172.21.15.143:0/2538629261 >> 172.21.15.143:0/2538629261 conn(0x7fd6340fc7d0 msgr2=0x7fd6340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 -- 172.21.15.143:0/2538629261 shutdown_connections 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 -- 172.21.15.143:0/2538629261 wait complete. 2024-09-10T10:42:51.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 Processor -- start 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.678+0000 7fd63ae41640 1 -- start start 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd634100e20 0x7fd634198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd634198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 0x7fd63419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd634111f50 con 0x7fd634102570 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd634111dd0 con 0x7fd634102ef0 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341120d0 con 0x7fd634100e20 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd634198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd634198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43294/0 (socket says 172.21.15.143:43294) 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 -- 172.21.15.143:0/2438662842 learned_addr learned my addr 172.21.15.143:0/2438662842 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd638bdd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd634100e20 0x7fd634198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd6393de640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 0x7fd63419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:51.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 -- 172.21.15.143:0/2438662842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd634100e20 msgr2=0x7fd634198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd634100e20 0x7fd634198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 -- 172.21.15.143:0/2438662842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 msgr2=0x7fd63419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 0x7fd63419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 -- 172.21.15.143:0/2438662842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63419d400 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd6393de640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 0x7fd63419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd633fff640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd634198990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd620006010 tx=0x7fd6200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62003f030 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd620003cb0 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63419d690 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.679+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620035070 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.680+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6341a4fd0 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.681+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd634101200 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.682+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd620042030 con 0x7fd634102570 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.682+0000 7fd631ffb640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6240a2b80 0x7fd6240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:51.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.682+0000 7fd638bdd640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6240a2b80 0x7fd6240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.682+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7fd6200ef480 con 0x7fd634102570 2024-09-10T10:42:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.683+0000 7fd638bdd640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6240a2b80 0x7fd6240a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd628005fd0 tx=0x7fd62800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:51.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.686+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6200bc710 con 0x7fd634102570 2024-09-10T10:42:51.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:51.851+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd634109090 con 0x7fd634102570 2024-09-10T10:42:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.224+0000 7fd631ffb640 1 -- 172.21.15.143:0/2438662842 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) ==== 171+0+0 (secure 0 0 0) 0x7fd6200bc010 con 0x7fd634102570 2024-09-10T10:42:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-09-10T10:42:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6240a2b80 msgr2=0x7fd6240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6240a2b80 0x7fd6240a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd628005fd0 tx=0x7fd62800b390 comp rx=0 tx=0).stop 2024-09-10T10:42:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 msgr2=0x7fd634198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd634198990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd620006010 tx=0x7fd6200042f0 comp rx=0 tx=0).stop 2024-09-10T10:42:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 shutdown_connections 2024-09-10T10:42:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6240a2b80 0x7fd6240a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd634102ef0 0x7fd63419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd634102570 0x7fd634198990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 --2- 172.21.15.143:0/2438662842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd634100e20 0x7fd634198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 >> 172.21.15.143:0/2438662842 conn(0x7fd6340fc7d0 msgr2=0x7fd6340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.226+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 shutdown_connections 2024-09-10T10:42:52.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:52.227+0000 7fd63ae41640 1 -- 172.21.15.143:0/2438662842 wait complete. 2024-09-10T10:42:52.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:52 smithi143 ceph-mon[31057]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:42:52.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:52 smithi143 ceph-mon[31057]: pgmap v274: 33 pgs: 1 creating+activating, 12 creating+peering, 1 active+clean, 19 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:52.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:52 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2438662842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T10:42:52.676 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:52 smithi178 ceph-mon[34813]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:42:52.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:52 smithi178 ceph-mon[34813]: pgmap v274: 33 pgs: 1 creating+activating, 12 creating+peering, 1 active+clean, 19 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:52.677 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:52 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2438662842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T10:42:52.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:52 smithi144 ceph-mon[34230]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:42:52.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:52 smithi144 ceph-mon[34230]: pgmap v274: 33 pgs: 1 creating+activating, 12 creating+peering, 1 active+clean, 19 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:52.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:52 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2438662842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T10:42:52.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:42:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:53.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:53 smithi143 ceph-mon[31057]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:42:53.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:53 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2438662842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T10:42:53.239 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:53 smithi143 ceph-mon[31057]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:42:53.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:53 smithi178 ceph-mon[34813]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:42:53.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:53 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2438662842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T10:42:53.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:53 smithi178 ceph-mon[34813]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:42:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:53 smithi144 ceph-mon[34230]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:42:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:53 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2438662842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T10:42:53.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:53 smithi144 ceph-mon[34230]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:42:53.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 -- 172.21.15.143:0/1846847444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 msgr2=0x7efc081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/1846847444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081029b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009900 tx=0x7efbfc02a480 comp rx=0 tx=0).stop 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 -- 172.21.15.143:0/1846847444 shutdown_connections 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/1846847444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08102ef0 0x7efc0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/1846847444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081029b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/1846847444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08100e20 0x7efc08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 -- 172.21.15.143:0/1846847444 >> 172.21.15.143:0/1846847444 conn(0x7efc080fc7d0 msgr2=0x7efc080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 -- 172.21.15.143:0/1846847444 shutdown_connections 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.745+0000 7efc0c8fd640 1 -- 172.21.15.143:0/1846847444 wait complete. 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 Processor -- start 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 -- start start 2024-09-10T10:42:53.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08100e20 0x7efc081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08102ef0 0x7efc0819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc08110e30 con 0x7efc08102570 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc08110cb0 con 0x7efc08102ef0 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0c8fd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc08110fb0 con 0x7efc08100e20 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43312/0 (socket says 172.21.15.143:43312) 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 -- 172.21.15.143:0/248239296 learned_addr learned my addr 172.21.15.143:0/248239296 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc0659c640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08100e20 0x7efc081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc06d9d640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08102ef0 0x7efc0819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 -- 172.21.15.143:0/248239296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08100e20 msgr2=0x7efc081984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:53.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08100e20 0x7efc081984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 -- 172.21.15.143:0/248239296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08102ef0 msgr2=0x7efc0819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08102ef0 0x7efc0819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc05d9b640 1 -- 172.21.15.143:0/248239296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0819d460 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.746+0000 7efc06d9d640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08102ef0 0x7efc0819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.747+0000 7efc05d9b640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081989f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efbfc006010 tx=0x7efbfc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.747+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc03f030 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.747+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0819d6f0 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.747+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbfc003cb0 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.747+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc035070 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.747+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc081a5030 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.748+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc08101200 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.749+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efbfc042030 con 0x7efc08102570 2024-09-10T10:42:53.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.749+0000 7efbf77fe640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efbf00a2b80 0x7efbf00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:53.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.750+0000 7efc0659c640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efbf00a2b80 0x7efbf00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:53.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.750+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7efbfc0ef6a0 con 0x7efc08102570 2024-09-10T10:42:53.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.750+0000 7efc0659c640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efbf00a2b80 0x7efbf00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efbf800bbe0 tx=0x7efbf800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:53.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.753+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efbfc0bc7b0 con 0x7efc08102570 2024-09-10T10:42:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:53.954+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efc08199810 con 0x7efc08102570 2024-09-10T10:42:54.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.242+0000 7efbf77fe640 1 -- 172.21.15.143:0/248239296 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7efc08199810 con 0x7efc08102570 2024-09-10T10:42:54.244 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efbf00a2b80 msgr2=0x7efbf00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efbf00a2b80 0x7efbf00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efbf800bbe0 tx=0x7efbf800b390 comp rx=0 tx=0).stop 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 msgr2=0x7efc081989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081989f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efbfc006010 tx=0x7efbfc0042f0 comp rx=0 tx=0).stop 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 shutdown_connections 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efbf00a2b80 0x7efbf00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08102ef0 0x7efc0819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:54.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc08102570 0x7efc081989f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:54.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 --2- 172.21.15.143:0/248239296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08100e20 0x7efc081984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:54.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 >> 172.21.15.143:0/248239296 conn(0x7efc080fc7d0 msgr2=0x7efc0806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:54.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 shutdown_connections 2024-09-10T10:42:54.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:54.245+0000 7efc0c8fd640 1 -- 172.21.15.143:0/248239296 wait complete. 2024-09-10T10:42:54.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:54 smithi143 ceph-mon[31057]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:42:54.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:54 smithi143 ceph-mon[31057]: pgmap v277: 97 pgs: 1 creating+activating, 40 creating+peering, 14 active+clean, 42 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:54.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:54 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/248239296' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:42:54.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:54 smithi143 ceph-mon[31057]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T10:42:54.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:54 smithi178 ceph-mon[34813]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:42:54.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:54 smithi178 ceph-mon[34813]: pgmap v277: 97 pgs: 1 creating+activating, 40 creating+peering, 14 active+clean, 42 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:54.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:54 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/248239296' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:42:54.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:54 smithi178 ceph-mon[34813]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T10:42:54.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:54 smithi144 ceph-mon[34230]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:42:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:54 smithi144 ceph-mon[34230]: pgmap v277: 97 pgs: 1 creating+activating, 40 creating+peering, 14 active+clean, 42 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:42:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:54 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/248239296' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:42:54.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:54 smithi144 ceph-mon[34230]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T10:42:54.987 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T10:42:54.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T10:42:55.236 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:55.519 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:55 smithi143 ceph-mon[31057]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:42:55.519 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:55 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/248239296' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:42:55.519 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:55 smithi143 ceph-mon[31057]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T10:42:55.520 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:55 smithi143 ceph-mon[31057]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T10:42:55.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:55 smithi178 ceph-mon[34813]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:42:55.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:55 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/248239296' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:42:55.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:55 smithi178 ceph-mon[34813]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T10:42:55.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:55 smithi178 ceph-mon[34813]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T10:42:55.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:55 smithi144 ceph-mon[34230]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:42:55.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:55 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/248239296' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:42:55.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:55 smithi144 ceph-mon[34230]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T10:42:55.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:55 smithi144 ceph-mon[34230]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 -- 172.21.15.143:0/317742915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106f10 msgr2=0x7f7a1010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/317742915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106f10 0x7f7a1010ea90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009ca0 tx=0x7f7a0802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 -- 172.21.15.143:0/317742915 shutdown_connections 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/317742915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106f10 0x7f7a1010ea90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/317742915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10102790 0x7f7a101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/317742915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10101de0 0x7f7a101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 -- 172.21.15.143:0/317742915 >> 172.21.15.143:0/317742915 conn(0x7f7a100fd8c0 msgr2=0x7f7a100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:55.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 -- 172.21.15.143:0/317742915 shutdown_connections 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.770+0000 7f7a16c9c640 1 -- 172.21.15.143:0/317742915 wait complete. 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 Processor -- start 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 -- start start 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101de0 0x7f7a10198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 0x7f7a10198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10106f10 0x7f7a1019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10111e40 con 0x7f7a10102790 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10111cc0 con 0x7f7a10101de0 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a16c9c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10111fc0 con 0x7f7a10106f10 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 0x7f7a10198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 0x7f7a10198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43330/0 (socket says 172.21.15.143:43330) 2024-09-10T10:42:55.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 -- 172.21.15.143:0/4049590342 learned_addr learned my addr 172.21.15.143:0/4049590342 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a14a38640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101de0 0x7f7a10198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a15239640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10106f10 0x7f7a1019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 -- 172.21.15.143:0/4049590342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10106f10 msgr2=0x7f7a1019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10106f10 0x7f7a1019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 -- 172.21.15.143:0/4049590342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101de0 msgr2=0x7f7a10198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101de0 0x7f7a10198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 -- 172.21.15.143:0/4049590342 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1019d3b0 con 0x7f7a10102790 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.771+0000 7f7a0ffff640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 0x7f7a10198940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00b4a0 tx=0x7f79fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.772+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc014070 con 0x7f7a10102790 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.772+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1019d6a0 con 0x7f7a10102790 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.772+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79fc010ac0 con 0x7f7a10102790 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.772+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc017c30 con 0x7f7a10102790 2024-09-10T10:42:55.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.772+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a4f80 con 0x7f7a10102790 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.773+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a101021c0 con 0x7f7a10102790 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.774+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f79fc017070 con 0x7f7a10102790 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.774+0000 7f7a0dffb640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a000a2b80 0x7f7a000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.775+0000 7f7a14a38640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a000a2b80 0x7f7a000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.775+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f79fc0ce680 con 0x7f7a10102790 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.775+0000 7f7a14a38640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a000a2b80 0x7f7a000a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a040058c0 tx=0x7f7a0400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:55.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.779+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79fc09b730 con 0x7f7a10102790 2024-09-10T10:42:55.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:55.953+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f7a1010a150 con 0x7f7a10102790 2024-09-10T10:42:56.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.260+0000 7f7a0dffb640 1 -- 172.21.15.143:0/4049590342 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) ==== 157+0+0 (secure 0 0 0) 0x7f79fc09b030 con 0x7f7a10102790 2024-09-10T10:42:56.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a000a2b80 msgr2=0x7f7a000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:56.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a000a2b80 0x7f7a000a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a040058c0 tx=0x7f7a0400a4c0 comp rx=0 tx=0).stop 2024-09-10T10:42:56.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 msgr2=0x7f7a10198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 0x7f7a10198940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00b4a0 tx=0x7f79fc00b960 comp rx=0 tx=0).stop 2024-09-10T10:42:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 shutdown_connections 2024-09-10T10:42:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7a000a2b80 0x7f7a000a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10106f10 0x7f7a1019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10102790 0x7f7a10198940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 --2- 172.21.15.143:0/4049590342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101de0 0x7f7a10198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.262+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 >> 172.21.15.143:0/4049590342 conn(0x7f7a100fd8c0 msgr2=0x7f7a100fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.263+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 shutdown_connections 2024-09-10T10:42:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:56.263+0000 7f7a16c9c640 1 -- 172.21.15.143:0/4049590342 wait complete. 2024-09-10T10:42:56.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:56 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] up:active 2024-09-10T10:42:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:56 smithi143 ceph-mon[31057]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:42:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:56 smithi143 ceph-mon[31057]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:42:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:56 smithi143 ceph-mon[31057]: pgmap v279: 97 pgs: 1 creating+activating, 40 creating+peering, 28 active+clean, 28 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T10:42:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:56 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4049590342' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T10:42:56.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:56 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] up:active 2024-09-10T10:42:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:56 smithi178 ceph-mon[34813]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:42:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:56 smithi178 ceph-mon[34813]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:42:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:56 smithi178 ceph-mon[34813]: pgmap v279: 97 pgs: 1 creating+activating, 40 creating+peering, 28 active+clean, 28 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T10:42:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:56 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4049590342' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T10:42:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:56 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] up:active 2024-09-10T10:42:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:56 smithi144 ceph-mon[34230]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:42:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:56 smithi144 ceph-mon[34230]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:42:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:56 smithi144 ceph-mon[34230]: pgmap v279: 97 pgs: 1 creating+activating, 40 creating+peering, 28 active+clean, 28 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T10:42:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:56 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4049590342' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T10:42:56.978 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:42:57.224 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:57.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:57 smithi143 ceph-mon[31057]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:42:57.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:57 smithi143 ceph-mon[31057]: Cluster is now healthy 2024-09-10T10:42:57.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:57 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4049590342' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T10:42:57.536 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:57 smithi143 ceph-mon[31057]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:42:57.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:57 smithi178 ceph-mon[34813]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:42:57.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:57 smithi178 ceph-mon[34813]: Cluster is now healthy 2024-09-10T10:42:57.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:57 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4049590342' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T10:42:57.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:57 smithi178 ceph-mon[34813]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:42:57.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:57 smithi144 ceph-mon[34230]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:42:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:57 smithi144 ceph-mon[34230]: Cluster is now healthy 2024-09-10T10:42:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:57 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4049590342' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T10:42:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:57 smithi144 ceph-mon[34230]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:42:57.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3738152406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c101df0 msgr2=0x7f2d2c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:57.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3738152406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c101df0 0x7f2d2c1021d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ca0 tx=0x7f2d1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3738152406 shutdown_connections 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3738152406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c106f20 0x7f2d2c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3738152406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3738152406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c101df0 0x7f2d2c1021d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3738152406 >> 172.21.15.143:0/3738152406 conn(0x7f2d2c0fd8f0 msgr2=0x7f2d2c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.783+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3738152406 shutdown_connections 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.784+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3738152406 wait complete. 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.784+0000 7f2d31b5d640 1 Processor -- start 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.784+0000 7f2d31b5d640 1 -- start start 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.784+0000 7f2d31b5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c101df0 0x7f2d2c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.784+0000 7f2d31b5d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d31b5d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c106f20 0x7f2d2c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d31b5d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c111e50 con 0x7f2d2c1027a0 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d31b5d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c111cd0 con 0x7f2d2c106f20 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d31b5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c111fd0 con 0x7f2d2c101df0 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48594/0 (socket says 172.21.15.143:48594) 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 -- 172.21.15.143:0/3899162071 learned_addr learned my addr 172.21.15.143:0/3899162071 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2b7fe640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c101df0 0x7f2d2c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2bfff640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c106f20 0x7f2d2c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 -- 172.21.15.143:0/3899162071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c101df0 msgr2=0x7f2d2c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c101df0 0x7f2d2c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 -- 172.21.15.143:0/3899162071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c106f20 msgr2=0x7f2d2c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c106f20 0x7f2d2c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 -- 172.21.15.143:0/3899162071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1a6980 con 0x7f2d2c1027a0 2024-09-10T10:42:57.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2bfff640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c106f20 0x7f2d2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2b7fe640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c101df0 0x7f2d2c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d2affd640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c074a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b4a0 tx=0x7f2d2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20014070 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d20010ac0 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20017c30 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c1a6c70 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.785+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c1a71b0 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.787+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c1021d0 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.788+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2d20017070 con 0x7f2d2c1027a0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.788+0000 7f2d28ff9640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2d140a2c00 0x7f2d140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.788+0000 7f2d2b7fe640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2d140a2c00 0x7f2d140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.788+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8277+0+0 (secure 0 0 0) 0x7f2d200ce750 con 0x7f2d2c1027a0 2024-09-10T10:42:57.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.789+0000 7f2d2b7fe640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2d140a2c00 0x7f2d140a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c000f80 tx=0x7f2d1c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:57.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.792+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d2009b680 con 0x7f2d2c1027a0 2024-09-10T10:42:57.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:57.962+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2d2c075a20 con 0x7f2d2c1027a0 2024-09-10T10:42:58.312 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:58 smithi143 ceph-mon[31057]: pgmap v281: 97 pgs: 23 creating+peering, 74 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-10T10:42:58.312 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:58 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3899162071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:42:58.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:58 smithi178 ceph-mon[34813]: pgmap v281: 97 pgs: 23 creating+peering, 74 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-10T10:42:58.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:58 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3899162071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:42:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:58 smithi144 ceph-mon[34230]: pgmap v281: 97 pgs: 23 creating+peering, 74 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-10T10:42:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:58 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3899162071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:42:59.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.265+0000 7f2d28ff9640 1 -- 172.21.15.143:0/3899162071 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) ==== 228+0+0 (secure 0 0 0) 0x7f2d2009af80 con 0x7f2d2c1027a0 2024-09-10T10:42:59.269 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data_ec' created 2024-09-10T10:42:59.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.268+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2d140a2c00 msgr2=0x7f2d140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:59.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.268+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2d140a2c00 0x7f2d140a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c000f80 tx=0x7f2d1c0093f0 comp rx=0 tx=0).stop 2024-09-10T10:42:59.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.268+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 msgr2=0x7f2d2c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:59.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.268+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c074a30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b4a0 tx=0x7f2d2000b960 comp rx=0 tx=0).stop 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.269+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 shutdown_connections 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.269+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2d140a2c00 0x7f2d140a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.269+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d2c106f20 0x7f2d2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.269+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d2c1027a0 0x7f2d2c074a30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.269+0000 7f2d31b5d640 1 --2- 172.21.15.143:0/3899162071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c101df0 0x7f2d2c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.269+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 >> 172.21.15.143:0/3899162071 conn(0x7f2d2c0fd8f0 msgr2=0x7f2d2c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.270+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 shutdown_connections 2024-09-10T10:42:59.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:42:59.270+0000 7f2d31b5d640 1 -- 172.21.15.143:0/3899162071 wait complete. 2024-09-10T10:42:59.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:42:59 smithi143 ceph-mon[31057]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:42:59.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:42:59 smithi178 ceph-mon[34813]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:42:59.675 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:42:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: 2024-09-10T10:42:59.488+0000 7f6fda90c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:42:59.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:42:59 smithi144 ceph-mon[34230]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:43:00.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T10:43:00.274 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:00.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:00 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3899162071' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:43:00.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:00 smithi143 ceph-mon[31057]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:43:00.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:00 smithi143 ceph-mon[31057]: pgmap v284: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-10T10:43:00.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:00 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3899162071' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:43:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:00 smithi178 ceph-mon[34813]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:43:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:00 smithi178 ceph-mon[34813]: pgmap v284: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-10T10:43:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:00 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3899162071' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:43:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:00 smithi144 ceph-mon[34230]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:43:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:00 smithi144 ceph-mon[34230]: pgmap v284: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-10T10:43:00.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 -- 172.21.15.143:0/2377883031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80103670 msgr2=0x7ffa8010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:00.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 --2- 172.21.15.143:0/2377883031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80103670 0x7ffa8010bb60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009900 tx=0x7ffa7402a480 comp rx=0 tx=0).stop 2024-09-10T10:43:00.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 -- 172.21.15.143:0/2377883031 shutdown_connections 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 --2- 172.21.15.143:0/2377883031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa8010c180 0x7ffa8010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 --2- 172.21.15.143:0/2377883031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80103670 0x7ffa8010bb60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 --2- 172.21.15.143:0/2377883031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa80102d50 0x7ffa80103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 -- 172.21.15.143:0/2377883031 >> 172.21.15.143:0/2377883031 conn(0x7ffa800fc7d0 msgr2=0x7ffa800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 -- 172.21.15.143:0/2377883031 shutdown_connections 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 -- 172.21.15.143:0/2377883031 wait complete. 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.781+0000 7ffa853e7640 1 Processor -- start 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 -- start start 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 0x7ffa801984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa80103670 0x7ffa80198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa8010c180 0x7ffa8019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:00.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111ef0 con 0x7ffa80102d50 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111d70 con 0x7ffa80103670 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa853e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80112070 con 0x7ffa8010c180 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 0x7ffa801984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 0x7ffa801984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48620/0 (socket says 172.21.15.143:48620) 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 -- 172.21.15.143:0/976473562 learned_addr learned my addr 172.21.15.143:0/976473562 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7e7fc640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa80103670 0x7ffa80198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 -- 172.21.15.143:0/976473562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa8010c180 msgr2=0x7ffa8019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa8010c180 0x7ffa8019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 -- 172.21.15.143:0/976473562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa80103670 msgr2=0x7ffa80198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa80103670 0x7ffa80198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 -- 172.21.15.143:0/976473562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8019d4a0 con 0x7ffa80102d50 2024-09-10T10:43:00.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.782+0000 7ffa7effd640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 0x7ffa801984f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000b4a0 tx=0x7ffa7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.783+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70014070 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.783+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa70010ac0 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.783+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8019d790 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.783+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70017c30 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.783+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801a5070 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.784+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa801031b0 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.785+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ffa70017070 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.786+0000 7ffa67fff640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa680a2b80 0x7ffa680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.786+0000 7ffa7e7fc640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa680a2b80 0x7ffa680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.786+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7ffa700cea60 con 0x7ffa80102d50 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.786+0000 7ffa7e7fc640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa680a2b80 0x7ffa680a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa74031440 tx=0x7ffa740095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:00.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.790+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa7009b7a0 con 0x7ffa80102d50 2024-09-10T10:43:00.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:00.956+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ffa80199850 con 0x7ffa80102d50 2024-09-10T10:43:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.276+0000 7ffa67fff640 1 -- 172.21.15.143:0/976473562 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7ffa7009b0a0 con 0x7ffa80102d50 2024-09-10T10:43:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T10:43:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa680a2b80 msgr2=0x7ffa680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa680a2b80 0x7ffa680a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa74031440 tx=0x7ffa740095f0 comp rx=0 tx=0).stop 2024-09-10T10:43:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 msgr2=0x7ffa801984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 0x7ffa801984f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000b4a0 tx=0x7ffa7000b960 comp rx=0 tx=0).stop 2024-09-10T10:43:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 shutdown_connections 2024-09-10T10:43:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa680a2b80 0x7ffa680a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa8010c180 0x7ffa8019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa80103670 0x7ffa80198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 --2- 172.21.15.143:0/976473562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa80102d50 0x7ffa801984f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 >> 172.21.15.143:0/976473562 conn(0x7ffa800fc7d0 msgr2=0x7ffa800fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 shutdown_connections 2024-09-10T10:43:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:01.279+0000 7ffa853e7640 1 -- 172.21.15.143:0/976473562 wait complete. 2024-09-10T10:43:01.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:01 smithi143 ceph-mon[31057]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:43:01.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:01 smithi143 ceph-mon[31057]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:43:01.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:01 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/976473562' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T10:43:01.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:01 smithi178 ceph-mon[34813]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:43:01.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:01 smithi178 ceph-mon[34813]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:43:01.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:01 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/976473562' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T10:43:01.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:01 smithi144 ceph-mon[34230]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:43:01.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:01 smithi144 ceph-mon[34230]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:43:01.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:01 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/976473562' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T10:43:02.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T10:43:02.374 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:02.496 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:02 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/976473562' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T10:43:02.496 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:02 smithi143 ceph-mon[31057]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:43:02.497 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:02 smithi143 ceph-mon[31057]: pgmap v287: 129 pgs: 3 creating+peering, 22 unknown, 104 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T10:43:02.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:02 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/976473562' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T10:43:02.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:02 smithi178 ceph-mon[34813]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:43:02.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:02 smithi178 ceph-mon[34813]: pgmap v287: 129 pgs: 3 creating+peering, 22 unknown, 104 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T10:43:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:02 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/976473562' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T10:43:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:02 smithi144 ceph-mon[34230]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:43:02.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:02 smithi144 ceph-mon[34230]: pgmap v287: 129 pgs: 3 creating+peering, 22 unknown, 104 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T10:43:02.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 -- 172.21.15.143:0/40973721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8801017e0 msgr2=0x7fc880101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 --2- 172.21.15.143:0/40973721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8801017e0 0x7fc880101bc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc870009900 tx=0x7fc87002a410 comp rx=0 tx=0).stop 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 -- 172.21.15.143:0/40973721 shutdown_connections 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 --2- 172.21.15.143:0/40973721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc880105e00 0x7fc88010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 --2- 172.21.15.143:0/40973721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc880102190 0x7fc8801056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 --2- 172.21.15.143:0/40973721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8801017e0 0x7fc880101bc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 -- 172.21.15.143:0/40973721 >> 172.21.15.143:0/40973721 conn(0x7fc8800fba90 msgr2=0x7fc8800fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 -- 172.21.15.143:0/40973721 shutdown_connections 2024-09-10T10:43:02.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.886+0000 7fc884c20640 1 -- 172.21.15.143:0/40973721 wait complete. 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 Processor -- start 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 -- start start 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8801017e0 0x7fc880196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 0x7fc880197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc880105e00 0x7fc880197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc880110cf0 con 0x7fc880102190 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc880110b70 con 0x7fc8801017e0 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc884c20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc880110e70 con 0x7fc880105e00 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 0x7fc880197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 0x7fc880197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48640/0 (socket says 172.21.15.143:48640) 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 -- 172.21.15.143:0/688697594 learned_addr learned my addr 172.21.15.143:0/688697594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc87e59c640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8801017e0 0x7fc880196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 -- 172.21.15.143:0/688697594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc880105e00 msgr2=0x7fc880197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc880105e00 0x7fc880197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 -- 172.21.15.143:0/688697594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8801017e0 msgr2=0x7fc880196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8801017e0 0x7fc880196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 -- 172.21.15.143:0/688697594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8801a6840 con 0x7fc880102190 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc87e59c640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8801017e0 0x7fc880196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.887+0000 7fc877fff640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 0x7fc880197510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc86800d770 tx=0x7fc86800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.888+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868017070 con 0x7fc880102190 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.888+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8680054d0 con 0x7fc880102190 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.888+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8801a6b30 con 0x7fc880102190 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.888+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004cf0 con 0x7fc880102190 2024-09-10T10:43:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.888+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801a7020 con 0x7fc880102190 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.889+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc880101bc0 con 0x7fc880102190 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.890+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc868004e90 con 0x7fc880102190 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.890+0000 7fc8777fe640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc86c0a2b80 0x7fc86c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.890+0000 7fc87e59c640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc86c0a2b80 0x7fc86c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.890+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7fc8680ced80 con 0x7fc880102190 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.891+0000 7fc87e59c640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc86c0a2b80 0x7fc86c0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc870000c00 tx=0x7fc870031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:02.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:02.895+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc86809bac0 con 0x7fc880102190 2024-09-10T10:43:03.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.098+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fc88002d7c0 con 0x7fc880102190 2024-09-10T10:43:03.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.293+0000 7fc8777fe640 1 -- 172.21.15.143:0/688697594 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fc86809b3c0 con 0x7fc880102190 2024-09-10T10:43:03.296 INFO:teuthology.orchestra.run.smithi143.stderr:added data pool 4 to fsmap 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc86c0a2b80 msgr2=0x7fc86c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc86c0a2b80 0x7fc86c0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc870000c00 tx=0x7fc870031400 comp rx=0 tx=0).stop 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 msgr2=0x7fc880197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 0x7fc880197510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc86800d770 tx=0x7fc86800dc30 comp rx=0 tx=0).stop 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 shutdown_connections 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc86c0a2b80 0x7fc86c0a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc880105e00 0x7fc880197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc880102190 0x7fc880197510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:03.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.295+0000 7fc884c20640 1 --2- 172.21.15.143:0/688697594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8801017e0 0x7fc880196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:03.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.296+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 >> 172.21.15.143:0/688697594 conn(0x7fc8800fba90 msgr2=0x7fc8800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:03.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.296+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 shutdown_connections 2024-09-10T10:43:03.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:03.296+0000 7fc884c20640 1 -- 172.21.15.143:0/688697594 wait complete. 2024-09-10T10:43:03.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/688697594' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T10:43:03.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/688697594' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T10:43:03.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/688697594' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T10:43:04.007 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:04.255 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:04.539 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:04 smithi143 ceph-mon[31057]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:43:04.539 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:04 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/688697594' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T10:43:04.539 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:04 smithi143 ceph-mon[31057]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:04.539 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:04 smithi143 ceph-mon[31057]: pgmap v289: 129 pgs: 3 creating+peering, 7 unknown, 119 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:04.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:04 smithi178 ceph-mon[34813]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:43:04.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:04 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/688697594' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T10:43:04.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:04 smithi178 ceph-mon[34813]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:04.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:04 smithi178 ceph-mon[34813]: pgmap v289: 129 pgs: 3 creating+peering, 7 unknown, 119 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:04.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:04 smithi144 ceph-mon[34230]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:43:04.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:04 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/688697594' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T10:43:04.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:04 smithi144 ceph-mon[34230]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:04.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:04 smithi144 ceph-mon[34230]: pgmap v289: 129 pgs: 3 creating+peering, 7 unknown, 119 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 -- 172.21.15.143:0/2766684816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 msgr2=0x7f980c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 --2- 172.21.15.143:0/2766684816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c110810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9804009ca0 tx=0x7f980402a950 comp rx=0 tx=0).stop 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 -- 172.21.15.143:0/2766684816 shutdown_connections 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 --2- 172.21.15.143:0/2766684816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c110810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 --2- 172.21.15.143:0/2766684816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 0x7f980c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 --2- 172.21.15.143:0/2766684816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 0x7f980c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 -- 172.21.15.143:0/2766684816 >> 172.21.15.143:0/2766684816 conn(0x7f980c0fd8c0 msgr2=0x7f980c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 -- 172.21.15.143:0/2766684816 shutdown_connections 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 -- 172.21.15.143:0/2766684816 wait complete. 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 Processor -- start 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.737+0000 7f98127c6640 1 -- start start 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f98127c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 0x7f980c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f98127c6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 0x7f980c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f98127c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f98127c6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f980c114060 con 0x7f980c10e3c0 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f98127c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f980c113ee0 con 0x7f980c074620 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f98127c6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f980c1141e0 con 0x7f980c074f40 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48670/0 (socket says 172.21.15.143:48670) 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 -- 172.21.15.143:0/3160465408 learned_addr learned my addr 172.21.15.143:0/3160465408 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:04.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f980bfff640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 0x7f980c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f980b7fe640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 0x7f980c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 -- 172.21.15.143:0/3160465408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 msgr2=0x7f980c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 0x7f980c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 -- 172.21.15.143:0/3160465408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 msgr2=0x7f980c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 0x7f980c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 -- 172.21.15.143:0/3160465408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c19f520 con 0x7f980c10e3c0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f980b7fe640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 0x7f980c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f980bfff640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 0x7f980c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.738+0000 7f9810d63640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c19ee40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9804006010 tx=0x7f980402f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.739+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804034040 con 0x7f980c10e3c0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.739+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9804003e00 con 0x7f980c10e3c0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.739+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c19f7b0 con 0x7f980c10e3c0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.739+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804045480 con 0x7f980c10e3c0 2024-09-10T10:43:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.739+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c1a70f0 con 0x7f980c10e3c0 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.740+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980c075cd0 con 0x7f980c10e3c0 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.741+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9804037020 con 0x7f980c10e3c0 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.742+0000 7f98097fa640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f97f40a2b80 0x7f97f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.742+0000 7f980bfff640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f97f40a2b80 0x7f97f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.742+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f98040f0440 con 0x7f980c10e3c0 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.742+0000 7f980bfff640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f97f40a2b80 0x7f97f40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0058c0 tx=0x7f97fc005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.746+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98040bd1f0 con 0x7f980c10e3c0 2024-09-10T10:43:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.947+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f980c109540 con 0x7f980c10e3c0 2024-09-10T10:43:04.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.948+0000 7f98097fa640 1 -- 172.21.15.143:0/3160465408 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11863 (secure 0 0 0) 0x7f98040bcaf0 con 0x7f980c10e3c0 2024-09-10T10:43:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:04.952 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"btime":"2024-09-10T10:43:03:285176+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:03.101191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24502,"qdb_cluster":[24502]},"id":1}]} 2024-09-10T10:43:04.952 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2024-09-10T10:43:04.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f97f40a2b80 msgr2=0x7f97f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f97f40a2b80 0x7f97f40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0058c0 tx=0x7f97fc005ec0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 msgr2=0x7f980c19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c19ee40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9804006010 tx=0x7f980402f640 comp rx=0 tx=0).stop 2024-09-10T10:43:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 shutdown_connections 2024-09-10T10:43:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f97f40a2b80 0x7f97f40a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f980c10e3c0 0x7f980c19ee40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f980c074f40 0x7f980c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 --2- 172.21.15.143:0/3160465408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f980c074620 0x7f980c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 >> 172.21.15.143:0/3160465408 conn(0x7f980c0fd8c0 msgr2=0x7f980c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 shutdown_connections 2024-09-10T10:43:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:04.951+0000 7f98127c6640 1 -- 172.21.15.143:0/3160465408 wait complete. 2024-09-10T10:43:05.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:05 smithi143 ceph-mon[31057]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T10:43:05.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:05 smithi143 ceph-mon[31057]: Cluster is now healthy 2024-09-10T10:43:05.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:05 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3160465408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:05.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:05 smithi178 ceph-mon[34813]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T10:43:05.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:05 smithi178 ceph-mon[34813]: Cluster is now healthy 2024-09-10T10:43:05.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:05 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3160465408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:05.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:05 smithi144 ceph-mon[34230]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T10:43:05.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:05 smithi144 ceph-mon[34230]: Cluster is now healthy 2024-09-10T10:43:05.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:05 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3160465408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:05.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:06.023 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:06.432 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:06 smithi143 ceph-mon[31057]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 -- 172.21.15.143:0/3172613264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 msgr2=0x7f6668101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 --2- 172.21.15.143:0/3172613264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668101200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6660009900 tx=0x7f666002a480 comp rx=0 tx=0).stop 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 -- 172.21.15.143:0/3172613264 shutdown_connections 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 --2- 172.21.15.143:0/3172613264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 0x7f666810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 --2- 172.21.15.143:0/3172613264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6668102570 0x7f66681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 --2- 172.21.15.143:0/3172613264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668101200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 -- 172.21.15.143:0/3172613264 >> 172.21.15.143:0/3172613264 conn(0x7f66680fc7d0 msgr2=0x7f66680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 -- 172.21.15.143:0/3172613264 shutdown_connections 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.508+0000 7f666d2b6640 1 -- 172.21.15.143:0/3172613264 wait complete. 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 Processor -- start 2024-09-10T10:43:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 -- start start 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6668102570 0x7f6668198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 0x7f666819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6668110ed0 con 0x7f6668100e20 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6668110d50 con 0x7f6668102ef0 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f666d2b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6668111050 con 0x7f6668102570 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f6666ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f6666ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40512/0 (socket says 172.21.15.143:40512) 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.509+0000 7f6666ffd640 1 -- 172.21.15.143:0/1073985423 learned_addr learned my addr 172.21.15.143:0/1073985423 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f66677fe640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 0x7f666819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f66667fc640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6668102570 0x7f6668198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f6666ffd640 1 -- 172.21.15.143:0/1073985423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6668102570 msgr2=0x7f6668198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f6666ffd640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6668102570 0x7f6668198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f6666ffd640 1 -- 172.21.15.143:0/1073985423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 msgr2=0x7f666819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f6666ffd640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 0x7f666819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f6666ffd640 1 -- 172.21.15.143:0/1073985423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666819d4f0 con 0x7f6668100e20 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f66677fe640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 0x7f666819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f6666ffd640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668198540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6660006010 tx=0x7f66600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666003f070 con 0x7f6668100e20 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6660003aa0 con 0x7f6668100e20 2024-09-10T10:43:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666819d780 con 0x7f6668100e20 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660035030 con 0x7f6668100e20 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.510+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66681a50c0 con 0x7f6668100e20 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.511+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66681021b0 con 0x7f6668100e20 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.512+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6660042070 con 0x7f6668100e20 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.513+0000 7f664ffff640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66500a2b80 0x7f66500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.513+0000 7f66667fc640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66500a2b80 0x7f66500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.513+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f66600f0320 con 0x7f6668100e20 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.513+0000 7f66667fc640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66500a2b80 0x7f66500a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66681000f0 tx=0x7f665c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:06.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.516+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66600bd040 con 0x7f6668100e20 2024-09-10T10:43:06.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:06 smithi178 ceph-mon[34813]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:06.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:06 smithi144 ceph-mon[34230]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:06.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.685+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6668101200 con 0x7f6668100e20 2024-09-10T10:43:06.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.687+0000 7f664ffff640 1 -- 172.21.15.143:0/1073985423 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7f6668101200 con 0x7f6668100e20 2024-09-10T10:43:06.689 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:06.689 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66500a2b80 msgr2=0x7f66500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66500a2b80 0x7f66500a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66681000f0 tx=0x7f665c008040 comp rx=0 tx=0).stop 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 msgr2=0x7f6668198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668198540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6660006010 tx=0x7f66600042f0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 shutdown_connections 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66500a2b80 0x7f66500a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6668102ef0 0x7f666819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6668102570 0x7f6668198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 --2- 172.21.15.143:0/1073985423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6668100e20 0x7f6668198540 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 >> 172.21.15.143:0/1073985423 conn(0x7f66680fc7d0 msgr2=0x7f666806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 shutdown_connections 2024-09-10T10:43:06.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:06.689+0000 7f666d2b6640 1 -- 172.21.15.143:0/1073985423 wait complete. 2024-09-10T10:43:07.464 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:07 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1073985423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:07.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:07.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:07 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1073985423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:07.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:07 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1073985423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:08.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- 172.21.15.143:0/1434144254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102570 msgr2=0x7fc66c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- 172.21.15.143:0/1434144254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102570 0x7fc66c1029b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc660009900 tx=0x7fc66002a480 comp rx=0 tx=0).stop 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- 172.21.15.143:0/1434144254 shutdown_connections 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- 172.21.15.143:0/1434144254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc66c102ef0 0x7fc66c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- 172.21.15.143:0/1434144254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102570 0x7fc66c1029b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- 172.21.15.143:0/1434144254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc66c100e20 0x7fc66c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- 172.21.15.143:0/1434144254 >> 172.21.15.143:0/1434144254 conn(0x7fc66c0fc7d0 msgr2=0x7fc66c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- 172.21.15.143:0/1434144254 shutdown_connections 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- 172.21.15.143:0/1434144254 wait complete. 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 Processor -- start 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- start start 2024-09-10T10:43:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc66c100e20 0x7fc66c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc66c102570 0x7fc66c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 0x7fc66c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c110e30 con 0x7fc66c102ef0 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c110cb0 con 0x7fc66c102570 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.228+0000 7fc674378640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c110fb0 con 0x7fc66c100e20 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 0x7fc66c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 0x7fc66c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40524/0 (socket says 172.21.15.143:40524) 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 -- 172.21.15.143:0/1827529210 learned_addr learned my addr 172.21.15.143:0/1827529210 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672114640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc66c100e20 0x7fc66c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc671913640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc66c102570 0x7fc66c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:08.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 -- 172.21.15.143:0/1827529210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc66c100e20 msgr2=0x7fc66c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc66c100e20 0x7fc66c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 -- 172.21.15.143:0/1827529210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc66c102570 msgr2=0x7fc66c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc66c102570 0x7fc66c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 -- 172.21.15.143:0/1827529210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c19c2c0 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc672915640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 0x7fc66c19bbe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00cad0 tx=0x7fc65c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00ae20 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc65c005020 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c19c5b0 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00f6b0 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.229+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c1a3e90 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.230+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c101200 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.232+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc65c0051d0 con 0x7fc66c102ef0 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.232+0000 7fc66b7fe640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc6540a2c00 0x7fc6540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.232+0000 7fc672114640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc6540a2c00 0x7fc6540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.232+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc65c0cf1a0 con 0x7fc66c102ef0 2024-09-10T10:43:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.233+0000 7fc672114640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc6540a2c00 0x7fc6540a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc66400bbe0 tx=0x7fc66400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.237+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc65c014030 con 0x7fc66c102ef0 2024-09-10T10:43:08.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.410+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc66c109090 con 0x7fc66c102ef0 2024-09-10T10:43:08.412 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:08 smithi143 ceph-mon[31057]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:08.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.412+0000 7fc66b7fe640 1 -- 172.21.15.143:0/1827529210 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7fc65c09b870 con 0x7fc66c102ef0 2024-09-10T10:43:08.413 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:08.413 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.414+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc6540a2c00 msgr2=0x7fc6540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.414+0000 7fc674378640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc6540a2c00 0x7fc6540a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc66400bbe0 tx=0x7fc66400b390 comp rx=0 tx=0).stop 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.414+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 msgr2=0x7fc66c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.414+0000 7fc674378640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 0x7fc66c19bbe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00cad0 tx=0x7fc65c00cf90 comp rx=0 tx=0).stop 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 shutdown_connections 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc6540a2c00 0x7fc6540a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc66c102ef0 0x7fc66c19bbe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc66c102570 0x7fc66c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 --2- 172.21.15.143:0/1827529210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc66c100e20 0x7fc66c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:08.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 >> 172.21.15.143:0/1827529210 conn(0x7fc66c0fc7d0 msgr2=0x7fc66c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:08.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 shutdown_connections 2024-09-10T10:43:08.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:08.415+0000 7fc674378640 1 -- 172.21.15.143:0/1827529210 wait complete. 2024-09-10T10:43:08.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:08 smithi178 ceph-mon[34813]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:08.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:08 smithi144 ceph-mon[34230]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:09.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:09.404 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:09.526 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:09 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1827529210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:09.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:09 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1827529210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:09.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:09 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1827529210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:09.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 -- 172.21.15.143:0/2983382088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a30109b90 msgr2=0x7f9a301158a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:09.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 --2- 172.21.15.143:0/2983382088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a30109b90 0x7f9a301158a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800aa10 tx=0x7f9a2802b720 comp rx=0 tx=0).stop 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 -- 172.21.15.143:0/2983382088 shutdown_connections 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 --2- 172.21.15.143:0/2983382088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a30109b90 0x7f9a301158a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 --2- 172.21.15.143:0/2983382088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a301091f0 0x7f9a30109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 --2- 172.21.15.143:0/2983382088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a3010eb10 0x7f9a3010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 -- 172.21.15.143:0/2983382088 >> 172.21.15.143:0/2983382088 conn(0x7f9a3006e2d0 msgr2=0x7f9a3006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 -- 172.21.15.143:0/2983382088 shutdown_connections 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.979+0000 7f9a372b6640 1 -- 172.21.15.143:0/2983382088 wait complete. 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 Processor -- start 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 -- start start 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a301091f0 0x7f9a301a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a30109b90 0x7f9a301a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 0x7f9a301a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:09.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30118470 con 0x7f9a3010eb10 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a301182f0 con 0x7f9a301091f0 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a372b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a301185f0 con 0x7f9a30109b90 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a35853640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 0x7f9a301a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a35853640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 0x7f9a301a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40538/0 (socket says 172.21.15.143:40538) 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a35853640 1 -- 172.21.15.143:0/3221335304 learned_addr learned my addr 172.21.15.143:0/3221335304 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.980+0000 7f9a34851640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a30109b90 0x7f9a301a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35052640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a301091f0 0x7f9a301a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35853640 1 -- 172.21.15.143:0/3221335304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a30109b90 msgr2=0x7f9a301a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35853640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a30109b90 0x7f9a301a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35853640 1 -- 172.21.15.143:0/3221335304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a301091f0 msgr2=0x7f9a301a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35853640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a301091f0 0x7f9a301a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35853640 1 -- 172.21.15.143:0/3221335304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a301a10a0 con 0x7f9a3010eb10 2024-09-10T10:43:09.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35052640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a301091f0 0x7f9a301a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a35853640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 0x7f9a301a0970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800a340 tx=0x7f9a28004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2803f070 con 0x7f9a3010eb10 2024-09-10T10:43:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a28003ca0 con 0x7f9a3010eb10 2024-09-10T10:43:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a301a1330 con 0x7f9a3010eb10 2024-09-10T10:43:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28036040 con 0x7f9a3010eb10 2024-09-10T10:43:09.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.981+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a301ad680 con 0x7f9a3010eb10 2024-09-10T10:43:09.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.982+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a30109650 con 0x7f9a3010eb10 2024-09-10T10:43:09.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.983+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9a28042070 con 0x7f9a3010eb10 2024-09-10T10:43:09.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.984+0000 7f9a267fc640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:09.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.984+0000 7f9a35052640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:09.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.984+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f9a280efe50 con 0x7f9a3010eb10 2024-09-10T10:43:09.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.985+0000 7f9a35052640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a301a1ab0 tx=0x7f9a2c009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:09.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:09.988+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a280bcb10 con 0x7f9a3010eb10 2024-09-10T10:43:10.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.185+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a3002d7c0 con 0x7f9a3010eb10 2024-09-10T10:43:10.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.186+0000 7f9a267fc640 1 -- 172.21.15.143:0/3221335304 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11863 (secure 0 0 0) 0x7f9a280bc410 con 0x7f9a3010eb10 2024-09-10T10:43:10.187 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:10.188 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"btime":"2024-09-10T10:43:03:285176+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:03.101191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24502,"qdb_cluster":[24502]},"id":1}]} 2024-09-10T10:43:10.188 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2024-09-10T10:43:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9a1c0a2c00 msgr2=0x7f9a1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a301a1ab0 tx=0x7f9a2c009570 comp rx=0 tx=0).stop 2024-09-10T10:43:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 msgr2=0x7f9a301a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:10.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 0x7f9a301a0970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800a340 tx=0x7f9a28004250 comp rx=0 tx=0).stop 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 shutdown_connections 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9a1c0a2c00 0x7f9a1c0a50c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a3010eb10 0x7f9a301a0970 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a30109b90 0x7f9a301a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 --2- 172.21.15.143:0/3221335304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a301091f0 0x7f9a301a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 >> 172.21.15.143:0/3221335304 conn(0x7f9a3006e2d0 msgr2=0x7f9a300725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 shutdown_connections 2024-09-10T10:43:10.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:10.188+0000 7f9a372b6640 1 -- 172.21.15.143:0/3221335304 wait complete. 2024-09-10T10:43:10.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:10 smithi143 ceph-mon[31057]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:10.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:10 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3221335304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:10.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:10 smithi178 ceph-mon[34813]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:10.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:10 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3221335304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:10.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:10 smithi144 ceph-mon[34230]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:10.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:10 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3221335304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:11.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:11.282 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:11.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 -- 172.21.15.143:0/3427542024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154102ef0 msgr2=0x7fc15410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 --2- 172.21.15.143:0/3427542024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154102ef0 0x7fc15410da40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009ca0 tx=0x7fc14c02a950 comp rx=0 tx=0).stop 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 -- 172.21.15.143:0/3427542024 shutdown_connections 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 --2- 172.21.15.143:0/3427542024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154102ef0 0x7fc15410da40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 --2- 172.21.15.143:0/3427542024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 0x7fc1541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 --2- 172.21.15.143:0/3427542024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc154100e20 0x7fc154101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 -- 172.21.15.143:0/3427542024 >> 172.21.15.143:0/3427542024 conn(0x7fc1540fc7d0 msgr2=0x7fc1540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 -- 172.21.15.143:0/3427542024 shutdown_connections 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 -- 172.21.15.143:0/3427542024 wait complete. 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 Processor -- start 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.791+0000 7fc15bac1640 1 -- start start 2024-09-10T10:43:11.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 0x7fc1541993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 0x7fc1541998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc154102ef0 0x7fc1541a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc154110ed0 con 0x7fc154100e20 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc154110d50 con 0x7fc154102570 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc154111050 con 0x7fc154102ef0 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 0x7fc1541993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 0x7fc1541993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40558/0 (socket says 172.21.15.143:40558) 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 -- 172.21.15.143:0/1538283804 learned_addr learned my addr 172.21.15.143:0/1538283804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:11.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15905c640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 0x7fc1541998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15a05e640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc154102ef0 0x7fc1541a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 -- 172.21.15.143:0/1538283804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc154102ef0 msgr2=0x7fc1541a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc154102ef0 0x7fc1541a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 -- 172.21.15.143:0/1538283804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 msgr2=0x7fc1541998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 0x7fc1541998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 -- 172.21.15.143:0/1538283804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1541a4960 con 0x7fc154100e20 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15905c640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 0x7fc1541998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:11.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15985d640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 0x7fc1541993a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc15000b4a0 tx=0x7fc15000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150014070 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc150010ac0 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1541a4b30 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150017c30 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.792+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1541a5040 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.793+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc154101280 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.795+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc150017070 con 0x7fc154100e20 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.795+0000 7fc14affd640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1400a2c00 0x7fc1400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.795+0000 7fc15905c640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1400a2c00 0x7fc1400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:11.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.796+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc1500ceab0 con 0x7fc154100e20 2024-09-10T10:43:11.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.796+0000 7fc15905c640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1400a2c00 0x7fc1400a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc144006fd0 tx=0x7fc144008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:11.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.799+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc15009b7d0 con 0x7fc154100e20 2024-09-10T10:43:11.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.963+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc15406a560 con 0x7fc154100e20 2024-09-10T10:43:11.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.965+0000 7fc14affd640 1 -- 172.21.15.143:0/1538283804 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7fc15406a560 con 0x7fc154100e20 2024-09-10T10:43:11.967 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:11.967 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:11.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1400a2c00 msgr2=0x7fc1400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:11.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1400a2c00 0x7fc1400a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc144006fd0 tx=0x7fc144008040 comp rx=0 tx=0).stop 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 msgr2=0x7fc1541993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 0x7fc1541993a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc15000b4a0 tx=0x7fc15000b960 comp rx=0 tx=0).stop 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 shutdown_connections 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1400a2c00 0x7fc1400a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc154102ef0 0x7fc1541a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc154102570 0x7fc1541998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 --2- 172.21.15.143:0/1538283804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154100e20 0x7fc1541993a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 >> 172.21.15.143:0/1538283804 conn(0x7fc1540fc7d0 msgr2=0x7fc1540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 shutdown_connections 2024-09-10T10:43:11.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:11.968+0000 7fc15bac1640 1 -- 172.21.15.143:0/1538283804 wait complete. 2024-09-10T10:43:12.752 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:12 smithi143 ceph-mon[31057]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:12.752 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:12 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1538283804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:12.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:42:53.956292+0000', 'modified': '2024-09-10T10:43:03.101191+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.178:6839/774166191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 774166191}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 774166191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24502, 'qdb_cluster': [24502]} 2024-09-10T10:43:12.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:43:12.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T10:43:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:12 smithi178 ceph-mon[34813]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:12.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:12 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1538283804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:12 smithi144 ceph-mon[34230]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:12 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1538283804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:13.039 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- 172.21.15.143:0/4111652107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 msgr2=0x7f99d4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- 172.21.15.143:0/4111652107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d4106e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009930 tx=0x7f99c402a420 comp rx=0 tx=0).stop 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- 172.21.15.143:0/4111652107 shutdown_connections 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- 172.21.15.143:0/4111652107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 0x7f99d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- 172.21.15.143:0/4111652107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d4100a80 0x7f99d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- 172.21.15.143:0/4111652107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d4106e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- 172.21.15.143:0/4111652107 >> 172.21.15.143:0/4111652107 conn(0x7f99d40fc7a0 msgr2=0x7f99d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- 172.21.15.143:0/4111652107 shutdown_connections 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- 172.21.15.143:0/4111652107 wait complete. 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 Processor -- start 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- start start 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d4100a80 0x7f99d40fce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 0x7f99d40fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d40fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4110d50 con 0x7f99d4106a80 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4110bd0 con 0x7f99d4101420 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.502+0000 7f99da703640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4110ed0 con 0x7f99d4100a80 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d40fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d40fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40582/0 (socket says 172.21.15.143:40582) 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 -- 172.21.15.143:0/3714164024 learned_addr learned my addr 172.21.15.143:0/3714164024 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d4100a80 0x7f99d40fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 0x7f99d40fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 -- 172.21.15.143:0/3714164024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d4100a80 msgr2=0x7f99d40fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d4100a80 0x7f99d40fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 -- 172.21.15.143:0/3714164024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 msgr2=0x7f99d40fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 0x7f99d40fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 -- 172.21.15.143:0/3714164024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d41a6900 con 0x7f99d4106a80 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 0x7f99d40fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d8ca0640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d40fd900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00eaa0 tx=0x7f99cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d1ffb640 1 -- 172.21.15.143:0/3714164024 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00cdb0 con 0x7f99d4106a80 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d41a6bf0 con 0x7f99d4106a80 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d1ffb640 1 -- 172.21.15.143:0/3714164024 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99cc005020 con 0x7f99d4106a80 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99d1ffb640 1 -- 172.21.15.143:0/3714164024 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0116d0 con 0x7f99d4106a80 2024-09-10T10:43:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.503+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d41a70e0 con 0x7f99d4106a80 2024-09-10T10:43:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.506+0000 7f99d1ffb640 1 -- 172.21.15.143:0/3714164024 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f99cc0051d0 con 0x7f99d4106a80 2024-09-10T10:43:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.506+0000 7f99dcadc740 1 Processor -- start 2024-09-10T10:43:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.506+0000 7f99dcadc740 1 -- start start 2024-09-10T10:43:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 0x55eab08b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08b5bb0 0x55eab08b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99dcadc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eab0811090 con 0x55eab08ab2f0 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99dcadc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eab07b8a70 con 0x55eab08b5bb0 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99dcadc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eab080d050 con 0x55eab08aa9d0 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 0x55eab08b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 0x55eab08b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40590/0 (socket says 172.21.15.143:40590) 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 -- 172.21.15.143:0/2696680687 learned_addr learned my addr 172.21.15.143:0/2696680687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99cbfff640 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99cbfff640 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08b5bb0 0x55eab08b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d1ffb640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99c00a2b80 0x7f99c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 -- 172.21.15.143:0/2696680687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 msgr2=0x55eab08aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 -- 172.21.15.143:0/2696680687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08b5bb0 msgr2=0x55eab08b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08b5bb0 0x55eab08b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 -- 172.21.15.143:0/2696680687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab07c5130 con 0x55eab08ab2f0 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d8ca0640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99c00a2b80 0x7f99c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d1ffb640 1 -- 172.21.15.143:0/3714164024 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f99cc0cf480 con 0x7f99d4106a80 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.507+0000 7f99d3fff640 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 0x55eab08b5670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009930 tx=0x7f99c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ba7d436a047940 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.508+0000 7f99ca7fc640 1 -- 172.21.15.143:0/2696680687 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4040070 con 0x55eab08ab2f0 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.508+0000 7f99ca7fc640 1 -- 172.21.15.143:0/2696680687 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99c40316a0 con 0x55eab08ab2f0 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.508+0000 7f99ca7fc640 1 -- 172.21.15.143:0/2696680687 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4035040 con 0x55eab08ab2f0 2024-09-10T10:43:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.508+0000 7f99d8ca0640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99c00a2b80 0x7f99c00a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99cc09b7a0 tx=0x7f99cc066bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.508+0000 7f99dcadc740 1 -- 172.21.15.143:0/2696680687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 msgr2=0x55eab08b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.508+0000 7f99dcadc740 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 0x55eab08b5670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009930 tx=0x7f99c4005fd0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- 172.21.15.143:0/2696680687 shutdown_connections 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08b5bb0 0x55eab08b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08ab2f0 0x55eab08b5670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 --2- 172.21.15.143:0/2696680687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- 172.21.15.143:0/2696680687 >> 172.21.15.143:0/2696680687 conn(0x55eab08a6f60 msgr2=0x55eab08a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- 172.21.15.143:0/2696680687 shutdown_connections 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- 172.21.15.143:0/2696680687 wait complete. 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- start start 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 Processor -- start 2024-09-10T10:43:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eab08b8780 con 0x55eab08b5bb0 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eab07cbf20 con 0x55eab08ab2f0 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eab08b8900 con 0x55eab08aa9d0 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.509+0000 7f99dcadc740 10 client.0 _ll_register_callbacks cb 0x55eab0795030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40594/0 (socket says 172.21.15.143:40594) 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 -- 172.21.15.143:0/3869132735 learned_addr learned my addr 172.21.15.143:0/3869132735 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 -- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 msgr2=0x55eab08b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 -- 172.21.15.143:0/3869132735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 msgr2=0x55eab08b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 -- 172.21.15.143:0/3869132735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab05822b0 con 0x55eab08b5bb0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99bc019e60 tx=0x7f99bc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c697ddc61e65c96e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99b37fe640 1 -- 172.21.15.143:0/3869132735 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc01f070 con 0x55eab08b5bb0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99b37fe640 1 -- 172.21.15.143:0/3869132735 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f99bc0165e0 con 0x55eab08b5bb0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c40316a0 con 0x55eab08b5bb0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99b37fe640 1 -- 172.21.15.143:0/3869132735 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f99bc0059c0 con 0x55eab08b5bb0 2024-09-10T10:43:13.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99dcadc740 10 client.14703 fetch_fsmap learned FSMap version 17 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55eab08c3f90 con 0x55eab08b5bb0 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.510+0000 7f99b37fe640 1 -- 172.21.15.143:0/3869132735 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99c40316a0 con 0x55eab08b5bb0 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99b37fe640 1 -- 172.21.15.143:0/3869132735 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f99bc022020 con 0x55eab08b5bb0 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99dcadc740 10 client.14703 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99dcadc740 10 client.14703 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99dcadc740 20 client.14703 populate_metadata read hostname 'smithi143' 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab08c6680 0x55eab08c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99dcadc740 4 client.14703 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eab081e600 con 0x55eab08c6680 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab08c6680 0x55eab08c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab08c6680 0x55eab08c7af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61948a953812999 server_cookie=dcc351b9434aff0f in_seq=0 out_seq=0 2024-09-10T10:43:13.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.511+0000 7f99b37fe640 10 client.14703 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:43:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99b37fe640 1 -- 172.21.15.143:0/3869132735 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eab081e600 con 0x55eab08c6680 2024-09-10T10:43:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99b37fe640 10 client.14703 handle_command_reply: tid=0 2024-09-10T10:43:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 client.14703 shutdown 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 msgr2=0x55eab08b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99bc019e60 tx=0x7f99bc0047d0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 shutdown_connections 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b2b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab08c6680 0x55eab08c7af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3869132735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 >> 172.21.15.143:0/3869132735 conn(0x55eab08a6f60 msgr2=0x55eab08acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.512+0000 7f99b37fe640 0 client.14703 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.513+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 shutdown_connections 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.513+0000 7f99dcadc740 1 -- 172.21.15.143:0/3869132735 wait complete. 2024-09-10T10:43:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.513+0000 7f99dcadc740 20 client.14703 trim_cache size 0 max 16384 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 Processor -- start 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 -- start start 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 0x55eab08c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08b5bb0 0x55eab08b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eab08c5e30 con 0x55eab08aa9d0 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eab08c6500 con 0x55eab08ab2f0 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99dcadc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eab08c6bd0 con 0x55eab08b5bb0 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 0x55eab08c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 0x55eab08c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40608/0 (socket says 172.21.15.143:40608) 2024-09-10T10:43:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.530+0000 7f99cbfff640 1 -- 172.21.15.143:0/1766528277 learned_addr learned my addr 172.21.15.143:0/1766528277 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99cbfff640 1 -- 172.21.15.143:0/1766528277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08b5bb0 msgr2=0x55eab08b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99cbfff640 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08b5bb0 0x55eab08b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99cbfff640 1 -- 172.21.15.143:0/1766528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 msgr2=0x55eab08b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99cbfff640 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99cbfff640 1 -- 172.21.15.143:0/1766528277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab08a6d90 con 0x55eab08aa9d0 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99cbfff640 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 0x55eab08c6d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99bc023880 tx=0x7f99bc00a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc1ffa5abe8f7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99c9ffb640 1 -- 172.21.15.143:0/1766528277 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc01f040 con 0x55eab08aa9d0 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99c9ffb640 1 -- 172.21.15.143:0/1766528277 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99bc031070 con 0x55eab08aa9d0 2024-09-10T10:43:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99c9ffb640 1 -- 172.21.15.143:0/1766528277 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc023900 con 0x55eab08aa9d0 2024-09-10T10:43:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99dcadc740 1 -- 172.21.15.143:0/1766528277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 msgr2=0x55eab08c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99dcadc740 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 0x55eab08c6d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99bc023880 tx=0x7f99bc00a6c0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99dcadc740 1 -- 172.21.15.143:0/1766528277 shutdown_connections 2024-09-10T10:43:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99dcadc740 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08b5bb0 0x55eab08b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99dcadc740 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.531+0000 7f99dcadc740 1 --2- 172.21.15.143:0/1766528277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08aa9d0 0x55eab08c6d50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- 172.21.15.143:0/1766528277 >> 172.21.15.143:0/1766528277 conn(0x55eab08a6f60 msgr2=0x55eab08b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- 172.21.15.143:0/1766528277 shutdown_connections 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- 172.21.15.143:0/1766528277 wait complete. 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- start start 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 Processor -- start 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eab08c7970 con 0x55eab08b5bb0 2024-09-10T10:43:13.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eab08c72a0 con 0x55eab08ab2f0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eab08c8040 con 0x55eab08aa9d0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99dcadc740 10 client.0 _ll_register_callbacks cb 0x7f99bc019cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40612/0 (socket says 172.21.15.143:40612) 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.532+0000 7f99cbfff640 1 -- 172.21.15.143:0/3382591852 learned_addr learned my addr 172.21.15.143:0/3382591852 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 -- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 msgr2=0x55eab08b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 -- 172.21.15.143:0/3382591852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 msgr2=0x55eab08b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 -- 172.21.15.143:0/3382591852 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab08cedc0 con 0x55eab08b5bb0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99cbfff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b6e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99bc023610 tx=0x7f99bc017610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e3477ccd4859a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99b37fe640 1 -- 172.21.15.143:0/3382591852 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc02c650 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99b37fe640 1 -- 172.21.15.143:0/3382591852 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f99bc02c7f0 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99bc031070 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99b37fe640 1 -- 172.21.15.143:0/3382591852 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f99bc019630 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99dcadc740 10 client.14709 fetch_fsmap learned FSMap version 17 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55eab08d0040 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.533+0000 7f99b37fe640 1 -- 172.21.15.143:0/3382591852 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99bc031070 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99b37fe640 1 -- 172.21.15.143:0/3382591852 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f99bc022070 con 0x55eab08b5bb0 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99dcadc740 10 client.14709 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T10:43:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99dcadc740 10 client.14709 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99dcadc740 20 client.14709 populate_metadata read hostname 'smithi143' 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab0934920 0x55eab0933630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99dcadc740 4 client.14709 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "status"}) -- 0x55eab0818b10 con 0x55eab0934920 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab0934920 0x55eab0933630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99d3fff640 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab0934920 0x55eab0933630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec789adc123a096 server_cookie=e2790f1f69bef666 in_seq=0 out_seq=0 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99b37fe640 10 client.14709 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99b37fe640 1 -- 172.21.15.143:0/3382591852 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eab0818b10 con 0x55eab0934920 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.534+0000 7f99b37fe640 10 client.14709 handle_command_reply: tid=0 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 client.14709 shutdown 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 msgr2=0x55eab08b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b6e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99bc023610 tx=0x7f99bc017610 comp rx=0 tx=0).stop 2024-09-10T10:43:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 shutdown_connections 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eab08b5bb0 0x55eab08b6e20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eab08ab2f0 0x55eab08b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eab0934920 0x55eab0933630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 --2- 172.21.15.143:0/3382591852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eab08aa9d0 0x55eab08b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 >> 172.21.15.143:0/3382591852 conn(0x55eab08a6f60 msgr2=0x55eab08b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99b37fe640 0 client.14709 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 shutdown_connections 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 1 -- 172.21.15.143:0/3382591852 wait complete. 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99dcadc740 20 client.14709 trim_cache size 0 max 16384 2024-09-10T10:43:13.542 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24502, 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 19.290358378000001, 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 17, 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 87, 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 39.438310688999998 2024-09-10T10:43:13.543 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:43:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99c00a2b80 msgr2=0x7f99c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99c00a2b80 0x7f99c00a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99cc09b7a0 tx=0x7f99cc066bc0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 msgr2=0x7f99d40fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d40fd900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00eaa0 tx=0x7f99cc00ef60 comp rx=0 tx=0).stop 2024-09-10T10:43:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 shutdown_connections 2024-09-10T10:43:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99c00a2b80 0x7f99c00a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99d4106a80 0x7f99d40fd900 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d4101420 0x7f99d40fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.535+0000 7f99da703640 1 --2- 172.21.15.143:0/3714164024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99d4100a80 0x7f99d40fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.536+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 >> 172.21.15.143:0/3714164024 conn(0x7f99d40fc7a0 msgr2=0x7f99d406ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.536+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 shutdown_connections 2024-09-10T10:43:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:13.536+0000 7f99da703640 1 -- 172.21.15.143:0/3714164024 wait complete. 2024-09-10T10:43:14.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:13 smithi178 ceph-mon[34813]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:14.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:13 smithi144 ceph-mon[34230]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:14.270 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:13 smithi143 ceph-mon[31057]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:14.298 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:43:14.299 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:14.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T10:43:14.545 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:15.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 -- 172.21.15.143:0/1129620164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0106f10 msgr2=0x7f3df010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:15.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/1129620164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0106f10 0x7f3df010ea90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009ca0 tx=0x7f3de802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 -- 172.21.15.143:0/1129620164 shutdown_connections 2024-09-10T10:43:15.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/1129620164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0106f10 0x7f3df010ea90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/1129620164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102790 0x7f3df01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/1129620164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 0x7f3df01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 -- 172.21.15.143:0/1129620164 >> 172.21.15.143:0/1129620164 conn(0x7f3df00fd8c0 msgr2=0x7f3df00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 -- 172.21.15.143:0/1129620164 shutdown_connections 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.035+0000 7f3df7bc3640 1 -- 172.21.15.143:0/1129620164 wait complete. 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 Processor -- start 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 -- start start 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 0x7f3df0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 0x7f3df0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0106f10 0x7f3df019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0111e20 con 0x7f3df0102790 2024-09-10T10:43:15.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0111ca0 con 0x7f3df0106f10 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df7bc3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0111fa0 con 0x7f3df0101de0 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 0x7f3df0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 0x7f3df0198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40620/0 (socket says 172.21.15.143:40620) 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 -- 172.21.15.143:0/3003796553 learned_addr learned my addr 172.21.15.143:0/3003796553 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df595f640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 0x7f3df0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df6160640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0106f10 0x7f3df019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 -- 172.21.15.143:0/3003796553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 msgr2=0x7f3df0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 0x7f3df0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 -- 172.21.15.143:0/3003796553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0106f10 msgr2=0x7f3df019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0106f10 0x7f3df019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 -- 172.21.15.143:0/3003796553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df019d2f0 con 0x7f3df0102790 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df595f640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 0x7f3df0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:15.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.036+0000 7f3df515e640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 0x7f3df0198880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00d950 tx=0x7f3dec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.037+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec017070 con 0x7f3df0102790 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.037+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dec009c30 con 0x7f3df0102790 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.037+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00f620 con 0x7f3df0102790 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.037+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df019d5e0 con 0x7f3df0102790 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.037+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a4ec0 con 0x7f3df0102790 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.038+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df01021c0 con 0x7f3df0102790 2024-09-10T10:43:15.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.039+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3dec0109a0 con 0x7f3df0102790 2024-09-10T10:43:15.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.040+0000 7f3de2ffd640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3ddc0a2c00 0x7f3ddc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:15.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.040+0000 7f3df595f640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3ddc0a2c00 0x7f3ddc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:15.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.040+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3dec0cee50 con 0x7f3df0102790 2024-09-10T10:43:15.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.040+0000 7f3df595f640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3ddc0a2c00 0x7f3ddc0a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de40058c0 tx=0x7f3de4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:15.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.043+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3dec009de0 con 0x7f3df0102790 2024-09-10T10:43:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.245+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3df002d7c0 con 0x7f3df0102790 2024-09-10T10:43:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.245+0000 7f3de2ffd640 1 -- 172.21.15.143:0/3003796553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f3dec09bbf0 con 0x7f3df0102790 2024-09-10T10:43:15.247 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:43:15.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.247+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3ddc0a2c00 msgr2=0x7f3ddc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.247+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3ddc0a2c00 0x7f3ddc0a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de40058c0 tx=0x7f3de4005850 comp rx=0 tx=0).stop 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.247+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 msgr2=0x7f3df0198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.247+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 0x7f3df0198880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00d950 tx=0x7f3dec00de10 comp rx=0 tx=0).stop 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 shutdown_connections 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3ddc0a2c00 0x7f3ddc0a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0106f10 0x7f3df019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3df0102790 0x7f3df0198880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 --2- 172.21.15.143:0/3003796553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df0101de0 0x7f3df0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 >> 172.21.15.143:0/3003796553 conn(0x7f3df00fd8c0 msgr2=0x7f3df00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 shutdown_connections 2024-09-10T10:43:15.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:15.248+0000 7f3df7bc3640 1 -- 172.21.15.143:0/3003796553 wait complete. 2024-09-10T10:43:15.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:15 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3003796553' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:43:15.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:15 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3003796553' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:43:15.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:15 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3003796553' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:43:15.975 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:43:15.975 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:43:15.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-09-10T10:43:15.975 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T10:43:15.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:15.976 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:43:15.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:43:15.976 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:43:15.976 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T10:43:15.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:15.976 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-09-10T10:43:15.993 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42295sec preferred_lft 42295sec 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link noprefixroute 2024-09-10T10:43:15.994 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:43:15.995 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:43:15.995 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:43:16.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:16.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:16.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:16.199 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:43:16.225 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-09-10T10:43:16.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:16.226 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-09-10T10:43:16.295 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-09-10T10:43:16.295 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:43:16.295 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:43:16.295 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:43:16.295 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:43:16.296 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:16.296 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:43:16.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:43:16.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:43:16.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:43:16.296 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:43:16.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:16.497 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:16 smithi143 ceph-mon[31057]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:16.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:16.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:16.593 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-09-10T10:43:16.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:16.609 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-09-10T10:43:16.664 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:16.664 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:43:16.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:16.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T10:43:16.665 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:43:16.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:16 smithi178 ceph-mon[34813]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:16.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:16 smithi144 ceph-mon[34230]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:16.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:16.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:16.782 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T10:43:16.782 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:16.782 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:43:16.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T10:43:16.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:43:16.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T10:43:16.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T10:43:16.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T10:43:16.783 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:43:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:16.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:16.905 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:16.905 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:43:16.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-09-10T10:43:16.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:43:16.906 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:43:16.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:16.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:16.981 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T10:43:16.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:43:16.981 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:16.996 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T10:43:16.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:43:16.997 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:17.052 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-09-10T10:43:17.117 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:43:17.174 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-09-10T10:43:17.175 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-09-10T10:43:17.176 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-09-10T10:43:17.176 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-09-10T10:43:17.176 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-09-10T10:43:17.176 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/selinux 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-09-10T10:43:17.177 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/9e89ccedca6f47585dae63a59261f6f61733668e8115ab8b06300dd6f90553cf/merged 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d902d0e612869f50dcaf70af0a29ce6a962ce9b6ad9a487a1ea15be5a8ccc46d/merged 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/0 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/3630a32f2bfa1ab51d64cf01622e1bb6f9fe829548d8ce08923612bf11c650d2/merged 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/a6f1284f5096bb30a5dcbb209d96bd0c5c09bcd9a096633246e0cbe1284bf312/merged 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d085932431a90f4493f59f8d45bdfc2b4934e5e2172a80d753307d4ace13960c/merged 2024-09-10T10:43:17.178 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/5bfc047991fa2d929c5acb5f4cdf099ee6e60da7bd43e0c36380c4097934c550/merged 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/91133b17bd7ed747cbdd8fa21d95faf3ab947dd653bac10bdd0c015f538690d5/merged 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/663c3868f6ed2a49572cd5f7a9d9c56e72e624e74b1aeae55e74d952c1f3f204/merged 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d932de3595f6eecb2d9a4b27dbcb2e366196f358310da0e6d403cae1e3436cab/merged 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/0e903cd12a8e8e5e0917bc6f3258f0a62de2f70a6fb9b13edea3e889bb460d2e/merged 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:17.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:17.179 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-09-10T10:43:17.230 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T10:43:17.231 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T10:43:17.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-09-10T10:43:17.298 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:43:17.355 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-09-10T10:43:17.355 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-09-10T10:43:17.355 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-09-10T10:43:17.355 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-09-10T10:43:17.355 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-09-10T10:43:17.355 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/selinux 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:43:17.356 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/9e89ccedca6f47585dae63a59261f6f61733668e8115ab8b06300dd6f90553cf/merged 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d902d0e612869f50dcaf70af0a29ce6a962ce9b6ad9a487a1ea15be5a8ccc46d/merged 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/0 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/3630a32f2bfa1ab51d64cf01622e1bb6f9fe829548d8ce08923612bf11c650d2/merged 2024-09-10T10:43:17.357 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/a6f1284f5096bb30a5dcbb209d96bd0c5c09bcd9a096633246e0cbe1284bf312/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d085932431a90f4493f59f8d45bdfc2b4934e5e2172a80d753307d4ace13960c/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/5bfc047991fa2d929c5acb5f4cdf099ee6e60da7bd43e0c36380c4097934c550/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/91133b17bd7ed747cbdd8fa21d95faf3ab947dd653bac10bdd0c015f538690d5/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/663c3868f6ed2a49572cd5f7a9d9c56e72e624e74b1aeae55e74d952c1f3f204/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d932de3595f6eecb2d9a4b27dbcb2e366196f358310da0e6d403cae1e3436cab/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/0e903cd12a8e8e5e0917bc6f3258f0a62de2f70a6fb9b13edea3e889bb460d2e/merged 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:17.358 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:17.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:17.359 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-09-10T10:43:17.383 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi143.stderr:2024-09-10T10:43:17.381+0000 7f8ed5121480 -1 init, newargv = 0x558cc7d948f0 newargc=15 2024-09-10T10:43:17.383 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi143.stderr:ceph-fuse[75626]: starting ceph client 2024-09-10T10:43:17.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi143.stderr:ceph-fuse[75626]: starting fuse 2024-09-10T10:43:17.398 INFO:teuthology.orchestra.run.smithi143.stdout:120 2024-09-10T10:43:17.398 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T10:43:17.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-09-10T10:43:17.398 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-09-10T10:43:17.399 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.admin")) 2024-09-10T10:43:17.400 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-09-10T10:43:17.519 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.admin.75626.asok 2024-09-10T10:43:17.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:17.529 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75626.asok 2024-09-10T10:43:17.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:17.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75626.asok status 2024-09-10T10:43:17.663 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "admin", 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "75626", 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-09-10T10:43:17.664 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 14721, 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 14721 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:43:17.665 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1704723530 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1704723530 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.14721 192.168.0.1:0/1704723530", 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/1704723530", 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 17, 2024-09-10T10:43:17.666 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 87, 2024-09-10T10:43:17.667 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-09-10T10:43:17.667 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-09-10T10:43:17.667 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-09-10T10:43:17.667 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:43:17.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:17.672 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-09-10T10:43:17.672 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:17.688 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-09-10T10:43:17.689 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:17.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:17.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:17.755 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T10:43:17.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:43:17.866 INFO:teuthology.orchestra.run.smithi143.stdout:# file: . 2024-09-10T10:43:17.866 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T10:43:17.866 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:17.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:17.872 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:43:17.872 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:43:17.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:43:18.176 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:18.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:18 smithi143 ceph-mon[31057]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:18.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 -- 172.21.15.143:0/3065746856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102570 msgr2=0x7fa7641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3065746856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102570 0x7fa7641029b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa758009ca0 tx=0x7fa75802a950 comp rx=0 tx=0).stop 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 -- 172.21.15.143:0/3065746856 shutdown_connections 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3065746856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa764102ef0 0x7fa76410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3065746856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102570 0x7fa7641029b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3065746856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa764100e20 0x7fa764101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 -- 172.21.15.143:0/3065746856 >> 172.21.15.143:0/3065746856 conn(0x7fa7640fc7d0 msgr2=0x7fa7640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:18.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 -- 172.21.15.143:0/3065746856 shutdown_connections 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 -- 172.21.15.143:0/3065746856 wait complete. 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 Processor -- start 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.684+0000 7fa76b36c640 1 -- start start 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa764100e20 0x7fa76419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa764102570 0x7fa76419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 0x7fa76419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa76410fd30 con 0x7fa764102ef0 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa76410fbb0 con 0x7fa764100e20 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa76410feb0 con 0x7fa764102570 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 0x7fa76419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 0x7fa76419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40028/0 (socket says 172.21.15.143:40028) 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 -- 172.21.15.143:0/3927804009 learned_addr learned my addr 172.21.15.143:0/3927804009 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769108640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa764100e20 0x7fa76419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa768907640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa764102570 0x7fa76419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 -- 172.21.15.143:0/3927804009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa764102570 msgr2=0x7fa76419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa764102570 0x7fa76419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 -- 172.21.15.143:0/3927804009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa764100e20 msgr2=0x7fa76419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa764100e20 0x7fa76419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 -- 172.21.15.143:0/3927804009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76419f5d0 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa769909640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 0x7fa76419eef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa75400c970 tx=0x7fa75400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754007bb0 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa754007d50 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76419f8c0 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754004e10 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.685+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7641a71a0 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.686+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa764101200 con 0x7fa764102ef0 2024-09-10T10:43:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.688+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa754005000 con 0x7fa764102ef0 2024-09-10T10:43:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.688+0000 7fa7627fc640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa74c0a2b70 0x7fa74c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.688+0000 7fa769108640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa74c0a2b70 0x7fa74c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.688+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fa7540cf1d0 con 0x7fa764102ef0 2024-09-10T10:43:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.689+0000 7fa769108640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa74c0a2b70 0x7fa74c0a5030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0058c0 tx=0x7fa75c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.692+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa754014030 con 0x7fa764102ef0 2024-09-10T10:43:18.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.868+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa76419b980 con 0x7fa764102ef0 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.869+0000 7fa7627fc640 1 -- 172.21.15.143:0/3927804009 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+601 (secure 0 0 0) 0x7fa76419b980 con 0x7fa764102ef0 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3629809110 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/212886119 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3153560299 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:18.870 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:18.871 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:18.871 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/1093984961 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:18.871 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/2707450669 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:18.871 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3503275671 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:listed 10 entries 2024-09-10T10:43:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa74c0a2b70 msgr2=0x7fa74c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa74c0a2b70 0x7fa74c0a5030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0058c0 tx=0x7fa75c00a2d0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 msgr2=0x7fa76419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 0x7fa76419eef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa75400c970 tx=0x7fa75400ce30 comp rx=0 tx=0).stop 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 shutdown_connections 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa74c0a2b70 0x7fa74c0a5030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa764102ef0 0x7fa76419eef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa764102570 0x7fa76419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 --2- 172.21.15.143:0/3927804009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa764100e20 0x7fa76419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 >> 172.21.15.143:0/3927804009 conn(0x7fa7640fc7d0 msgr2=0x7fa7640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 shutdown_connections 2024-09-10T10:43:18.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:18.871+0000 7fa76b36c640 1 -- 172.21.15.143:0/3927804009 wait complete. 2024-09-10T10:43:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:18 smithi178 ceph-mon[34813]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:18.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:18 smithi144 ceph-mon[34230]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:19.626 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:19 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3927804009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:43:19.656 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:43:19.656 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:43:19.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:43:19.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:19 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3927804009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:43:19.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:19 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3927804009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:43:19.960 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 -- 172.21.15.143:0/2864879688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 msgr2=0x7f4b0c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2864879688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c1031c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ca0 tx=0x7f4b0802a950 comp rx=0 tx=0).stop 2024-09-10T10:43:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 -- 172.21.15.143:0/2864879688 shutdown_connections 2024-09-10T10:43:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2864879688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 0x7f4b0c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2864879688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 0x7f4b0c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2864879688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c1031c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 -- 172.21.15.143:0/2864879688 >> 172.21.15.143:0/2864879688 conn(0x7f4b0c0fc7a0 msgr2=0x7f4b0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 -- 172.21.15.143:0/2864879688 shutdown_connections 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.425+0000 7f4b1347a640 1 -- 172.21.15.143:0/2864879688 wait complete. 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 Processor -- start 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 -- start start 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 0x7f4b0c100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 0x7f4b0c1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c111e60 con 0x7f4b0c102de0 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c111ce0 con 0x7f4b0c103700 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b1347a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c111fe0 con 0x7f4b0c10c180 2024-09-10T10:43:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40038/0 (socket says 172.21.15.143:40038) 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 -- 172.21.15.143:0/2984116914 learned_addr learned my addr 172.21.15.143:0/2984116914 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 -- 172.21.15.143:0/2984116914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 msgr2=0x7f4b0c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11a17640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 0x7f4b0c1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b10a15640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 0x7f4b0c100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 0x7f4b0c1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 -- 172.21.15.143:0/2984116914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 msgr2=0x7f4b0c100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 0x7f4b0c100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11216640 1 -- 172.21.15.143:0/2984116914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c1a68e0 con 0x7f4b0c102de0 2024-09-10T10:43:20.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b11a17640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 0x7f4b0c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.426+0000 7f4b10a15640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 0x7f4b0c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.427+0000 7f4b11216640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c100a40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b08005cc0 tx=0x7f4b0802f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.427+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08034040 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.427+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c1a6b70 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.427+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b08003e40 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.427+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08045480 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.427+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c1a70a0 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.428+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c1031c0 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.429+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4b08037020 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.429+0000 7f4b027fc640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4af80a2b80 0x7f4af80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.430+0000 7f4b10a15640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4af80a2b80 0x7f4af80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.430+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f4b080f0540 con 0x7f4b0c102de0 2024-09-10T10:43:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.430+0000 7f4b10a15640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4af80a2b80 0x7f4af80a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:20.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.434+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b080bd260 con 0x7f4b0c102de0 2024-09-10T10:43:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.613+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b0c1072a0 con 0x7f4b0c102de0 2024-09-10T10:43:20.614 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:20 smithi143 ceph-mon[31057]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.613+0000 7f4b027fc640 1 -- 172.21.15.143:0/2984116914 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+601 (secure 0 0 0) 0x7f4b080bcb60 con 0x7f4b0c102de0 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3629809110 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/212886119 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3153560299 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/1093984961 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:20.615 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/2707450669 2024-09-11T10:35:59.338289+0000 2024-09-10T10:43:20.616 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3503275671 2024-09-11T10:35:32.440973+0000 2024-09-10T10:43:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:listed 10 entries 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.615+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4af80a2b80 msgr2=0x7f4af80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.615+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4af80a2b80 0x7f4af80a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc008040 comp rx=0 tx=0).stop 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 msgr2=0x7f4b0c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c100a40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b08005cc0 tx=0x7f4b0802f650 comp rx=0 tx=0).stop 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 shutdown_connections 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4af80a2b80 0x7f4af80a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c10c180 0x7f4b0c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b0c103700 0x7f4b0c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 --2- 172.21.15.143:0/2984116914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c102de0 0x7f4b0c100a40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:20.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 >> 172.21.15.143:0/2984116914 conn(0x7f4b0c0fc7a0 msgr2=0x7f4b0c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:20.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 shutdown_connections 2024-09-10T10:43:20.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:20.616+0000 7f4b1347a640 1 -- 172.21.15.143:0/2984116914 wait complete. 2024-09-10T10:43:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:20 smithi178 ceph-mon[34813]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:20 smithi144 ceph-mon[34230]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:21.438 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-09-10T10:43:21.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:21.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:21.466 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi143.stderr:ceph-fuse[75626]: fuse finished with error 0 and tester_r 0 2024-09-10T10:43:21.467 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:43:21.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T10:43:21.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:21.566 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T10:43:21.594 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:21.657 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:21 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2984116914' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:43:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:21.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:21 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2984116914' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:43:21.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:21 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2984116914' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:43:22.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2313279705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f64102d30 msgr2=0x7f8f64103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2313279705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f64102d30 0x7f8f64103110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009cd0 tx=0x7f8f5402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2313279705 shutdown_connections 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2313279705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641082a0 0x7f8f6410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2313279705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f641036e0 0x7f8f64107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2313279705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f64102d30 0x7f8f64103110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2313279705 >> 172.21.15.143:0/2313279705 conn(0x7f8f640fc7d0 msgr2=0x7f8f640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2313279705 shutdown_connections 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2313279705 wait complete. 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.368+0000 7f8f6b6fc640 1 Processor -- start 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 -- start start 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64102d30 0x7f8f64100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 0x7f8f64100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641082a0 0x7f8f64101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64111f50 con 0x7f8f641036e0 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64111dd0 con 0x7f8f64102d30 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f6b6fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641120d0 con 0x7f8f641082a0 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 0x7f8f64100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 0x7f8f64100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40058/0 (socket says 172.21.15.143:40058) 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 -- 172.21.15.143:0/2722371882 learned_addr learned my addr 172.21.15.143:0/2722371882 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f69498640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64102d30 0x7f8f64100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f69c99640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641082a0 0x7f8f64101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 -- 172.21.15.143:0/2722371882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641082a0 msgr2=0x7f8f64101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641082a0 0x7f8f64101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 -- 172.21.15.143:0/2722371882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64102d30 msgr2=0x7f8f64100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64102d30 0x7f8f64100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.369+0000 7f8f68c97640 1 -- 172.21.15.143:0/2722371882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f641a69b0 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.370+0000 7f8f68c97640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 0x7f8f64100ed0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800e9f0 tx=0x7f8f5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.370+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800ccc0 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.370+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f580108e0 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.370+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f641a6ca0 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.370+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f580187e0 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.370+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641a7190 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.371+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f64103110 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.372+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8f58010a90 con 0x7f8f641036e0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.372+0000 7f8f627fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.373+0000 7f8f69498640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.373+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f8f580cf6f0 con 0x7f8f641036e0 2024-09-10T10:43:22.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.373+0000 7f8f69498640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000f80 tx=0x7f8f540093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:22.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.377+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f5809c410 con 0x7f8f641036e0 2024-09-10T10:43:22.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.546+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f6410a2b0 con 0x7f8f641036e0 2024-09-10T10:43:22.547 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:22 smithi143 ceph-mon[31057]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:22.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.547+0000 7f8f627fc640 1 -- 172.21.15.143:0/2722371882 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7f8f5809bd10 con 0x7f8f641036e0 2024-09-10T10:43:22.549 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:22.549 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f4c0a2b80 msgr2=0x7f8f4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000f80 tx=0x7f8f540093f0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 msgr2=0x7f8f64100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 0x7f8f64100ed0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800e9f0 tx=0x7f8f5800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 shutdown_connections 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641082a0 0x7f8f64101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f641036e0 0x7f8f64100ed0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 --2- 172.21.15.143:0/2722371882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64102d30 0x7f8f64100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 >> 172.21.15.143:0/2722371882 conn(0x7f8f640fc7d0 msgr2=0x7f8f64105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:22.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 shutdown_connections 2024-09-10T10:43:22.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:22.550+0000 7f8f6b6fc640 1 -- 172.21.15.143:0/2722371882 wait complete. 2024-09-10T10:43:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:22 smithi178 ceph-mon[34813]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:22 smithi144 ceph-mon[34230]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:23.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:23.666 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:23.787 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:23 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2722371882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:23 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2722371882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:23.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:23 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2722371882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:24.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 -- 172.21.15.143:0/2268766047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c100e20 msgr2=0x7f5d3c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/2268766047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c100e20 0x7f5d3c101200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009900 tx=0x7f5d2402a480 comp rx=0 tx=0).stop 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 -- 172.21.15.143:0/2268766047 shutdown_connections 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/2268766047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/2268766047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d3c102570 0x7f5d3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/2268766047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c100e20 0x7f5d3c101200 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 -- 172.21.15.143:0/2268766047 >> 172.21.15.143:0/2268766047 conn(0x7f5d3c0fc7d0 msgr2=0x7f5d3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 -- 172.21.15.143:0/2268766047 shutdown_connections 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.150+0000 7f5d40fe0640 1 -- 172.21.15.143:0/2268766047 wait complete. 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 Processor -- start 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 -- start start 2024-09-10T10:43:24.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c110e50 con 0x7f5d3c102570 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c110cd0 con 0x7f5d3c100e20 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d40fe0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c110fd0 con 0x7f5d3c102ef0 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40072/0 (socket says 172.21.15.143:40072) 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 -- 172.21.15.143:0/751860631 learned_addr learned my addr 172.21.15.143:0/751860631 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 -- 172.21.15.143:0/751860631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d3c102ef0 msgr2=0x7f5d3c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 -- 172.21.15.143:0/751860631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d3c100e20 msgr2=0x7f5d3c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.151+0000 7f5d39d9b640 1 -- 172.21.15.143:0/751860631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a68d0 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.152+0000 7f5d39d9b640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c920 tx=0x7f5d2c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.152+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c007c70 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.152+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d2c007e10 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.152+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a6bc0 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.152+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c018820 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.152+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a7100 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.153+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c101200 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.155+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5d2c0189c0 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.155+0000 7f5d337fe640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5d280a2b80 0x7f5d280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.156+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f5d2c0cf730 con 0x7f5d3c102570 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.156+0000 7f5d3a59c640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5d280a2b80 0x7f5d280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.156+0000 7f5d3a59c640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5d280a2b80 0x7f5d280a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d240096c0 tx=0x7f5d24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.158+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d2c014030 con 0x7f5d3c102570 2024-09-10T10:43:24.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.329+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d3c0fe1b0 con 0x7f5d3c102570 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.330+0000 7f5d337fe640 1 -- 172.21.15.143:0/751860631 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7f5d2c09c4e0 con 0x7f5d3c102570 2024-09-10T10:43:24.332 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:24.332 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5d280a2b80 msgr2=0x7f5d280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5d280a2b80 0x7f5d280a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d240096c0 tx=0x7f5d24000bc0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 msgr2=0x7f5d3c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c920 tx=0x7f5d2c00cde0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 shutdown_connections 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5d280a2b80 0x7f5d280a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 --2- 172.21.15.143:0/751860631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 >> 172.21.15.143:0/751860631 conn(0x7f5d3c0fc7d0 msgr2=0x7f5d3c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 shutdown_connections 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:24.333+0000 7f5d40fe0640 1 -- 172.21.15.143:0/751860631 wait complete. 2024-09-10T10:43:24.705 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:24 smithi143 ceph-mon[31057]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:24.705 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:24 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/751860631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:24 smithi178 ceph-mon[34813]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:24 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/751860631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:24 smithi144 ceph-mon[34230]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:24 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/751860631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:25.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:43:25.359 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:25.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 -- 172.21.15.143:0/380177956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c101de0 msgr2=0x7ff51c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 --2- 172.21.15.143:0/380177956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c101de0 0x7ff51c1021c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009900 tx=0x7ff50c02a410 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 -- 172.21.15.143:0/380177956 shutdown_connections 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 --2- 172.21.15.143:0/380177956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c106f10 0x7ff51c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 --2- 172.21.15.143:0/380177956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff51c102790 0x7ff51c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 --2- 172.21.15.143:0/380177956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c101de0 0x7ff51c1021c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 -- 172.21.15.143:0/380177956 >> 172.21.15.143:0/380177956 conn(0x7ff51c0fd8c0 msgr2=0x7ff51c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 -- 172.21.15.143:0/380177956 shutdown_connections 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.869+0000 7ff521b44640 1 -- 172.21.15.143:0/380177956 wait complete. 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 Processor -- start 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 -- start start 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c101de0 0x7ff51c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 0x7ff51c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff51c106f10 0x7ff51c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111e40 con 0x7ff51c102790 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111cc0 con 0x7ff51c106f10 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff521b44640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111fc0 con 0x7ff51c101de0 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 0x7ff51c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 0x7ff51c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40094/0 (socket says 172.21.15.143:40094) 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 -- 172.21.15.143:0/813415147 learned_addr learned my addr 172.21.15.143:0/813415147 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51bfff640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff51c106f10 0x7ff51c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51b7fe640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c101de0 0x7ff51c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 -- 172.21.15.143:0/813415147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c101de0 msgr2=0x7ff51c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c101de0 0x7ff51c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 -- 172.21.15.143:0/813415147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff51c106f10 msgr2=0x7ff51c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff51c106f10 0x7ff51c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51affd640 1 -- 172.21.15.143:0/813415147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c19d2f0 con 0x7ff51c102790 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.870+0000 7ff51b7fe640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c101de0 0x7ff51c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.871+0000 7ff51affd640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 0x7ff51c198880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c920 tx=0x7ff51000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.871+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510007c70 con 0x7ff51c102790 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.871+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c19d5e0 con 0x7ff51c102790 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.871+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff510007e10 con 0x7ff51c102790 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.871+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510018820 con 0x7ff51c102790 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.871+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1a4ec0 con 0x7ff51c102790 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.872+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c1021c0 con 0x7ff51c102790 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.873+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff5100189c0 con 0x7ff51c102790 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.873+0000 7ff518ff9640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff5040a2b80 0x7ff5040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.874+0000 7ff51b7fe640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff5040a2b80 0x7ff5040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.874+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7ff5100cf3b0 con 0x7ff51c102790 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.874+0000 7ff51b7fe640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff5040a2b80 0x7ff5040a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff50c031440 tx=0x7ff50c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:25.878+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff510014030 con 0x7ff51c102790 2024-09-10T10:43:26.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.082+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff51c02d7c0 con 0x7ff51c102790 2024-09-10T10:43:26.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.503+0000 7ff518ff9640 1 -- 172.21.15.143:0/813415147 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff51009c0d0 con 0x7ff51c102790 2024-09-10T10:43:26.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.505+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff5040a2b80 msgr2=0x7ff5040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.505+0000 7ff521b44640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff5040a2b80 0x7ff5040a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff50c031440 tx=0x7ff50c0095f0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.505+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 msgr2=0x7ff51c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.505+0000 7ff521b44640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 0x7ff51c198880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c920 tx=0x7ff51000cde0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 shutdown_connections 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff5040a2b80 0x7ff5040a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff51c106f10 0x7ff51c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff51c102790 0x7ff51c198880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 --2- 172.21.15.143:0/813415147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff51c101de0 0x7ff51c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 >> 172.21.15.143:0/813415147 conn(0x7ff51c0fd8c0 msgr2=0x7ff51c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 shutdown_connections 2024-09-10T10:43:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:26.506+0000 7ff521b44640 1 -- 172.21.15.143:0/813415147 wait complete. 2024-09-10T10:43:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:26 smithi143 ceph-mon[31057]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:26 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/813415147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:43:26.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:26 smithi178 ceph-mon[34813]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:26.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:26 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/813415147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:43:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:26 smithi144 ceph-mon[34230]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:26 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/813415147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:43:27.334 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-10T10:43:27.334 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T10:43:27.586 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:27.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:27 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/813415147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:43:27.707 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:27 smithi143 ceph-mon[31057]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:27 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/813415147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:43:27.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:27 smithi178 ceph-mon[34813]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:27 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/813415147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:43:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:27 smithi144 ceph-mon[34230]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:28.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 -- 172.21.15.143:0/818850627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 msgr2=0x7f14e8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:28.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/818850627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8103110 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009930 tx=0x7f14d802a420 comp rx=0 tx=0).stop 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 -- 172.21.15.143:0/818850627 shutdown_connections 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/818850627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 0x7f14e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/818850627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e81036e0 0x7f14e8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/818850627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8103110 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 -- 172.21.15.143:0/818850627 >> 172.21.15.143:0/818850627 conn(0x7f14e80fc7d0 msgr2=0x7f14e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 -- 172.21.15.143:0/818850627 shutdown_connections 2024-09-10T10:43:28.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.091+0000 7f14ecd7b640 1 -- 172.21.15.143:0/818850627 wait complete. 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 Processor -- start 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 -- start start 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e81036e0 0x7f14e8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 0x7f14e819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8111f50 con 0x7f14e8102d30 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8111dd0 con 0x7f14e81082a0 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ecd7b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81120d0 con 0x7f14e81036e0 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14e659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14e659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48410/0 (socket says 172.21.15.143:48410) 2024-09-10T10:43:28.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14e659c640 1 -- 172.21.15.143:0/2813865102 learned_addr learned my addr 172.21.15.143:0/2813865102 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14e6d9d640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 0x7f14e819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.092+0000 7f14ddd9b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e81036e0 0x7f14e8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e659c640 1 -- 172.21.15.143:0/2813865102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e81036e0 msgr2=0x7f14e8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e659c640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e81036e0 0x7f14e8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e659c640 1 -- 172.21.15.143:0/2813865102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 msgr2=0x7f14e819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e659c640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 0x7f14e819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e659c640 1 -- 172.21.15.143:0/2813865102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e819d4e0 con 0x7f14e8102d30 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e6d9d640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 0x7f14e819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14e659c640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8198530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004770 tx=0x7f14d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d803f070 con 0x7f14e8102d30 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e819d770 con 0x7f14e8102d30 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d803b460 con 0x7f14e8102d30 2024-09-10T10:43:28.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8035070 con 0x7f14e8102d30 2024-09-10T10:43:28.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.093+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a50b0 con 0x7f14e8102d30 2024-09-10T10:43:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.094+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e8103110 con 0x7f14e8102d30 2024-09-10T10:43:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.095+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f14d803b610 con 0x7f14e8102d30 2024-09-10T10:43:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.096+0000 7f14dffff640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.096+0000 7f14ddd9b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.096+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f14d80ef9c0 con 0x7f14e8102d30 2024-09-10T10:43:28.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.097+0000 7f14ddd9b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005fd0 tx=0x7f14d0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:28.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.103+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14d80bc6e0 con 0x7f14e8102d30 2024-09-10T10:43:28.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.310+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f14e8199890 con 0x7f14e8102d30 2024-09-10T10:43:28.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.509+0000 7f14dffff640 1 -- 172.21.15.143:0/2813865102 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f14d80bbfe0 con 0x7f14e8102d30 2024-09-10T10:43:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 msgr2=0x7f14d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005fd0 tx=0x7f14d0005850 comp rx=0 tx=0).stop 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 msgr2=0x7f14e8198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8198530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004770 tx=0x7f14d80042f0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 shutdown_connections 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14e81082a0 0x7f14e819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e81036e0 0x7f14e8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 --2- 172.21.15.143:0/2813865102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e8102d30 0x7f14e8198530 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.512+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 >> 172.21.15.143:0/2813865102 conn(0x7f14e80fc7d0 msgr2=0x7f14e8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.513+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 shutdown_connections 2024-09-10T10:43:28.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:28.513+0000 7f14ecd7b640 1 -- 172.21.15.143:0/2813865102 wait complete. 2024-09-10T10:43:28.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:28 smithi143 ceph-mon[31057]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:28.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:28 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2813865102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T10:43:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:28 smithi178 ceph-mon[34813]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:28 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2813865102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T10:43:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:28 smithi144 ceph-mon[34230]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:28 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2813865102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T10:43:29.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T10:43:29.647 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:29.768 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:29 smithi143 ceph-mon[31057]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T10:43:29.768 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:29 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2813865102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T10:43:29.768 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:29 smithi143 ceph-mon[31057]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T10:43:29.768 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:29 smithi143 ceph-mon[31057]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:29.768 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:29 smithi143 ceph-mon[31057]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-10T10:43:29.768 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:29 smithi143 ceph-mon[31057]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T10:43:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:29 smithi178 ceph-mon[34813]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T10:43:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:29 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2813865102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T10:43:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:29 smithi178 ceph-mon[34813]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T10:43:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:29 smithi178 ceph-mon[34813]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:29 smithi178 ceph-mon[34813]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-10T10:43:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:29 smithi178 ceph-mon[34813]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T10:43:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:29 smithi144 ceph-mon[34230]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T10:43:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:29 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2813865102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T10:43:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:29 smithi144 ceph-mon[34230]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T10:43:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:29 smithi144 ceph-mon[34230]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:43:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:29 smithi144 ceph-mon[34230]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-10T10:43:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:29 smithi144 ceph-mon[34230]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 -- 172.21.15.143:0/3122656487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102ef0 msgr2=0x7fa25810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3122656487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102ef0 0x7fa25810da40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa250009ca0 tx=0x7fa25002a950 comp rx=0 tx=0).stop 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 -- 172.21.15.143:0/3122656487 shutdown_connections 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3122656487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102ef0 0x7fa25810da40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3122656487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa258102570 0x7fa2581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3122656487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 0x7fa258101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 -- 172.21.15.143:0/3122656487 >> 172.21.15.143:0/3122656487 conn(0x7fa2580fc7d0 msgr2=0x7fa2580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 -- 172.21.15.143:0/3122656487 shutdown_connections 2024-09-10T10:43:30.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 -- 172.21.15.143:0/3122656487 wait complete. 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.126+0000 7fa25e77f640 1 Processor -- start 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 -- start start 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 0x7fa258199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 0x7fa2581997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa258102ef0 0x7fa2581a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110e50 con 0x7fa258102570 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110cd0 con 0x7fa258102ef0 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110fd0 con 0x7fa258100e20 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 0x7fa2581997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 0x7fa2581997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48426/0 (socket says 172.21.15.143:48426) 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 -- 172.21.15.143:0/3518666859 learned_addr learned my addr 172.21.15.143:0/3518666859 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25cd1c640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa258102ef0 0x7fa2581a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:30.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa257fff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 0x7fa258199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 -- 172.21.15.143:0/3518666859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 msgr2=0x7fa258199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 0x7fa258199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 -- 172.21.15.143:0/3518666859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa258102ef0 msgr2=0x7fa2581a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa258102ef0 0x7fa2581a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 -- 172.21.15.143:0/3518666859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2581a4820 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa257fff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 0x7fa258199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa24ffff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 0x7fa2581997a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa24000d950 tx=0x7fa24000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240017070 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa240009c30 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000f620 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.127+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2581a49f0 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.128+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a4f00 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.128+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa258101200 con 0x7fa258102570 2024-09-10T10:43:30.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.130+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa2400109a0 con 0x7fa258102570 2024-09-10T10:43:30.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.130+0000 7fa255ffb640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2440a2b80 0x7fa2440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:30.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.130+0000 7fa257fff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2440a2b80 0x7fa2440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:30.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.130+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fa2400cee90 con 0x7fa258102570 2024-09-10T10:43:30.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.131+0000 7fa257fff640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2440a2b80 0x7fa2440a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa248009930 tx=0x7fa248008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:30.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.133+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa240009de0 con 0x7fa258102570 2024-09-10T10:43:30.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.337+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa25806a560 con 0x7fa258102570 2024-09-10T10:43:30.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.526+0000 7fa255ffb640 1 -- 172.21.15.143:0/3518666859 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fa25806a560 con 0x7fa258102570 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.528+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2440a2b80 msgr2=0x7fa2440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.528+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2440a2b80 0x7fa2440a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa248009930 tx=0x7fa248008040 comp rx=0 tx=0).stop 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.528+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 msgr2=0x7fa2581997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.528+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 0x7fa2581997a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa24000d950 tx=0x7fa24000de10 comp rx=0 tx=0).stop 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 shutdown_connections 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2440a2b80 0x7fa2440a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa258102ef0 0x7fa2581a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa258102570 0x7fa2581997a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 --2- 172.21.15.143:0/3518666859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa258100e20 0x7fa258199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 >> 172.21.15.143:0/3518666859 conn(0x7fa2580fc7d0 msgr2=0x7fa2580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 shutdown_connections 2024-09-10T10:43:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:30.529+0000 7fa25e77f640 1 -- 172.21.15.143:0/3518666859 wait complete. 2024-09-10T10:43:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] up:active 2024-09-10T10:43:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T10:43:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-10T10:43:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-10T10:43:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T10:43:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:30 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3518666859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:43:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] up:active 2024-09-10T10:43:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T10:43:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-10T10:43:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-10T10:43:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T10:43:30.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:30 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3518666859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] up:active 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T10:43:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:30 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3518666859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:43:31.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T10:43:31.636 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:31.756 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:31 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3518666859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:43:31.756 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:31 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] up:active 2024-09-10T10:43:31.756 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:31 smithi143 ceph-mon[31057]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T10:43:31.756 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:31 smithi143 ceph-mon[31057]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-10T10:43:31.757 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:31 smithi143 ceph-mon[31057]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-10T10:43:31.757 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:31 smithi143 ceph-mon[31057]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-10T10:43:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:31 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3518666859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:43:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:31 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] up:active 2024-09-10T10:43:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:31 smithi178 ceph-mon[34813]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T10:43:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:31 smithi178 ceph-mon[34813]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-10T10:43:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:31 smithi178 ceph-mon[34813]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-10T10:43:31.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:31 smithi178 ceph-mon[34813]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-10T10:43:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:31 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3518666859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:43:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:31 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] up:active 2024-09-10T10:43:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:31 smithi144 ceph-mon[34230]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T10:43:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:31 smithi144 ceph-mon[34230]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-10T10:43:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:31 smithi144 ceph-mon[34230]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-10T10:43:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:31 smithi144 ceph-mon[34230]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-10T10:43:32.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 -- 172.21.15.143:0/2325845478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f810c170 msgr2=0x7fc7f810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/2325845478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f810c170 0x7fc7f810e610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009ca0 tx=0x7fc7f002a950 comp rx=0 tx=0).stop 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 -- 172.21.15.143:0/2325845478 shutdown_connections 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/2325845478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f810c170 0x7fc7f810e610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/2325845478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f8103660 0x7fc7f810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/2325845478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7f8102d40 0x7fc7f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 -- 172.21.15.143:0/2325845478 >> 172.21.15.143:0/2325845478 conn(0x7fc7f80fc7a0 msgr2=0x7fc7f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.168+0000 7fc7fcf06640 1 -- 172.21.15.143:0/2325845478 shutdown_connections 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 -- 172.21.15.143:0/2325845478 wait complete. 2024-09-10T10:43:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 Processor -- start 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 -- start start 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 0x7fc7f8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f8103660 0x7fc7f8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7f810c170 0x7fc7f819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8111e20 con 0x7fc7f8102d40 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8111ca0 con 0x7fc7f8103660 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.169+0000 7fc7fcf06640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8111fa0 con 0x7fc7f810c170 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 0x7fc7f8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 0x7fc7f8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48446/0 (socket says 172.21.15.143:48446) 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 -- 172.21.15.143:0/3630033153 learned_addr learned my addr 172.21.15.143:0/3630033153 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f6d9d640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7f810c170 0x7fc7f819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f5d9b640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f8103660 0x7fc7f8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 -- 172.21.15.143:0/3630033153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7f810c170 msgr2=0x7fc7f819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7f810c170 0x7fc7f819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 -- 172.21.15.143:0/3630033153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f8103660 msgr2=0x7fc7f8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f8103660 0x7fc7f8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 -- 172.21.15.143:0/3630033153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f819d3c0 con 0x7fc7f8102d40 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7f659c640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 0x7fc7f8198410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800e9f0 tx=0x7fc7e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800cd80 con 0x7fc7f8102d40 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f819d6b0 con 0x7fc7f8102d40 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7e80054d0 con 0x7fc7f8102d40 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8010490 con 0x7fc7f8102d40 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.170+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81a4f90 con 0x7fc7f8102d40 2024-09-10T10:43:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.171+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8103120 con 0x7fc7f8102d40 2024-09-10T10:43:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.172+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc7e8005020 con 0x7fc7f8102d40 2024-09-10T10:43:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.173+0000 7fc7e77fe640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc7e00a2b80 0x7fc7e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.173+0000 7fc7f5d9b640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc7e00a2b80 0x7fc7e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.173+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc7e80cf760 con 0x7fc7f8102d40 2024-09-10T10:43:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.174+0000 7fc7f5d9b640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc7e00a2b80 0x7fc7e00a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005d90 tx=0x7fc7ec005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.177+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7e809c480 con 0x7fc7f8102d40 2024-09-10T10:43:32.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.382+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc7f8107210 con 0x7fc7f8102d40 2024-09-10T10:43:32.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.546+0000 7fc7e77fe640 1 -- 172.21.15.143:0/3630033153 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fc7e809bd80 con 0x7fc7f8102d40 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc7e00a2b80 msgr2=0x7fc7e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc7e00a2b80 0x7fc7e00a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005d90 tx=0x7fc7ec005d20 comp rx=0 tx=0).stop 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 msgr2=0x7fc7f8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 0x7fc7f8198410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800e9f0 tx=0x7fc7e800edf0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 shutdown_connections 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc7e00a2b80 0x7fc7e00a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7f810c170 0x7fc7f819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f8103660 0x7fc7f8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 --2- 172.21.15.143:0/3630033153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f8102d40 0x7fc7f8198410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:32.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 >> 172.21.15.143:0/3630033153 conn(0x7fc7f80fc7a0 msgr2=0x7fc7f80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:32.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 shutdown_connections 2024-09-10T10:43:32.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:32.549+0000 7fc7fcf06640 1 -- 172.21.15.143:0/3630033153 wait complete. 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 3 op/s 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] up:active 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: Cluster is now healthy 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-10T10:43:32.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:32 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3630033153' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:43:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 3 op/s 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] up:active 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: Cluster is now healthy 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-10T10:43:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:32 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3630033153' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 3 op/s 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] up:active 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: Cluster is now healthy 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-10T10:43:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:32 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3630033153' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:43:33.385 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-10T10:43:33.385 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T10:43:33.637 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:33.758 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:33 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3630033153' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:43:33.758 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:33 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] up:active 2024-09-10T10:43:33.759 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:33 smithi143 ceph-mon[31057]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T10:43:33.759 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:33 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:43:33.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:33 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3630033153' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:43:33.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:33 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] up:active 2024-09-10T10:43:33.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:33 smithi178 ceph-mon[34813]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T10:43:33.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:33 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:43:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:33 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3630033153' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:43:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:33 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] up:active 2024-09-10T10:43:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:33 smithi144 ceph-mon[34230]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T10:43:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:33 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:43:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 -- 172.21.15.143:0/1456013031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 msgr2=0x7f54641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 --2- 172.21.15.143:0/1456013031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f54641021c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5454009900 tx=0x7f545402a410 comp rx=0 tx=0).stop 2024-09-10T10:43:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 -- 172.21.15.143:0/1456013031 shutdown_connections 2024-09-10T10:43:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 --2- 172.21.15.143:0/1456013031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5464106f10 0x7f546410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 --2- 172.21.15.143:0/1456013031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5464102790 0x7f54641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 --2- 172.21.15.143:0/1456013031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f54641021c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 -- 172.21.15.143:0/1456013031 >> 172.21.15.143:0/1456013031 conn(0x7f54640fd8c0 msgr2=0x7f54640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 -- 172.21.15.143:0/1456013031 shutdown_connections 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.114+0000 7f5468a02640 1 -- 172.21.15.143:0/1456013031 wait complete. 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.115+0000 7f5468a02640 1 Processor -- start 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.115+0000 7f5468a02640 1 -- start start 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f5464198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5464102790 0x7f54641988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5464106f10 0x7f546419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5464111e20 con 0x7f5464101de0 2024-09-10T10:43:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5464111ca0 con 0x7f5464102790 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5464111fa0 con 0x7f5464106f10 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f5464198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f5464198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48476/0 (socket says 172.21.15.143:48476) 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 -- 172.21.15.143:0/160722133 learned_addr learned my addr 172.21.15.143:0/160722133 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5462d9d640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5464106f10 0x7f546419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5461d9b640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5464102790 0x7f54641988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 -- 172.21.15.143:0/160722133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5464106f10 msgr2=0x7f546419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5464106f10 0x7f546419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 -- 172.21.15.143:0/160722133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5464102790 msgr2=0x7f54641988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5464102790 0x7f54641988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 -- 172.21.15.143:0/160722133 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546419d310 con 0x7f5464101de0 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f546259c640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f5464198360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5454005940 tx=0x7f5454004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545403f030 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5454003aa0 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454035070 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546419d5a0 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.116+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54641a4ee0 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.117+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54641032c0 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.118+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5454042030 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.119+0000 7f545b7fe640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f54500a2b80 0x7f54500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.119+0000 7f5461d9b640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f54500a2b80 0x7f54500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.119+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f54540efa20 con 0x7f5464101de0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.120+0000 7f5461d9b640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f54500a2b80 0x7f54500a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54640fdd50 tx=0x7f544c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.123+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54540bc090 con 0x7f5464101de0 2024-09-10T10:43:34.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.311+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f546410a160 con 0x7f54500a2b80 2024-09-10T10:43:34.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.339+0000 7f545b7fe640 1 -- 172.21.15.143:0/160722133 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f544c010be0 con 0x7f54500a2b80 2024-09-10T10:43:34.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f54500a2b80 msgr2=0x7f54500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f54500a2b80 0x7f54500a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54640fdd50 tx=0x7f544c00b390 comp rx=0 tx=0).stop 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 msgr2=0x7f5464198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f5464198360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5454005940 tx=0x7f5454004270 comp rx=0 tx=0).stop 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 shutdown_connections 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f54500a2b80 0x7f54500a5040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5464106f10 0x7f546419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5464102790 0x7f54641988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 --2- 172.21.15.143:0/160722133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5464101de0 0x7f5464198360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 >> 172.21.15.143:0/160722133 conn(0x7f54640fd8c0 msgr2=0x7f54640fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 shutdown_connections 2024-09-10T10:43:34.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:34.342+0000 7f5468a02640 1 -- 172.21.15.143:0/160722133 wait complete. 2024-09-10T10:43:34.790 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:34 smithi143 ceph-mon[31057]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T10:43:34.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:43:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:43:34.316+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:43:34.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:43:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:43:34.316+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:43:34.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:43:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:43:34.316+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:43:34.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:43:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:43:34.316+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:43:34.791 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:43:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:43:34.316+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:43:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:34 smithi178 ceph-mon[34813]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T10:43:34.925 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:43:34 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: 2024-09-10T10:43:34.490+0000 7f6fda90c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T10:43:34.925 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:43:34 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: 2024-09-10T10:43:34.490+0000 7f6fda90c640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-10T10:43:34.925 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:43:34 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: 2024-09-10T10:43:34.490+0000 7f6fda90c640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-10T10:43:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:34 smithi144 ceph-mon[34230]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T10:43:35.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T10:43:35.389 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:35.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:35 smithi143 ceph-mon[31057]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:35.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:35 smithi178 ceph-mon[34813]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:35.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:35 smithi144 ceph-mon[34230]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:35.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1338511238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4102570 msgr2=0x7f94a41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:35.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1338511238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4102570 0x7f94a41029b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9498009900 tx=0x7f949802a480 comp rx=0 tx=0).stop 2024-09-10T10:43:35.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1338511238 shutdown_connections 2024-09-10T10:43:35.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1338511238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a4102ef0 0x7f94a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1338511238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4102570 0x7f94a41029b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1338511238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a4100e20 0x7f94a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1338511238 >> 172.21.15.143:0/1338511238 conn(0x7f94a40fc7d0 msgr2=0x7f94a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1338511238 shutdown_connections 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1338511238 wait complete. 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.942+0000 7f94a8ef9640 1 Processor -- start 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 -- start start 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 0x7f94a41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a4102570 0x7f94a41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a4102ef0 0x7f94a419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4110e00 con 0x7f94a4100e20 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4110c80 con 0x7f94a4102ef0 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a8ef9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4110f80 con 0x7f94a4102570 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 0x7f94a41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 0x7f94a41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48510/0 (socket says 172.21.15.143:48510) 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 -- 172.21.15.143:0/1443513373 learned_addr learned my addr 172.21.15.143:0/1443513373 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a2d9d640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a4102ef0 0x7f94a419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a1d9b640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a4102570 0x7f94a41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 -- 172.21.15.143:0/1443513373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a4102570 msgr2=0x7f94a41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a4102570 0x7f94a41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 -- 172.21.15.143:0/1443513373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a4102ef0 msgr2=0x7f94a419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a4102ef0 0x7f94a419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 -- 172.21.15.143:0/1443513373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a419d460 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94a259c640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 0x7f94a41984b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f949400c920 tx=0x7f949400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.943+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494007c70 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.944+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9494007e10 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.944+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a419d750 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.944+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494018820 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.944+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a41a5030 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.945+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a4101200 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.946+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f94940189c0 con 0x7f94a4100e20 2024-09-10T10:43:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.946+0000 7f94937fe640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f948c0a2c90 0x7f948c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.946+0000 7f94a1d9b640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f948c0a2c90 0x7f948c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.946+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f94940cf4e0 con 0x7f94a4100e20 2024-09-10T10:43:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.947+0000 7f94a1d9b640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f948c0a2c90 0x7f948c0a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f94a4100120 tx=0x7f9498005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:35.950+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9494014030 con 0x7f94a4100e20 2024-09-10T10:43:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.141+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f94a4062630 con 0x7f948c0a2c90 2024-09-10T10:43:36.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.167+0000 7f94937fe640 1 -- 172.21.15.143:0/1443513373 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f94a4062630 con 0x7f948c0a2c90 2024-09-10T10:43:36.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f948c0a2c90 msgr2=0x7f948c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:36.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f948c0a2c90 0x7f948c0a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f94a4100120 tx=0x7f9498005a20 comp rx=0 tx=0).stop 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 msgr2=0x7f94a41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 0x7f94a41984b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f949400c920 tx=0x7f949400cde0 comp rx=0 tx=0).stop 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 shutdown_connections 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f948c0a2c90 0x7f948c0a5150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a4102ef0 0x7f94a419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a4102570 0x7f94a41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 --2- 172.21.15.143:0/1443513373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94a4100e20 0x7f94a41984b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 >> 172.21.15.143:0/1443513373 conn(0x7f94a40fc7d0 msgr2=0x7f94a406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 shutdown_connections 2024-09-10T10:43:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:36.169+0000 7f94a8ef9640 1 -- 172.21.15.143:0/1443513373 wait complete. 2024-09-10T10:43:36.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:36 smithi143 ceph-mon[31057]: pgmap v305: 129 pgs: 129 active+clean; 585 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-10T10:43:36.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:36 smithi143 ceph-mon[31057]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T10:43:36.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:36 smithi178 ceph-mon[34813]: pgmap v305: 129 pgs: 129 active+clean; 585 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-10T10:43:36.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:36 smithi178 ceph-mon[34813]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T10:43:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:36 smithi144 ceph-mon[34230]: pgmap v305: 129 pgs: 129 active+clean; 585 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 3.7 KiB/s wr, 12 op/s 2024-09-10T10:43:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:36 smithi144 ceph-mon[34230]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T10:43:36.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T10:43:37.245 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:37.656 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:37 smithi143 ceph-mon[31057]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 -- 172.21.15.143:0/1261866899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec100310 msgr2=0x7f2cec1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/1261866899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec100310 0x7f2cec1006f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009ca0 tx=0x7f2cdc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 -- 172.21.15.143:0/1261866899 shutdown_connections 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/1261866899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec106f30 0x7f2cec10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/1261866899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec100cc0 0x7f2cec106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/1261866899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec100310 0x7f2cec1006f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 -- 172.21.15.143:0/1261866899 >> 172.21.15.143:0/1261866899 conn(0x7f2cec077210 msgr2=0x7f2cec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 -- 172.21.15.143:0/1261866899 shutdown_connections 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.738+0000 7f2cf0b94640 1 -- 172.21.15.143:0/1261866899 wait complete. 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 Processor -- start 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 -- start start 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec100310 0x7f2cec198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec100cc0 0x7f2cec198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 0x7f2cec19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec111e00 con 0x7f2cec106f30 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec111c80 con 0x7f2cec100cc0 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cf0b94640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec111f80 con 0x7f2cec100310 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cead9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 0x7f2cec19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cead9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 0x7f2cec19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44450/0 (socket says 172.21.15.143:44450) 2024-09-10T10:43:37.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cead9d640 1 -- 172.21.15.143:0/3262728795 learned_addr learned my addr 172.21.15.143:0/3262728795 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2ce9d9b640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec100cc0 0x7f2cec198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.739+0000 7f2cea59c640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec100310 0x7f2cec198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cead9d640 1 -- 172.21.15.143:0/3262728795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec100310 msgr2=0x7f2cec198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cead9d640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec100310 0x7f2cec198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cead9d640 1 -- 172.21.15.143:0/3262728795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec100cc0 msgr2=0x7f2cec198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cead9d640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec100cc0 0x7f2cec198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cead9d640 1 -- 172.21.15.143:0/3262728795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec19d2d0 con 0x7f2cec106f30 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cea59c640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec100310 0x7f2cec198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cead9d640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 0x7f2cec19cbf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400d750 tx=0x7f2ce400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4017070 con 0x7f2cec106f30 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ce40054d0 con 0x7f2cec106f30 2024-09-10T10:43:37.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec19d5c0 con 0x7f2cec106f30 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004c60 con 0x7f2cec106f30 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.740+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec1a4ea0 con 0x7f2cec106f30 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.741+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec1006f0 con 0x7f2cec106f30 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.742+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2ce4004e00 con 0x7f2cec106f30 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.743+0000 7f2cdb7fe640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cd40a2c10 0x7f2cd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.743+0000 7f2cea59c640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cd40a2c10 0x7f2cd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.743+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f2ce40ceda0 con 0x7f2cec106f30 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.743+0000 7f2cea59c640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cd40a2c10 0x7f2cd40a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f80 tx=0x7f2cdc005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:37.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.747+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ce409b900 con 0x7f2cec106f30 2024-09-10T10:43:37.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:37 smithi178 ceph-mon[34813]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:37.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:37 smithi144 ceph-mon[34230]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:37.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.939+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2cec0626c0 con 0x7f2cd40a2c10 2024-09-10T10:43:37.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.945+0000 7f2cdb7fe640 1 -- 172.21.15.143:0/3262728795 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2cec0626c0 con 0x7f2cd40a2c10 2024-09-10T10:43:37.947 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73 2024-09-10T10:43:37.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cd40a2c10 msgr2=0x7f2cd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:37.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cd40a2c10 0x7f2cd40a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f80 tx=0x7f2cdc005ef0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 msgr2=0x7f2cec19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:37.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 0x7f2cec19cbf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400d750 tx=0x7f2ce400dc10 comp rx=0 tx=0).stop 2024-09-10T10:43:37.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 shutdown_connections 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cd40a2c10 0x7f2cd40a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cec106f30 0x7f2cec19cbf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec100cc0 0x7f2cec198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 --2- 172.21.15.143:0/3262728795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2cec100310 0x7f2cec198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 >> 172.21.15.143:0/3262728795 conn(0x7f2cec077210 msgr2=0x7f2cec105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 shutdown_connections 2024-09-10T10:43:37.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:37.947+0000 7f2cf0b94640 1 -- 172.21.15.143:0/3262728795 wait complete. 2024-09-10T10:43:38.696 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:38 smithi143 ceph-mon[31057]: pgmap v306: 129 pgs: 129 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-10T10:43:38.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T10:43:38.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:38 smithi178 ceph-mon[34813]: pgmap v306: 129 pgs: 129 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-10T10:43:38.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:38 smithi144 ceph-mon[34230]: pgmap v306: 129 pgs: 129 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-10T10:43:38.970 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 -- 172.21.15.143:0/1420404421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074510 msgr2=0x7f62980748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 --2- 172.21.15.143:0/1420404421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074510 0x7f62980748f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6288009ca0 tx=0x7f628802a950 comp rx=0 tx=0).stop 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 -- 172.21.15.143:0/1420404421 shutdown_connections 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 --2- 172.21.15.143:0/1420404421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f629810e430 0x7f6298110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 --2- 172.21.15.143:0/1420404421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6298074e30 0x7f629810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 --2- 172.21.15.143:0/1420404421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074510 0x7f62980748f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 -- 172.21.15.143:0/1420404421 >> 172.21.15.143:0/1420404421 conn(0x7f62980fd8b0 msgr2=0x7f62980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:39.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 -- 172.21.15.143:0/1420404421 shutdown_connections 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.458+0000 7f629d729640 1 -- 172.21.15.143:0/1420404421 wait complete. 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 Processor -- start 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 -- start start 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6298074510 0x7f629819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 0x7f629819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f629810e430 0x7f629819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114080 con 0x7f6298074e30 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6298113f00 con 0x7f629810e430 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f629d729640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114200 con 0x7f6298074510 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f62967fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 0x7f629819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f62967fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 0x7f629819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44478/0 (socket says 172.21.15.143:44478) 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.459+0000 7f62967fc640 1 -- 172.21.15.143:0/1124770658 learned_addr learned my addr 172.21.15.143:0/1124770658 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:39.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62977fe640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f629810e430 0x7f629819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f6296ffd640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6298074510 0x7f629819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62967fc640 1 -- 172.21.15.143:0/1124770658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6298074510 msgr2=0x7f629819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62967fc640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6298074510 0x7f629819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62967fc640 1 -- 172.21.15.143:0/1124770658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f629810e430 msgr2=0x7f629819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62967fc640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f629810e430 0x7f629819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62967fc640 1 -- 172.21.15.143:0/1124770658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629819f5f0 con 0x7f6298074e30 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f6296ffd640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6298074510 0x7f629819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62977fe640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f629810e430 0x7f629819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f62967fc640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 0x7f629819ab80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f628c00d950 tx=0x7f628c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c017070 con 0x7f6298074e30 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f628c009c30 con 0x7f6298074e30 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629819f8e0 con 0x7f6298074e30 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c00f620 con 0x7f6298074e30 2024-09-10T10:43:39.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.460+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62981a71c0 con 0x7f6298074e30 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.461+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62980748f0 con 0x7f6298074e30 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.462+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f628c0109a0 con 0x7f6298074e30 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.463+0000 7f627ffff640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f62800a2c10 0x7f62800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.463+0000 7f6296ffd640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f62800a2c10 0x7f62800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.463+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f628c0cf0d0 con 0x7f6298074e30 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.463+0000 7f6296ffd640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f62800a2c10 0x7f62800a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6288006010 tx=0x7f6288005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:39.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.466+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f628c009de0 con 0x7f6298074e30 2024-09-10T10:43:39.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.661+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6298062630 con 0x7f62800a2c10 2024-09-10T10:43:39.663 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:39 smithi143 ceph-mon[31057]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:39.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.685+0000 7f627ffff640 1 -- 172.21.15.143:0/1124770658 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6298062630 con 0x7f62800a2c10 2024-09-10T10:43:39.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f62800a2c10 msgr2=0x7f62800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f62800a2c10 0x7f62800a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6288006010 tx=0x7f6288005f80 comp rx=0 tx=0).stop 2024-09-10T10:43:39.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 msgr2=0x7f629819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 0x7f629819ab80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f628c00d950 tx=0x7f628c00de10 comp rx=0 tx=0).stop 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 shutdown_connections 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f62800a2c10 0x7f62800a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f629810e430 0x7f629819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6298074e30 0x7f629819ab80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 --2- 172.21.15.143:0/1124770658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6298074510 0x7f629819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 >> 172.21.15.143:0/1124770658 conn(0x7f62980fd8b0 msgr2=0x7f62980fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 shutdown_connections 2024-09-10T10:43:39.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:39.687+0000 7f629d729640 1 -- 172.21.15.143:0/1124770658 wait complete. 2024-09-10T10:43:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:39 smithi178 ceph-mon[34813]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:39 smithi144 ceph-mon[34230]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:40.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T10:43:40.686 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:40.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:40 smithi143 ceph-mon[31057]: pgmap v307: 129 pgs: 129 active+clean; 608 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-09-10T10:43:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:40 smithi178 ceph-mon[34813]: pgmap v307: 129 pgs: 129 active+clean; 608 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-09-10T10:43:40.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:40 smithi144 ceph-mon[34230]: pgmap v307: 129 pgs: 129 active+clean; 608 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 KiB/s wr, 20 op/s 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2916247939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 msgr2=0x7f3b78100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2916247939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78100ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009930 tx=0x7f3b6c02a420 comp rx=0 tx=0).stop 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2916247939 shutdown_connections 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2916247939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b78101420 0x7f3b7810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2916247939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78100ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2916247939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78106a80 0x7f3b78106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2916247939 >> 172.21.15.143:0/2916247939 conn(0x7f3b780fc7a0 msgr2=0x7f3b780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2916247939 shutdown_connections 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.186+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2916247939 wait complete. 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 Processor -- start 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 -- start start 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b78101420 0x7f3b78198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78106a80 0x7f3b7819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78111e50 con 0x7f3b78100a80 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78111cd0 con 0x7f3b78101420 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7f26e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78111fd0 con 0x7f3b78106a80 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7d00a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7d00a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44488/0 (socket says 172.21.15.143:44488) 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7d00a640 1 -- 172.21.15.143:0/2752817711 learned_addr learned my addr 172.21.15.143:0/2752817711 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.187+0000 7f3b7d80b640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78106a80 0x7f3b7819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7c809640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b78101420 0x7f3b78198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7d00a640 1 -- 172.21.15.143:0/2752817711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78106a80 msgr2=0x7f3b7819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7d00a640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78106a80 0x7f3b7819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7d00a640 1 -- 172.21.15.143:0/2752817711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b78101420 msgr2=0x7f3b78198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7d00a640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b78101420 0x7f3b78198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7d00a640 1 -- 172.21.15.143:0/2752817711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7819d300 con 0x7f3b78100a80 2024-09-10T10:43:41.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7d00a640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78198350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400c920 tx=0x7f3b7400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74007c70 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b74007e10 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7819d5f0 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74018820 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.188+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a4ed0 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.189+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78100ee0 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.190+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3b740189c0 con 0x7f3b78100a80 2024-09-10T10:43:41.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.191+0000 7f3b6a7fc640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3b640a2c90 0x7f3b640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.191+0000 7f3b7c809640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3b640a2c90 0x7f3b640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.191+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f3b740cf770 con 0x7f3b78100a80 2024-09-10T10:43:41.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.191+0000 7f3b7c809640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3b640a2c90 0x7f3b640a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c006010 tx=0x7f3b6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.194+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b74014030 con 0x7f3b78100a80 2024-09-10T10:43:41.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.392+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3b78062630 con 0x7f3b640a2c90 2024-09-10T10:43:41.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.397+0000 7f3b6a7fc640 1 -- 172.21.15.143:0/2752817711 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3b78062630 con 0x7f3b640a2c90 2024-09-10T10:43:41.399 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/qa/sv_1/784f37d3-3181-4910-b5c8-10b2584e46b3 2024-09-10T10:43:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3b640a2c90 msgr2=0x7f3b640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3b640a2c90 0x7f3b640a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c006010 tx=0x7f3b6c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 msgr2=0x7f3b78198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78198350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400c920 tx=0x7f3b7400cde0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 shutdown_connections 2024-09-10T10:43:41.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3b640a2c90 0x7f3b640a5150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78106a80 0x7f3b7819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b78101420 0x7f3b78198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 --2- 172.21.15.143:0/2752817711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78100a80 0x7f3b78198350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 >> 172.21.15.143:0/2752817711 conn(0x7f3b780fc7a0 msgr2=0x7f3b780fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 shutdown_connections 2024-09-10T10:43:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:41.399+0000 7f3b7f26e640 1 -- 172.21.15.143:0/2752817711 wait complete. 2024-09-10T10:43:41.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:41 smithi143 ceph-mon[31057]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:41.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:41 smithi178 ceph-mon[34813]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:41.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:41 smithi144 ceph-mon[34230]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:42.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:42.357 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:42.772 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:42 smithi143 ceph-mon[31057]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:42.773 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:42 smithi143 ceph-mon[31057]: pgmap v308: 129 pgs: 129 active+clean; 608 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 -- 172.21.15.143:0/662635250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 msgr2=0x7fddd81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 --2- 172.21.15.143:0/662635250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd81006d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009930 tx=0x7fddd402a420 comp rx=0 tx=0).stop 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 -- 172.21.15.143:0/662635250 shutdown_connections 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 --2- 172.21.15.143:0/662635250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd8106f50 0x7fddd810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 --2- 172.21.15.143:0/662635250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd8100ca0 0x7fddd8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 --2- 172.21.15.143:0/662635250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd81006d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 -- 172.21.15.143:0/662635250 >> 172.21.15.143:0/662635250 conn(0x7fddd8077210 msgr2=0x7fddd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 -- 172.21.15.143:0/662635250 shutdown_connections 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.849+0000 7fdddf742640 1 -- 172.21.15.143:0/662635250 wait complete. 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 Processor -- start 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 -- start start 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd8100ca0 0x7fddd81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd8106f50 0x7fddd819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8111e50 con 0x7fddd81002f0 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8111cd0 con 0x7fddd8106f50 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddf742640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8111fd0 con 0x7fddd8100ca0 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddd4de640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddd4de640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44516/0 (socket says 172.21.15.143:44516) 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.850+0000 7fdddd4de640 1 -- 172.21.15.143:0/2282400922 learned_addr learned my addr 172.21.15.143:0/2282400922 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:42.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddccdd640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd8100ca0 0x7fddd81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fddddcdf640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd8106f50 0x7fddd819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddd4de640 1 -- 172.21.15.143:0/2282400922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd8100ca0 msgr2=0x7fddd81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddd4de640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd8100ca0 0x7fddd81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddd4de640 1 -- 172.21.15.143:0/2282400922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd8106f50 msgr2=0x7fddd819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddd4de640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd8106f50 0x7fddd819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddd4de640 1 -- 172.21.15.143:0/2282400922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd819d330 con 0x7fddd81002f0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddd4de640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd8198380 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005940 tx=0x7fddd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd403f030 con 0x7fddd81002f0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd819d5c0 con 0x7fddd81002f0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddd40318a0 con 0x7fddd81002f0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4035070 con 0x7fddd81002f0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.851+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81a4f00 con 0x7fddd81002f0 2024-09-10T10:43:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.852+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd81006d0 con 0x7fddd81002f0 2024-09-10T10:43:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.853+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fddd4042030 con 0x7fddd81002f0 2024-09-10T10:43:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.854+0000 7fddce7fc640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fddc40a2c10 0x7fddc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.854+0000 7fdddccdd640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fddc40a2c10 0x7fddc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.854+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fddd40efc40 con 0x7fddd81002f0 2024-09-10T10:43:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.854+0000 7fdddccdd640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fddc40a2c10 0x7fddc40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fddd80ff860 tx=0x7fddc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:42.858+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddd40f6080 con 0x7fddd81002f0 2024-09-10T10:43:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:42 smithi178 ceph-mon[34813]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:42 smithi178 ceph-mon[34813]: pgmap v308: 129 pgs: 129 active+clean; 608 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-10T10:43:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:42 smithi144 ceph-mon[34230]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:42 smithi144 ceph-mon[34230]: pgmap v308: 129 pgs: 129 active+clean; 608 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-10T10:43:43.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.059+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddd810a100 con 0x7fddd81002f0 2024-09-10T10:43:43.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.060+0000 7fddce7fc640 1 -- 172.21.15.143:0/2282400922 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7fddd40bc800 con 0x7fddd81002f0 2024-09-10T10:43:43.063 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:43.063 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"btime":"2024-09-10T10:43:32:550770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:32.550735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:43:43.063 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fddc40a2c10 msgr2=0x7fddc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fddc40a2c10 0x7fddc40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fddd80ff860 tx=0x7fddc8005f90 comp rx=0 tx=0).stop 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 msgr2=0x7fddd8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd8198380 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005940 tx=0x7fddd40042f0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 shutdown_connections 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fddc40a2c10 0x7fddc40a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd8106f50 0x7fddd819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd8100ca0 0x7fddd81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 --2- 172.21.15.143:0/2282400922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fddd81002f0 0x7fddd8198380 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.062+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 >> 172.21.15.143:0/2282400922 conn(0x7fddd8077210 msgr2=0x7fddd8105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:43.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.063+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 shutdown_connections 2024-09-10T10:43:43.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:43.063+0000 7fdddf742640 1 -- 172.21.15.143:0/2282400922 wait complete. 2024-09-10T10:43:43.762 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:43 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2282400922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:43.762 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:43 smithi143 ceph-mon[31057]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-10T10:43:43.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:43.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:43 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2282400922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:43.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:43 smithi178 ceph-mon[34813]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-10T10:43:43.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:43 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2282400922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:43.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:43 smithi144 ceph-mon[34230]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-10T10:43:44.037 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.543+0000 7f9df5a4a640 1 -- 172.21.15.143:0/584779167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0102ef0 msgr2=0x7f9df010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.543+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/584779167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0102ef0 0x7f9df010da40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009900 tx=0x7f9de802a480 comp rx=0 tx=0).stop 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 -- 172.21.15.143:0/584779167 shutdown_connections 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/584779167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0102ef0 0x7f9df010da40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/584779167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df0102570 0x7f9df01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/584779167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0100e20 0x7f9df0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 -- 172.21.15.143:0/584779167 >> 172.21.15.143:0/584779167 conn(0x7f9df00fc7d0 msgr2=0x7f9df00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 -- 172.21.15.143:0/584779167 shutdown_connections 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 -- 172.21.15.143:0/584779167 wait complete. 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 Processor -- start 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.544+0000 7f9df5a4a640 1 -- start start 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 0x7f9df0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df0102570 0x7f9df01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0102ef0 0x7f9df01a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0110ed0 con 0x7f9df0100e20 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0110d50 con 0x7f9df0102ef0 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0111050 con 0x7f9df0102570 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 0x7f9df0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 0x7f9df0199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44534/0 (socket says 172.21.15.143:44534) 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 -- 172.21.15.143:0/821986953 learned_addr learned my addr 172.21.15.143:0/821986953 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9dee7fc640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df0102570 0x7f9df01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9def7fe640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0102ef0 0x7f9df01a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 -- 172.21.15.143:0/821986953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df0102570 msgr2=0x7f9df01998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df0102570 0x7f9df01998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 -- 172.21.15.143:0/821986953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0102ef0 msgr2=0x7f9df01a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0102ef0 0x7f9df01a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 -- 172.21.15.143:0/821986953 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df01a4920 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9def7fe640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0102ef0 0x7f9df01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9deeffd640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 0x7f9df0199360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9de000ba50 tx=0x7f9de000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0019070 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9de000cbe0 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df01a4af0 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00075d0 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.545+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df01a5080 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.546+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df0101200 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.547+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9de001c070 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.548+0000 7f9df4a48640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9ddc0a2c90 0x7f9ddc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.548+0000 7f9dee7fc640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9ddc0a2c90 0x7f9ddc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.548+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f9de00ce900 con 0x7f9df0100e20 2024-09-10T10:43:44.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.549+0000 7f9dee7fc640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9ddc0a2c90 0x7f9ddc0a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9df006ab00 tx=0x7f9dd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.552+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9de009b5d0 con 0x7f9df0100e20 2024-09-10T10:43:44.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.723+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9df0109090 con 0x7f9df0100e20 2024-09-10T10:43:44.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.725+0000 7f9df4a48640 1 -- 172.21.15.143:0/821986953 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7f9de009aed0 con 0x7f9df0100e20 2024-09-10T10:43:44.726 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:44.726 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:44.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9ddc0a2c90 msgr2=0x7f9ddc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9ddc0a2c90 0x7f9ddc0a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9df006ab00 tx=0x7f9dd8008040 comp rx=0 tx=0).stop 2024-09-10T10:43:44.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 msgr2=0x7f9df0199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 0x7f9df0199360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9de000ba50 tx=0x7f9de000bf10 comp rx=0 tx=0).stop 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 shutdown_connections 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9ddc0a2c90 0x7f9ddc0a5150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df0102ef0 0x7f9df01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9df0102570 0x7f9df01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 --2- 172.21.15.143:0/821986953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df0100e20 0x7f9df0199360 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 >> 172.21.15.143:0/821986953 conn(0x7f9df00fc7d0 msgr2=0x7f9df00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 shutdown_connections 2024-09-10T10:43:44.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:44.727+0000 7f9df5a4a640 1 -- 172.21.15.143:0/821986953 wait complete. 2024-09-10T10:43:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:44 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/821986953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:45.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:44 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/821986953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:45.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:44 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/821986953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:45.443 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:43:45.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:46.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:45 smithi143 ceph-mon[31057]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-10T10:43:46.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:45 smithi178 ceph-mon[34813]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-10T10:43:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:45 smithi144 ceph-mon[34230]: pgmap v310: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-09-10T10:43:46.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 -- 172.21.15.143:0/2382924036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10106f10 msgr2=0x7f7f1010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 --2- 172.21.15.143:0/2382924036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10106f10 0x7f7f1010ea90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009ca0 tx=0x7f7f0802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 -- 172.21.15.143:0/2382924036 shutdown_connections 2024-09-10T10:43:46.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 --2- 172.21.15.143:0/2382924036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10106f10 0x7f7f1010ea90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 --2- 172.21.15.143:0/2382924036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f10102790 0x7f7f101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 --2- 172.21.15.143:0/2382924036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f10101de0 0x7f7f101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 -- 172.21.15.143:0/2382924036 >> 172.21.15.143:0/2382924036 conn(0x7f7f100fd8c0 msgr2=0x7f7f100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 -- 172.21.15.143:0/2382924036 shutdown_connections 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.211+0000 7f7f148bb640 1 -- 172.21.15.143:0/2382924036 wait complete. 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 Processor -- start 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 -- start start 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 0x7f7f10198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f10102790 0x7f7f10198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f10106f10 0x7f7f1019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10111e20 con 0x7f7f10101de0 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10111ca0 con 0x7f7f10102790 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f148bb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10111fa0 con 0x7f7f10106f10 2024-09-10T10:43:46.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f0e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 0x7f7f10198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f0e59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 0x7f7f10198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44544/0 (socket says 172.21.15.143:44544) 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f0e59c640 1 -- 172.21.15.143:0/3551914607 learned_addr learned my addr 172.21.15.143:0/3551914607 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f0ed9d640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f10106f10 0x7f7f1019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.212+0000 7f7f07fff640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f10102790 0x7f7f10198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f0e59c640 1 -- 172.21.15.143:0/3551914607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f10106f10 msgr2=0x7f7f1019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f0e59c640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f10106f10 0x7f7f1019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f0e59c640 1 -- 172.21.15.143:0/3551914607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f10102790 msgr2=0x7f7f10198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f0e59c640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f10102790 0x7f7f10198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f0e59c640 1 -- 172.21.15.143:0/3551914607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1019d2f0 con 0x7f7f10101de0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f0e59c640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 0x7f7f10198340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c890 tx=0x7f7f0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00004220 con 0x7f7f10101de0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f000043c0 con 0x7f7f10101de0 2024-09-10T10:43:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00004a00 con 0x7f7f10101de0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1019d5e0 con 0x7f7f10101de0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.213+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f101a4ec0 con 0x7f7f10101de0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.214+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f101021c0 con 0x7f7f10101de0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.215+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7f00004ba0 con 0x7f7f10101de0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.216+0000 7f7f077fe640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7efc0a2c10 0x7f7efc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.216+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f7f000cf0d0 con 0x7f7f10101de0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.216+0000 7f7f07fff640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7efc0a2c10 0x7f7efc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.216+0000 7f7f07fff640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7efc0a2c10 0x7f7efc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005fd0 tx=0x7f7ef800b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.220+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f00014030 con 0x7f7f10101de0 2024-09-10T10:43:46.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.383+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f10062630 con 0x7f7efc0a2c10 2024-09-10T10:43:46.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.392+0000 7f7f077fe640 1 -- 172.21.15.143:0/3551914607 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340388 (secure 0 0 0) 0x7f7f10062630 con 0x7f7efc0a2c10 2024-09-10T10:43:46.397 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:46.399 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:43:46.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7efc0a2c10 msgr2=0x7f7efc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7efc0a2c10 0x7f7efc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005fd0 tx=0x7f7ef800b510 comp rx=0 tx=0).stop 2024-09-10T10:43:46.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 msgr2=0x7f7f10198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 0x7f7f10198340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c890 tx=0x7f7f0000cd50 comp rx=0 tx=0).stop 2024-09-10T10:43:46.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 shutdown_connections 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7efc0a2c10 0x7f7efc0a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f10106f10 0x7f7f1019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f10102790 0x7f7f10198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 --2- 172.21.15.143:0/3551914607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f10101de0 0x7f7f10198340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 >> 172.21.15.143:0/3551914607 conn(0x7f7f100fd8c0 msgr2=0x7f7f100fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 shutdown_connections 2024-09-10T10:43:46.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:46.396+0000 7f7f148bb640 1 -- 172.21.15.143:0/3551914607 wait complete. 2024-09-10T10:43:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:46 smithi143 ceph-mon[31057]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:43:47.141 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-10T10:43:45.430302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641155,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":259,"num_read_kb":234,"num_write":415,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375328,"kb_used_data":13600,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566496,"statfs":{"total":1151940427776,"available":1151556091904,"internally_reserved":0,"allocated":13926400,"data_stored":7661359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42661,"num_objects":32,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":133,"num_read_kb":125,"num_write":85,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003193"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:39.574086+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:43:39.574086+0000","last_peered":"2024-09-10T10:43:39.574086+0000","last_clean":"2024-09-10T10:43:39.574086+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:43:39.574086+0000","last_undegraded":"2024-09-10T10:43:39.574086+0000","last_fullsized":"2024-09-10T10:43:39.574086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611720+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:43:03.611720+0000","last_peered":"2024-09-10T10:43:03.611720+0000","last_clean":"2024-09-10T10:43:03.611720+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:43:03.611720+0000","last_undegraded":"2024-09-10T10:43:03.611720+0000","last_fullsized":"2024-09-10T10:43:03.611720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611567+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:43:03.611567+0000","last_peered":"2024-09-10T10:43:03.611567+0000","last_clean":"2024-09-10T10:43:03.611567+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:43:03.611567+0000","last_undegraded":"2024-09-10T10:43:03.611567+0000","last_fullsized":"2024-09-10T10:43:03.611567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612034+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:43:03.612034+0000","last_peered":"2024-09-10T10:43:03.612034+0000","last_clean":"2024-09-10T10:43:03.612034+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:43:03.612034+0000","last_undegraded":"2024-09-10T10:43:03.612034+0000","last_fullsized":"2024-09-10T10:43:03.612034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290276+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:43:03.290276+0000","last_peered":"2024-09-10T10:43:03.290276+0000","last_clean":"2024-09-10T10:43:03.290276+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:43:03.290276+0000","last_undegraded":"2024-09-10T10:43:03.290276+0000","last_fullsized":"2024-09-10T10:43:03.290276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290570+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:43:03.290570+0000","last_peered":"2024-09-10T10:43:03.290570+0000","last_clean":"2024-09-10T10:43:03.290570+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:43:03.290570+0000","last_undegraded":"2024-09-10T10:43:03.290570+0000","last_fullsized":"2024-09-10T10:43:03.290570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:32.558737+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:43:32.558737+0000","last_peered":"2024-09-10T10:43:32.558737+0000","last_clean":"2024-09-10T10:43:32.558737+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:43:32.558737+0000","last_undegraded":"2024-09-10T10:43:32.558737+0000","last_fullsized":"2024-09-10T10:43:32.558737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427336+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:43:03.427336+0000","last_peered":"2024-09-10T10:43:03.427336+0000","last_clean":"2024-09-10T10:43:03.427336+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:43:03.427336+0000","last_undegraded":"2024-09-10T10:43:03.427336+0000","last_fullsized":"2024-09-10T10:43:03.427336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290728+0000","last_change":"2024-09-10T10:43:00.293191+0000","last_active":"2024-09-10T10:43:03.290728+0000","last_peered":"2024-09-10T10:43:03.290728+0000","last_clean":"2024-09-10T10:43:03.290728+0000","last_became_active":"2024-09-10T10:43:00.292604+0000","last_became_peered":"2024-09-10T10:43:00.292604+0000","last_unstale":"2024-09-10T10:43:03.290728+0000","last_undegraded":"2024-09-10T10:43:03.290728+0000","last_fullsized":"2024-09-10T10:43:03.290728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:01.555905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.552105+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:43:31.552105+0000","last_peered":"2024-09-10T10:43:31.552105+0000","last_clean":"2024-09-10T10:43:31.552105+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:43:31.552105+0000","last_undegraded":"2024-09-10T10:43:31.552105+0000","last_fullsized":"2024-09-10T10:43:31.552105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612666+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:43:03.612666+0000","last_peered":"2024-09-10T10:43:03.612666+0000","last_clean":"2024-09-10T10:43:03.612666+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:43:03.612666+0000","last_undegraded":"2024-09-10T10:43:03.612666+0000","last_fullsized":"2024-09-10T10:43:03.612666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290635+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:43:03.290635+0000","last_peered":"2024-09-10T10:43:03.290635+0000","last_clean":"2024-09-10T10:43:03.290635+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:43:03.290635+0000","last_undegraded":"2024-09-10T10:43:03.290635+0000","last_fullsized":"2024-09-10T10:43:03.290635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612335+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:43:03.612335+0000","last_peered":"2024-09-10T10:43:03.612335+0000","last_clean":"2024-09-10T10:43:03.612335+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:43:03.612335+0000","last_undegraded":"2024-09-10T10:43:03.612335+0000","last_fullsized":"2024-09-10T10:43:03.612335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:28.529292+0000","last_change":"2024-09-10T10:42:51.227833+0000","last_active":"2024-09-10T10:43:28.529292+0000","last_peered":"2024-09-10T10:43:28.529292+0000","last_clean":"2024-09-10T10:43:28.529292+0000","last_became_active":"2024-09-10T10:42:51.227280+0000","last_became_peered":"2024-09-10T10:42:51.227280+0000","last_unstale":"2024-09-10T10:43:28.529292+0000","last_undegraded":"2024-09-10T10:43:28.529292+0000","last_fullsized":"2024-09-10T10:43:28.529292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.331910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.291030+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:43:03.291030+0000","last_peered":"2024-09-10T10:43:03.291030+0000","last_clean":"2024-09-10T10:43:03.291030+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:43:03.291030+0000","last_undegraded":"2024-09-10T10:43:03.291030+0000","last_fullsized":"2024-09-10T10:43:03.291030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611355+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:43:03.611355+0000","last_peered":"2024-09-10T10:43:03.611355+0000","last_clean":"2024-09-10T10:43:03.611355+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:43:03.611355+0000","last_undegraded":"2024-09-10T10:43:03.611355+0000","last_fullsized":"2024-09-10T10:43:03.611355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.540824+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:43:30.540824+0000","last_peered":"2024-09-10T10:43:30.540824+0000","last_clean":"2024-09-10T10:43:30.540824+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:43:30.540824+0000","last_undegraded":"2024-09-10T10:43:30.540824+0000","last_fullsized":"2024-09-10T10:43:30.540824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.494495+0000","last_change":"2024-09-10T10:43:00.290136+0000","last_active":"2024-09-10T10:43:03.494495+0000","last_peered":"2024-09-10T10:43:03.494495+0000","last_clean":"2024-09-10T10:43:03.494495+0000","last_became_active":"2024-09-10T10:43:00.289490+0000","last_became_peered":"2024-09-10T10:43:00.289490+0000","last_unstale":"2024-09-10T10:43:03.494495+0000","last_undegraded":"2024-09-10T10:43:03.494495+0000","last_fullsized":"2024-09-10T10:43:03.494495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.771225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612180+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:43:03.612180+0000","last_peered":"2024-09-10T10:43:03.612180+0000","last_clean":"2024-09-10T10:43:03.612180+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:43:03.612180+0000","last_undegraded":"2024-09-10T10:43:03.612180+0000","last_fullsized":"2024-09-10T10:43:03.612180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.549618+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:43:31.549618+0000","last_peered":"2024-09-10T10:43:31.549618+0000","last_clean":"2024-09-10T10:43:31.549618+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:43:31.549618+0000","last_undegraded":"2024-09-10T10:43:31.549618+0000","last_fullsized":"2024-09-10T10:43:31.549618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612550+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:43:03.612550+0000","last_peered":"2024-09-10T10:43:03.612550+0000","last_clean":"2024-09-10T10:43:03.612550+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:43:03.612550+0000","last_undegraded":"2024-09-10T10:43:03.612550+0000","last_fullsized":"2024-09-10T10:43:03.612550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288663+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:43:03.288663+0000","last_peered":"2024-09-10T10:43:03.288663+0000","last_clean":"2024-09-10T10:43:03.288663+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:43:03.288663+0000","last_undegraded":"2024-09-10T10:43:03.288663+0000","last_fullsized":"2024-09-10T10:43:03.288663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:28.526524+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:43:28.526524+0000","last_peered":"2024-09-10T10:43:28.526524+0000","last_clean":"2024-09-10T10:43:28.526524+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:43:28.526524+0000","last_undegraded":"2024-09-10T10:43:28.526524+0000","last_fullsized":"2024-09-10T10:43:28.526524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.291256+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:43:03.291256+0000","last_peered":"2024-09-10T10:43:03.291256+0000","last_clean":"2024-09-10T10:43:03.291256+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:43:03.291256+0000","last_undegraded":"2024-09-10T10:43:03.291256+0000","last_fullsized":"2024-09-10T10:43:03.291256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290206+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:43:03.290206+0000","last_peered":"2024-09-10T10:43:03.290206+0000","last_clean":"2024-09-10T10:43:03.290206+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:43:03.290206+0000","last_undegraded":"2024-09-10T10:43:03.290206+0000","last_fullsized":"2024-09-10T10:43:03.290206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551504+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:43:31.551504+0000","last_peered":"2024-09-10T10:43:31.551504+0000","last_clean":"2024-09-10T10:43:31.551504+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:43:31.551504+0000","last_undegraded":"2024-09-10T10:43:31.551504+0000","last_fullsized":"2024-09-10T10:43:31.551504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611879+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:43:03.611879+0000","last_peered":"2024-09-10T10:43:03.611879+0000","last_clean":"2024-09-10T10:43:03.611879+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:43:03.611879+0000","last_undegraded":"2024-09-10T10:43:03.611879+0000","last_fullsized":"2024-09-10T10:43:03.611879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612048+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:43:03.612048+0000","last_peered":"2024-09-10T10:43:03.612048+0000","last_clean":"2024-09-10T10:43:03.612048+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:43:03.612048+0000","last_undegraded":"2024-09-10T10:43:03.612048+0000","last_fullsized":"2024-09-10T10:43:03.612048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'11","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:39.256166+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:43:39.256166+0000","last_peered":"2024-09-10T10:43:39.256166+0000","last_clean":"2024-09-10T10:43:39.256166+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:43:39.256166+0000","last_undegraded":"2024-09-10T10:43:39.256166+0000","last_fullsized":"2024-09-10T10:43:39.256166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610779+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:43:03.610779+0000","last_peered":"2024-09-10T10:43:03.610779+0000","last_clean":"2024-09-10T10:43:03.610779+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:43:03.610779+0000","last_undegraded":"2024-09-10T10:43:03.610779+0000","last_fullsized":"2024-09-10T10:43:03.610779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611473+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:43:03.611473+0000","last_peered":"2024-09-10T10:43:03.611473+0000","last_clean":"2024-09-10T10:43:03.611473+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:43:03.611473+0000","last_undegraded":"2024-09-10T10:43:03.611473+0000","last_fullsized":"2024-09-10T10:43:03.611473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551505+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:43:31.551505+0000","last_peered":"2024-09-10T10:43:31.551505+0000","last_clean":"2024-09-10T10:43:31.551505+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:43:31.551505+0000","last_undegraded":"2024-09-10T10:43:31.551505+0000","last_fullsized":"2024-09-10T10:43:31.551505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611491+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:43:03.611491+0000","last_peered":"2024-09-10T10:43:03.611491+0000","last_clean":"2024-09-10T10:43:03.611491+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:43:03.611491+0000","last_undegraded":"2024-09-10T10:43:03.611491+0000","last_fullsized":"2024-09-10T10:43:03.611491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611913+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:43:03.611913+0000","last_peered":"2024-09-10T10:43:03.611913+0000","last_clean":"2024-09-10T10:43:03.611913+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:43:03.611913+0000","last_undegraded":"2024-09-10T10:43:03.611913+0000","last_fullsized":"2024-09-10T10:43:03.611913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612519+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:43:03.612519+0000","last_peered":"2024-09-10T10:43:03.612519+0000","last_clean":"2024-09-10T10:43:03.612519+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:43:03.612519+0000","last_undegraded":"2024-09-10T10:43:03.612519+0000","last_fullsized":"2024-09-10T10:43:03.612519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288755+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:43:03.288755+0000","last_peered":"2024-09-10T10:43:03.288755+0000","last_clean":"2024-09-10T10:43:03.288755+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:43:03.288755+0000","last_undegraded":"2024-09-10T10:43:03.288755+0000","last_fullsized":"2024-09-10T10:43:03.288755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611686+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:43:03.611686+0000","last_peered":"2024-09-10T10:43:03.611686+0000","last_clean":"2024-09-10T10:43:03.611686+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:43:03.611686+0000","last_undegraded":"2024-09-10T10:43:03.611686+0000","last_fullsized":"2024-09-10T10:43:03.611686+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:38.573426+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:43:38.573426+0000","last_peered":"2024-09-10T10:43:38.573426+0000","last_clean":"2024-09-10T10:43:38.573426+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:43:38.573426+0000","last_undegraded":"2024-09-10T10:43:38.573426+0000","last_fullsized":"2024-09-10T10:43:38.573426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611662+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:43:03.611662+0000","last_peered":"2024-09-10T10:43:03.611662+0000","last_clean":"2024-09-10T10:43:03.611662+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:43:03.611662+0000","last_undegraded":"2024-09-10T10:43:03.611662+0000","last_fullsized":"2024-09-10T10:43:03.611662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612738+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:43:03.612738+0000","last_peered":"2024-09-10T10:43:03.612738+0000","last_clean":"2024-09-10T10:43:03.612738+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:43:03.612738+0000","last_undegraded":"2024-09-10T10:43:03.612738+0000","last_fullsized":"2024-09-10T10:43:03.612738+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:42.578336+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:43:42.578336+0000","last_peered":"2024-09-10T10:43:42.578336+0000","last_clean":"2024-09-10T10:43:42.578336+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:43:42.578336+0000","last_undegraded":"2024-09-10T10:43:42.578336+0000","last_fullsized":"2024-09-10T10:43:42.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611830+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:43:03.611830+0000","last_peered":"2024-09-10T10:43:03.611830+0000","last_clean":"2024-09-10T10:43:03.611830+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:43:03.611830+0000","last_undegraded":"2024-09-10T10:43:03.611830+0000","last_fullsized":"2024-09-10T10:43:03.611830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610712+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:43:03.610712+0000","last_peered":"2024-09-10T10:43:03.610712+0000","last_clean":"2024-09-10T10:43:03.610712+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:43:03.610712+0000","last_undegraded":"2024-09-10T10:43:03.610712+0000","last_fullsized":"2024-09-10T10:43:03.610712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:29.535103+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:43:29.535103+0000","last_peered":"2024-09-10T10:43:29.535103+0000","last_clean":"2024-09-10T10:43:29.535103+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:43:29.535103+0000","last_undegraded":"2024-09-10T10:43:29.535103+0000","last_fullsized":"2024-09-10T10:43:29.535103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.613078+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:43:03.613078+0000","last_peered":"2024-09-10T10:43:03.613078+0000","last_clean":"2024-09-10T10:43:03.613078+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:43:03.613078+0000","last_undegraded":"2024-09-10T10:43:03.613078+0000","last_fullsized":"2024-09-10T10:43:03.613078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611579+0000","last_change":"2024-09-10T10:42:53.233215+0000","last_active":"2024-09-10T10:43:03.611579+0000","last_peered":"2024-09-10T10:43:03.611579+0000","last_clean":"2024-09-10T10:43:03.611579+0000","last_became_active":"2024-09-10T10:42:53.232897+0000","last_became_peered":"2024-09-10T10:42:53.232897+0000","last_unstale":"2024-09-10T10:43:03.611579+0000","last_undegraded":"2024-09-10T10:43:03.611579+0000","last_fullsized":"2024-09-10T10:43:03.611579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:32.458081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:32.560847+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:43:32.560847+0000","last_peered":"2024-09-10T10:43:32.560847+0000","last_clean":"2024-09-10T10:43:32.560847+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:43:32.560847+0000","last_undegraded":"2024-09-10T10:43:32.560847+0000","last_fullsized":"2024-09-10T10:43:32.560847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612631+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:43:03.612631+0000","last_peered":"2024-09-10T10:43:03.612631+0000","last_clean":"2024-09-10T10:43:03.612631+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:43:03.612631+0000","last_undegraded":"2024-09-10T10:43:03.612631+0000","last_fullsized":"2024-09-10T10:43:03.612631+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612438+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:43:03.612438+0000","last_peered":"2024-09-10T10:43:03.612438+0000","last_clean":"2024-09-10T10:43:03.612438+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:43:03.612438+0000","last_undegraded":"2024-09-10T10:43:03.612438+0000","last_fullsized":"2024-09-10T10:43:03.612438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612208+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:43:03.612208+0000","last_peered":"2024-09-10T10:43:03.612208+0000","last_clean":"2024-09-10T10:43:03.612208+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:43:03.612208+0000","last_undegraded":"2024-09-10T10:43:03.612208+0000","last_fullsized":"2024-09-10T10:43:03.612208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:39.588811+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:43:39.588811+0000","last_peered":"2024-09-10T10:43:39.588811+0000","last_clean":"2024-09-10T10:43:39.588811+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:43:39.588811+0000","last_undegraded":"2024-09-10T10:43:39.588811+0000","last_fullsized":"2024-09-10T10:43:39.588811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427262+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:43:03.427262+0000","last_peered":"2024-09-10T10:43:03.427262+0000","last_clean":"2024-09-10T10:43:03.427262+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:43:03.427262+0000","last_undegraded":"2024-09-10T10:43:03.427262+0000","last_fullsized":"2024-09-10T10:43:03.427262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611967+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:43:03.611967+0000","last_peered":"2024-09-10T10:43:03.611967+0000","last_clean":"2024-09-10T10:43:03.611967+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:43:03.611967+0000","last_undegraded":"2024-09-10T10:43:03.611967+0000","last_fullsized":"2024-09-10T10:43:03.611967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.550868+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:43:31.550868+0000","last_peered":"2024-09-10T10:43:31.550868+0000","last_clean":"2024-09-10T10:43:31.550868+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:43:31.550868+0000","last_undegraded":"2024-09-10T10:43:31.550868+0000","last_fullsized":"2024-09-10T10:43:31.550868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289972+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:43:03.289972+0000","last_peered":"2024-09-10T10:43:03.289972+0000","last_clean":"2024-09-10T10:43:03.289972+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:43:03.289972+0000","last_undegraded":"2024-09-10T10:43:03.289972+0000","last_fullsized":"2024-09-10T10:43:03.289972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612185+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:43:03.612185+0000","last_peered":"2024-09-10T10:43:03.612185+0000","last_clean":"2024-09-10T10:43:03.612185+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:43:03.612185+0000","last_undegraded":"2024-09-10T10:43:03.612185+0000","last_fullsized":"2024-09-10T10:43:03.612185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551110+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:43:31.551110+0000","last_peered":"2024-09-10T10:43:31.551110+0000","last_clean":"2024-09-10T10:43:31.551110+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:43:31.551110+0000","last_undegraded":"2024-09-10T10:43:31.551110+0000","last_fullsized":"2024-09-10T10:43:31.551110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612319+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:43:03.612319+0000","last_peered":"2024-09-10T10:43:03.612319+0000","last_clean":"2024-09-10T10:43:03.612319+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:43:03.612319+0000","last_undegraded":"2024-09-10T10:43:03.612319+0000","last_fullsized":"2024-09-10T10:43:03.612319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611708+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:43:03.611708+0000","last_peered":"2024-09-10T10:43:03.611708+0000","last_clean":"2024-09-10T10:43:03.611708+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:43:03.611708+0000","last_undegraded":"2024-09-10T10:43:03.611708+0000","last_fullsized":"2024-09-10T10:43:03.611708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.531214+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:43:30.531214+0000","last_peered":"2024-09-10T10:43:30.531214+0000","last_clean":"2024-09-10T10:43:30.531214+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:43:30.531214+0000","last_undegraded":"2024-09-10T10:43:30.531214+0000","last_fullsized":"2024-09-10T10:43:30.531214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612182+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:43:03.612182+0000","last_peered":"2024-09-10T10:43:03.612182+0000","last_clean":"2024-09-10T10:43:03.612182+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:43:03.612182+0000","last_undegraded":"2024-09-10T10:43:03.612182+0000","last_fullsized":"2024-09-10T10:43:03.612182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612381+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:43:03.612381+0000","last_peered":"2024-09-10T10:43:03.612381+0000","last_clean":"2024-09-10T10:43:03.612381+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:43:03.612381+0000","last_undegraded":"2024-09-10T10:43:03.612381+0000","last_fullsized":"2024-09-10T10:43:03.612381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551186+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:43:31.551186+0000","last_peered":"2024-09-10T10:43:31.551186+0000","last_clean":"2024-09-10T10:43:31.551186+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:43:31.551186+0000","last_undegraded":"2024-09-10T10:43:31.551186+0000","last_fullsized":"2024-09-10T10:43:31.551186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611587+0000","last_change":"2024-09-10T10:43:00.289800+0000","last_active":"2024-09-10T10:43:03.611587+0000","last_peered":"2024-09-10T10:43:03.611587+0000","last_clean":"2024-09-10T10:43:03.611587+0000","last_became_active":"2024-09-10T10:43:00.289440+0000","last_became_peered":"2024-09-10T10:43:00.289440+0000","last_unstale":"2024-09-10T10:43:03.611587+0000","last_undegraded":"2024-09-10T10:43:03.611587+0000","last_fullsized":"2024-09-10T10:43:03.611587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:02.145265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612508+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:43:03.612508+0000","last_peered":"2024-09-10T10:43:03.612508+0000","last_clean":"2024-09-10T10:43:03.612508+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:43:03.612508+0000","last_undegraded":"2024-09-10T10:43:03.612508+0000","last_fullsized":"2024-09-10T10:43:03.612508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:41.577579+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:43:41.577579+0000","last_peered":"2024-09-10T10:43:41.577579+0000","last_clean":"2024-09-10T10:43:41.577579+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:43:41.577579+0000","last_undegraded":"2024-09-10T10:43:41.577579+0000","last_fullsized":"2024-09-10T10:43:41.577579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612605+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:43:03.612605+0000","last_peered":"2024-09-10T10:43:03.612605+0000","last_clean":"2024-09-10T10:43:03.612605+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:43:03.612605+0000","last_undegraded":"2024-09-10T10:43:03.612605+0000","last_fullsized":"2024-09-10T10:43:03.612605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611858+0000","last_change":"2024-09-10T10:42:53.255587+0000","last_active":"2024-09-10T10:43:03.611858+0000","last_peered":"2024-09-10T10:43:03.611858+0000","last_clean":"2024-09-10T10:43:03.611858+0000","last_became_active":"2024-09-10T10:42:53.254961+0000","last_became_peered":"2024-09-10T10:42:53.254961+0000","last_unstale":"2024-09-10T10:43:03.611858+0000","last_undegraded":"2024-09-10T10:43:03.611858+0000","last_fullsized":"2024-09-10T10:43:03.611858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:04.319949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.550871+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:43:31.550871+0000","last_peered":"2024-09-10T10:43:31.550871+0000","last_clean":"2024-09-10T10:43:31.550871+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:43:31.550871+0000","last_undegraded":"2024-09-10T10:43:31.550871+0000","last_fullsized":"2024-09-10T10:43:31.550871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612342+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:43:03.612342+0000","last_peered":"2024-09-10T10:43:03.612342+0000","last_clean":"2024-09-10T10:43:03.612342+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:43:03.612342+0000","last_undegraded":"2024-09-10T10:43:03.612342+0000","last_fullsized":"2024-09-10T10:43:03.612342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611076+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:43:03.611076+0000","last_peered":"2024-09-10T10:43:03.611076+0000","last_clean":"2024-09-10T10:43:03.611076+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:43:03.611076+0000","last_undegraded":"2024-09-10T10:43:03.611076+0000","last_fullsized":"2024-09-10T10:43:03.611076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:32.558874+0000","last_change":"2024-09-10T10:42:51.225741+0000","last_active":"2024-09-10T10:43:32.558874+0000","last_peered":"2024-09-10T10:43:32.558874+0000","last_clean":"2024-09-10T10:43:32.558874+0000","last_became_active":"2024-09-10T10:42:51.225363+0000","last_became_peered":"2024-09-10T10:42:51.225363+0000","last_unstale":"2024-09-10T10:43:32.558874+0000","last_undegraded":"2024-09-10T10:43:32.558874+0000","last_fullsized":"2024-09-10T10:43:32.558874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:28.548670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612699+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:43:03.612699+0000","last_peered":"2024-09-10T10:43:03.612699+0000","last_clean":"2024-09-10T10:43:03.612699+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:43:03.612699+0000","last_undegraded":"2024-09-10T10:43:03.612699+0000","last_fullsized":"2024-09-10T10:43:03.612699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289187+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:43:03.289187+0000","last_peered":"2024-09-10T10:43:03.289187+0000","last_clean":"2024-09-10T10:43:03.289187+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:43:03.289187+0000","last_undegraded":"2024-09-10T10:43:03.289187+0000","last_fullsized":"2024-09-10T10:43:03.289187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:40.577907+0000","last_change":"2024-09-10T10:42:51.225026+0000","last_active":"2024-09-10T10:43:40.577907+0000","last_peered":"2024-09-10T10:43:40.577907+0000","last_clean":"2024-09-10T10:43:40.577907+0000","last_became_active":"2024-09-10T10:42:51.224678+0000","last_became_peered":"2024-09-10T10:42:51.224678+0000","last_unstale":"2024-09-10T10:43:40.577907+0000","last_undegraded":"2024-09-10T10:43:40.577907+0000","last_fullsized":"2024-09-10T10:43:40.577907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:24.154989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":54,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612239+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:43:03.612239+0000","last_peered":"2024-09-10T10:43:03.612239+0000","last_clean":"2024-09-10T10:43:03.612239+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:43:03.612239+0000","last_undegraded":"2024-09-10T10:43:03.612239+0000","last_fullsized":"2024-09-10T10:43:03.612239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427397+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:43:03.427397+0000","last_peered":"2024-09-10T10:43:03.427397+0000","last_clean":"2024-09-10T10:43:03.427397+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:43:03.427397+0000","last_undegraded":"2024-09-10T10:43:03.427397+0000","last_fullsized":"2024-09-10T10:43:03.427397+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.541927+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:43:30.541927+0000","last_peered":"2024-09-10T10:43:30.541927+0000","last_clean":"2024-09-10T10:43:30.541927+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:43:30.541927+0000","last_undegraded":"2024-09-10T10:43:30.541927+0000","last_fullsized":"2024-09-10T10:43:30.541927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290945+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:43:03.290945+0000","last_peered":"2024-09-10T10:43:03.290945+0000","last_clean":"2024-09-10T10:43:03.290945+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:43:03.290945+0000","last_undegraded":"2024-09-10T10:43:03.290945+0000","last_fullsized":"2024-09-10T10:43:03.290945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290294+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:43:03.290294+0000","last_peered":"2024-09-10T10:43:03.290294+0000","last_clean":"2024-09-10T10:43:03.290294+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:43:03.290294+0000","last_undegraded":"2024-09-10T10:43:03.290294+0000","last_fullsized":"2024-09-10T10:43:03.290294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.550624+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:43:31.550624+0000","last_peered":"2024-09-10T10:43:31.550624+0000","last_clean":"2024-09-10T10:43:31.550624+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:43:31.550624+0000","last_undegraded":"2024-09-10T10:43:31.550624+0000","last_fullsized":"2024-09-10T10:43:31.550624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611135+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:43:03.611135+0000","last_peered":"2024-09-10T10:43:03.611135+0000","last_clean":"2024-09-10T10:43:03.611135+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:43:03.611135+0000","last_undegraded":"2024-09-10T10:43:03.611135+0000","last_fullsized":"2024-09-10T10:43:03.611135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427585+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:43:03.427585+0000","last_peered":"2024-09-10T10:43:03.427585+0000","last_clean":"2024-09-10T10:43:03.427585+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:43:03.427585+0000","last_undegraded":"2024-09-10T10:43:03.427585+0000","last_fullsized":"2024-09-10T10:43:03.427585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:40.576858+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:43:40.576858+0000","last_peered":"2024-09-10T10:43:40.576858+0000","last_clean":"2024-09-10T10:43:40.576858+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:43:40.576858+0000","last_undegraded":"2024-09-10T10:43:40.576858+0000","last_fullsized":"2024-09-10T10:43:40.576858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611849+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:43:03.611849+0000","last_peered":"2024-09-10T10:43:03.611849+0000","last_clean":"2024-09-10T10:43:03.611849+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:43:03.611849+0000","last_undegraded":"2024-09-10T10:43:03.611849+0000","last_fullsized":"2024-09-10T10:43:03.611849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290094+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:43:03.290094+0000","last_peered":"2024-09-10T10:43:03.290094+0000","last_clean":"2024-09-10T10:43:03.290094+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:43:03.290094+0000","last_undegraded":"2024-09-10T10:43:03.290094+0000","last_fullsized":"2024-09-10T10:43:03.290094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612036+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:43:03.612036+0000","last_peered":"2024-09-10T10:43:03.612036+0000","last_clean":"2024-09-10T10:43:03.612036+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:43:03.612036+0000","last_undegraded":"2024-09-10T10:43:03.612036+0000","last_fullsized":"2024-09-10T10:43:03.612036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.552174+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:43:31.552174+0000","last_peered":"2024-09-10T10:43:31.552174+0000","last_clean":"2024-09-10T10:43:31.552174+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:43:31.552174+0000","last_undegraded":"2024-09-10T10:43:31.552174+0000","last_fullsized":"2024-09-10T10:43:31.552174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612366+0000","last_change":"2024-09-10T10:43:00.294591+0000","last_active":"2024-09-10T10:43:03.612366+0000","last_peered":"2024-09-10T10:43:03.612366+0000","last_clean":"2024-09-10T10:43:03.612366+0000","last_became_active":"2024-09-10T10:43:00.294194+0000","last_became_peered":"2024-09-10T10:43:00.294194+0000","last_unstale":"2024-09-10T10:43:03.612366+0000","last_undegraded":"2024-09-10T10:43:03.612366+0000","last_fullsized":"2024-09-10T10:43:03.612366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:48:40.230579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610945+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:43:03.610945+0000","last_peered":"2024-09-10T10:43:03.610945+0000","last_clean":"2024-09-10T10:43:03.610945+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:43:03.610945+0000","last_undegraded":"2024-09-10T10:43:03.610945+0000","last_fullsized":"2024-09-10T10:43:03.610945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:29.535069+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:43:29.535069+0000","last_peered":"2024-09-10T10:43:29.535069+0000","last_clean":"2024-09-10T10:43:29.535069+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:43:29.535069+0000","last_undegraded":"2024-09-10T10:43:29.535069+0000","last_fullsized":"2024-09-10T10:43:29.535069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.541280+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:43:30.541280+0000","last_peered":"2024-09-10T10:43:30.541280+0000","last_clean":"2024-09-10T10:43:30.541280+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:43:30.541280+0000","last_undegraded":"2024-09-10T10:43:30.541280+0000","last_fullsized":"2024-09-10T10:43:30.541280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611973+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:43:03.611973+0000","last_peered":"2024-09-10T10:43:03.611973+0000","last_clean":"2024-09-10T10:43:03.611973+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:43:03.611973+0000","last_undegraded":"2024-09-10T10:43:03.611973+0000","last_fullsized":"2024-09-10T10:43:03.611973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611661+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:43:03.611661+0000","last_peered":"2024-09-10T10:43:03.611661+0000","last_clean":"2024-09-10T10:43:03.611661+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:43:03.611661+0000","last_undegraded":"2024-09-10T10:43:03.611661+0000","last_fullsized":"2024-09-10T10:43:03.611661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:40.577345+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:43:40.577345+0000","last_peered":"2024-09-10T10:43:40.577345+0000","last_clean":"2024-09-10T10:43:40.577345+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:43:40.577345+0000","last_undegraded":"2024-09-10T10:43:40.577345+0000","last_fullsized":"2024-09-10T10:43:40.577345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288839+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:43:03.288839+0000","last_peered":"2024-09-10T10:43:03.288839+0000","last_clean":"2024-09-10T10:43:03.288839+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:43:03.288839+0000","last_undegraded":"2024-09-10T10:43:03.288839+0000","last_fullsized":"2024-09-10T10:43:03.288839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612373+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:43:03.612373+0000","last_peered":"2024-09-10T10:43:03.612373+0000","last_clean":"2024-09-10T10:43:03.612373+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:43:03.612373+0000","last_undegraded":"2024-09-10T10:43:03.612373+0000","last_fullsized":"2024-09-10T10:43:03.612373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612084+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:43:03.612084+0000","last_peered":"2024-09-10T10:43:03.612084+0000","last_clean":"2024-09-10T10:43:03.612084+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:43:03.612084+0000","last_undegraded":"2024-09-10T10:43:03.612084+0000","last_fullsized":"2024-09-10T10:43:03.612084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612019+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:43:03.612019+0000","last_peered":"2024-09-10T10:43:03.612019+0000","last_clean":"2024-09-10T10:43:03.612019+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:43:03.612019+0000","last_undegraded":"2024-09-10T10:43:03.612019+0000","last_fullsized":"2024-09-10T10:43:03.612019+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611359+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:43:03.611359+0000","last_peered":"2024-09-10T10:43:03.611359+0000","last_clean":"2024-09-10T10:43:03.611359+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:43:03.611359+0000","last_undegraded":"2024-09-10T10:43:03.611359+0000","last_fullsized":"2024-09-10T10:43:03.611359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289920+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:43:03.289920+0000","last_peered":"2024-09-10T10:43:03.289920+0000","last_clean":"2024-09-10T10:43:03.289920+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:43:03.289920+0000","last_undegraded":"2024-09-10T10:43:03.289920+0000","last_fullsized":"2024-09-10T10:43:03.289920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611390+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:43:03.611390+0000","last_peered":"2024-09-10T10:43:03.611390+0000","last_clean":"2024-09-10T10:43:03.611390+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:43:03.611390+0000","last_undegraded":"2024-09-10T10:43:03.611390+0000","last_fullsized":"2024-09-10T10:43:03.611390+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611935+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:43:03.611935+0000","last_peered":"2024-09-10T10:43:03.611935+0000","last_clean":"2024-09-10T10:43:03.611935+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:43:03.611935+0000","last_undegraded":"2024-09-10T10:43:03.611935+0000","last_fullsized":"2024-09-10T10:43:03.611935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612155+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:43:03.612155+0000","last_peered":"2024-09-10T10:43:03.612155+0000","last_clean":"2024-09-10T10:43:03.612155+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:43:03.612155+0000","last_undegraded":"2024-09-10T10:43:03.612155+0000","last_fullsized":"2024-09-10T10:43:03.612155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611962+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:43:03.611962+0000","last_peered":"2024-09-10T10:43:03.611962+0000","last_clean":"2024-09-10T10:43:03.611962+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:43:03.611962+0000","last_undegraded":"2024-09-10T10:43:03.611962+0000","last_fullsized":"2024-09-10T10:43:03.611962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612637+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:43:03.612637+0000","last_peered":"2024-09-10T10:43:03.612637+0000","last_clean":"2024-09-10T10:43:03.612637+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:43:03.612637+0000","last_undegraded":"2024-09-10T10:43:03.612637+0000","last_fullsized":"2024-09-10T10:43:03.612637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610995+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:43:03.610995+0000","last_peered":"2024-09-10T10:43:03.610995+0000","last_clean":"2024-09-10T10:43:03.610995+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:43:03.610995+0000","last_undegraded":"2024-09-10T10:43:03.610995+0000","last_fullsized":"2024-09-10T10:43:03.610995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427569+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:43:03.427569+0000","last_peered":"2024-09-10T10:43:03.427569+0000","last_clean":"2024-09-10T10:43:03.427569+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:43:03.427569+0000","last_undegraded":"2024-09-10T10:43:03.427569+0000","last_fullsized":"2024-09-10T10:43:03.427569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611274+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:43:03.611274+0000","last_peered":"2024-09-10T10:43:03.611274+0000","last_clean":"2024-09-10T10:43:03.611274+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:43:03.611274+0000","last_undegraded":"2024-09-10T10:43:03.611274+0000","last_fullsized":"2024-09-10T10:43:03.611274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612398+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:43:03.612398+0000","last_peered":"2024-09-10T10:43:03.612398+0000","last_clean":"2024-09-10T10:43:03.612398+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:43:03.612398+0000","last_undegraded":"2024-09-10T10:43:03.612398+0000","last_fullsized":"2024-09-10T10:43:03.612398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612530+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:43:03.612530+0000","last_peered":"2024-09-10T10:43:03.612530+0000","last_clean":"2024-09-10T10:43:03.612530+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:43:03.612530+0000","last_undegraded":"2024-09-10T10:43:03.612530+0000","last_fullsized":"2024-09-10T10:43:03.612530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612414+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:43:03.612414+0000","last_peered":"2024-09-10T10:43:03.612414+0000","last_clean":"2024-09-10T10:43:03.612414+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:43:03.612414+0000","last_undegraded":"2024-09-10T10:43:03.612414+0000","last_fullsized":"2024-09-10T10:43:03.612414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612699+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:43:03.612699+0000","last_peered":"2024-09-10T10:43:03.612699+0000","last_clean":"2024-09-10T10:43:03.612699+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:43:03.612699+0000","last_undegraded":"2024-09-10T10:43:03.612699+0000","last_fullsized":"2024-09-10T10:43:03.612699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427021+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:43:03.427021+0000","last_peered":"2024-09-10T10:43:03.427021+0000","last_clean":"2024-09-10T10:43:03.427021+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:43:03.427021+0000","last_undegraded":"2024-09-10T10:43:03.427021+0000","last_fullsized":"2024-09-10T10:43:03.427021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289692+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:43:03.289692+0000","last_peered":"2024-09-10T10:43:03.289692+0000","last_clean":"2024-09-10T10:43:03.289692+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:43:03.289692+0000","last_undegraded":"2024-09-10T10:43:03.289692+0000","last_fullsized":"2024-09-10T10:43:03.289692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289723+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:43:03.289723+0000","last_peered":"2024-09-10T10:43:03.289723+0000","last_clean":"2024-09-10T10:43:03.289723+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:43:03.289723+0000","last_undegraded":"2024-09-10T10:43:03.289723+0000","last_fullsized":"2024-09-10T10:43:03.289723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611299+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:43:03.611299+0000","last_peered":"2024-09-10T10:43:03.611299+0000","last_clean":"2024-09-10T10:43:03.611299+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:43:03.611299+0000","last_undegraded":"2024-09-10T10:43:03.611299+0000","last_fullsized":"2024-09-10T10:43:03.611299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.291084+0000","last_change":"2024-09-10T10:42:53.252841+0000","last_active":"2024-09-10T10:43:03.291084+0000","last_peered":"2024-09-10T10:43:03.291084+0000","last_clean":"2024-09-10T10:43:03.291084+0000","last_became_active":"2024-09-10T10:42:53.252500+0000","last_became_peered":"2024-09-10T10:42:53.252500+0000","last_unstale":"2024-09-10T10:43:03.291084+0000","last_undegraded":"2024-09-10T10:43:03.291084+0000","last_fullsized":"2024-09-10T10:43:03.291084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:28.502382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612116+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:43:03.612116+0000","last_peered":"2024-09-10T10:43:03.612116+0000","last_clean":"2024-09-10T10:43:03.612116+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:43:03.612116+0000","last_undegraded":"2024-09-10T10:43:03.612116+0000","last_fullsized":"2024-09-10T10:43:03.612116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612215+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:43:03.612215+0000","last_peered":"2024-09-10T10:43:03.612215+0000","last_clean":"2024-09-10T10:43:03.612215+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:43:03.612215+0000","last_undegraded":"2024-09-10T10:43:03.612215+0000","last_fullsized":"2024-09-10T10:43:03.612215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427716+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:43:03.427716+0000","last_peered":"2024-09-10T10:43:03.427716+0000","last_clean":"2024-09-10T10:43:03.427716+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:43:03.427716+0000","last_undegraded":"2024-09-10T10:43:03.427716+0000","last_fullsized":"2024-09-10T10:43:03.427716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610880+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:43:03.610880+0000","last_peered":"2024-09-10T10:43:03.610880+0000","last_clean":"2024-09-10T10:43:03.610880+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:43:03.610880+0000","last_undegraded":"2024-09-10T10:43:03.610880+0000","last_fullsized":"2024-09-10T10:43:03.610880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611704+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:43:03.611704+0000","last_peered":"2024-09-10T10:43:03.611704+0000","last_clean":"2024-09-10T10:43:03.611704+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:43:03.611704+0000","last_undegraded":"2024-09-10T10:43:03.611704+0000","last_fullsized":"2024-09-10T10:43:03.611704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611687+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:43:03.611687+0000","last_peered":"2024-09-10T10:43:03.611687+0000","last_clean":"2024-09-10T10:43:03.611687+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:43:03.611687+0000","last_undegraded":"2024-09-10T10:43:03.611687+0000","last_fullsized":"2024-09-10T10:43:03.611687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288785+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:43:03.288785+0000","last_peered":"2024-09-10T10:43:03.288785+0000","last_clean":"2024-09-10T10:43:03.288785+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:43:03.288785+0000","last_undegraded":"2024-09-10T10:43:03.288785+0000","last_fullsized":"2024-09-10T10:43:03.288785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290977+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:43:03.290977+0000","last_peered":"2024-09-10T10:43:03.290977+0000","last_clean":"2024-09-10T10:43:03.290977+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:43:03.290977+0000","last_undegraded":"2024-09-10T10:43:03.290977+0000","last_fullsized":"2024-09-10T10:43:03.290977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289933+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:43:03.289933+0000","last_peered":"2024-09-10T10:43:03.289933+0000","last_clean":"2024-09-10T10:43:03.289933+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:43:03.289933+0000","last_undegraded":"2024-09-10T10:43:03.289933+0000","last_fullsized":"2024-09-10T10:43:03.289933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290097+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:43:03.290097+0000","last_peered":"2024-09-10T10:43:03.290097+0000","last_clean":"2024-09-10T10:43:03.290097+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:43:03.290097+0000","last_undegraded":"2024-09-10T10:43:03.290097+0000","last_fullsized":"2024-09-10T10:43:03.290097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612605+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:43:03.612605+0000","last_peered":"2024-09-10T10:43:03.612605+0000","last_clean":"2024-09-10T10:43:03.612605+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:43:03.612605+0000","last_undegraded":"2024-09-10T10:43:03.612605+0000","last_fullsized":"2024-09-10T10:43:03.612605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50553,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":133,"num_read_kb":125,"num_write":232,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":151659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":481508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776156,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1111669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":485141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168614,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":481423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1073826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":476549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1073111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":481016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":476257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40532,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704620,"statfs":{"total":95995035648,"available":95953530880,"internally_reserved":0,"allocated":1052672,"data_stored":521537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":520924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":478398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:43:47.148 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:43:47.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:46 smithi178 ceph-mon[34813]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:43:47.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:46 smithi144 ceph-mon[34230]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:43:47.395 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 -- 172.21.15.143:0/3172831888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c103660 msgr2=0x7f788c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 --2- 172.21.15.143:0/3172831888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c103660 0x7f788c10bb50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002a480 comp rx=0 tx=0).stop 2024-09-10T10:43:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 -- 172.21.15.143:0/3172831888 shutdown_connections 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 --2- 172.21.15.143:0/3172831888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10c170 0x7f788c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 --2- 172.21.15.143:0/3172831888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c103660 0x7f788c10bb50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 --2- 172.21.15.143:0/3172831888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c102d40 0x7f788c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 -- 172.21.15.143:0/3172831888 >> 172.21.15.143:0/3172831888 conn(0x7f788c0fc7a0 msgr2=0x7f788c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 -- 172.21.15.143:0/3172831888 shutdown_connections 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.886+0000 7f789150a640 1 -- 172.21.15.143:0/3172831888 wait complete. 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 Processor -- start 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 -- start start 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 0x7f788c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c103660 0x7f788c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c10c170 0x7f788c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f788c111ee0 con 0x7f788c102d40 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f788c111d60 con 0x7f788c10c170 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f789150a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f788c112060 con 0x7f788c103660 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f788affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 0x7f788c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f788affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 0x7f788c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47142/0 (socket says 172.21.15.143:47142) 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.887+0000 7f788affd640 1 -- 172.21.15.143:0/3728245542 learned_addr learned my addr 172.21.15.143:0/3728245542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788b7fe640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c10c170 0x7f788c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788a7fc640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c103660 0x7f788c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788affd640 1 -- 172.21.15.143:0/3728245542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c103660 msgr2=0x7f788c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788affd640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c103660 0x7f788c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788affd640 1 -- 172.21.15.143:0/3728245542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c10c170 msgr2=0x7f788c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788affd640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c10c170 0x7f788c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788affd640 1 -- 172.21.15.143:0/3728245542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c19d4c0 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788a7fc640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c103660 0x7f788c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788b7fe640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c10c170 0x7f788c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f788affd640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 0x7f788c198510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b4a0 tx=0x7f787c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c014070 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f787c010ac0 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c19d7b0 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c017c30 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.888+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c1a5090 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.889+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c1031a0 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.890+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f787c017070 con 0x7f788c102d40 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.891+0000 7f7873fff640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78740a2c90 0x7f78740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.891+0000 7f788a7fc640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78740a2c90 0x7f78740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.891+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f787c0cea90 con 0x7f788c102d40 2024-09-10T10:43:47.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.891+0000 7f788a7fc640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78740a2c90 0x7f78740a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f7880031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:47.894+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f787c09b760 con 0x7f788c102d40 2024-09-10T10:43:48.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:47 smithi143 ceph-mon[31057]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 18 op/s 2024-09-10T10:43:48.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.060+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f788c062630 con 0x7f78740a2c90 2024-09-10T10:43:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.068+0000 7f7873fff640 1 -- 172.21.15.143:0/3728245542 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340389 (secure 0 0 0) 0x7f788c062630 con 0x7f78740a2c90 2024-09-10T10:43:48.072 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.071+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78740a2c90 msgr2=0x7f78740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78740a2c90 0x7f78740a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f7880031400 comp rx=0 tx=0).stop 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 msgr2=0x7f788c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 0x7f788c198510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b4a0 tx=0x7f787c00b960 comp rx=0 tx=0).stop 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 shutdown_connections 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78740a2c90 0x7f78740a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788c10c170 0x7f788c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c103660 0x7f788c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 --2- 172.21.15.143:0/3728245542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f788c102d40 0x7f788c198510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 >> 172.21.15.143:0/3728245542 conn(0x7f788c0fc7a0 msgr2=0x7f788c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 shutdown_connections 2024-09-10T10:43:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:48.072+0000 7f789150a640 1 -- 172.21.15.143:0/3728245542 wait complete. 2024-09-10T10:43:48.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:47 smithi178 ceph-mon[34813]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 18 op/s 2024-09-10T10:43:48.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:47 smithi144 ceph-mon[34230]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 18 op/s 2024-09-10T10:43:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-10T10:43:47.430922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654483,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":300,"num_read_kb":268,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375368,"kb_used_data":13640,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566456,"statfs":{"total":1151940427776,"available":1151556050944,"internally_reserved":0,"allocated":13967360,"data_stored":7705439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":55199,"num_objects":15,"num_object_clones":0,"num_object_copies":47,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":162,"num_read_kb":148,"num_write":63,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003220"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:44.580224+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:43:44.580224+0000","last_peered":"2024-09-10T10:43:44.580224+0000","last_clean":"2024-09-10T10:43:44.580224+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:43:44.580224+0000","last_undegraded":"2024-09-10T10:43:44.580224+0000","last_fullsized":"2024-09-10T10:43:44.580224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611720+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:43:03.611720+0000","last_peered":"2024-09-10T10:43:03.611720+0000","last_clean":"2024-09-10T10:43:03.611720+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:43:03.611720+0000","last_undegraded":"2024-09-10T10:43:03.611720+0000","last_fullsized":"2024-09-10T10:43:03.611720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611567+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:43:03.611567+0000","last_peered":"2024-09-10T10:43:03.611567+0000","last_clean":"2024-09-10T10:43:03.611567+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:43:03.611567+0000","last_undegraded":"2024-09-10T10:43:03.611567+0000","last_fullsized":"2024-09-10T10:43:03.611567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612034+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:43:03.612034+0000","last_peered":"2024-09-10T10:43:03.612034+0000","last_clean":"2024-09-10T10:43:03.612034+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:43:03.612034+0000","last_undegraded":"2024-09-10T10:43:03.612034+0000","last_fullsized":"2024-09-10T10:43:03.612034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290276+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:43:03.290276+0000","last_peered":"2024-09-10T10:43:03.290276+0000","last_clean":"2024-09-10T10:43:03.290276+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:43:03.290276+0000","last_undegraded":"2024-09-10T10:43:03.290276+0000","last_fullsized":"2024-09-10T10:43:03.290276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290570+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:43:03.290570+0000","last_peered":"2024-09-10T10:43:03.290570+0000","last_clean":"2024-09-10T10:43:03.290570+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:43:03.290570+0000","last_undegraded":"2024-09-10T10:43:03.290570+0000","last_fullsized":"2024-09-10T10:43:03.290570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:32.558737+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:43:32.558737+0000","last_peered":"2024-09-10T10:43:32.558737+0000","last_clean":"2024-09-10T10:43:32.558737+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:43:32.558737+0000","last_undegraded":"2024-09-10T10:43:32.558737+0000","last_fullsized":"2024-09-10T10:43:32.558737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427336+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:43:03.427336+0000","last_peered":"2024-09-10T10:43:03.427336+0000","last_clean":"2024-09-10T10:43:03.427336+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:43:03.427336+0000","last_undegraded":"2024-09-10T10:43:03.427336+0000","last_fullsized":"2024-09-10T10:43:03.427336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290728+0000","last_change":"2024-09-10T10:43:00.293191+0000","last_active":"2024-09-10T10:43:03.290728+0000","last_peered":"2024-09-10T10:43:03.290728+0000","last_clean":"2024-09-10T10:43:03.290728+0000","last_became_active":"2024-09-10T10:43:00.292604+0000","last_became_peered":"2024-09-10T10:43:00.292604+0000","last_unstale":"2024-09-10T10:43:03.290728+0000","last_undegraded":"2024-09-10T10:43:03.290728+0000","last_fullsized":"2024-09-10T10:43:03.290728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:01.555905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.552105+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:43:31.552105+0000","last_peered":"2024-09-10T10:43:31.552105+0000","last_clean":"2024-09-10T10:43:31.552105+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:43:31.552105+0000","last_undegraded":"2024-09-10T10:43:31.552105+0000","last_fullsized":"2024-09-10T10:43:31.552105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612666+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:43:03.612666+0000","last_peered":"2024-09-10T10:43:03.612666+0000","last_clean":"2024-09-10T10:43:03.612666+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:43:03.612666+0000","last_undegraded":"2024-09-10T10:43:03.612666+0000","last_fullsized":"2024-09-10T10:43:03.612666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290635+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:43:03.290635+0000","last_peered":"2024-09-10T10:43:03.290635+0000","last_clean":"2024-09-10T10:43:03.290635+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:43:03.290635+0000","last_undegraded":"2024-09-10T10:43:03.290635+0000","last_fullsized":"2024-09-10T10:43:03.290635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612335+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:43:03.612335+0000","last_peered":"2024-09-10T10:43:03.612335+0000","last_clean":"2024-09-10T10:43:03.612335+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:43:03.612335+0000","last_undegraded":"2024-09-10T10:43:03.612335+0000","last_fullsized":"2024-09-10T10:43:03.612335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:28.529292+0000","last_change":"2024-09-10T10:42:51.227833+0000","last_active":"2024-09-10T10:43:28.529292+0000","last_peered":"2024-09-10T10:43:28.529292+0000","last_clean":"2024-09-10T10:43:28.529292+0000","last_became_active":"2024-09-10T10:42:51.227280+0000","last_became_peered":"2024-09-10T10:42:51.227280+0000","last_unstale":"2024-09-10T10:43:28.529292+0000","last_undegraded":"2024-09-10T10:43:28.529292+0000","last_fullsized":"2024-09-10T10:43:28.529292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.331910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.291030+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:43:03.291030+0000","last_peered":"2024-09-10T10:43:03.291030+0000","last_clean":"2024-09-10T10:43:03.291030+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:43:03.291030+0000","last_undegraded":"2024-09-10T10:43:03.291030+0000","last_fullsized":"2024-09-10T10:43:03.291030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611355+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:43:03.611355+0000","last_peered":"2024-09-10T10:43:03.611355+0000","last_clean":"2024-09-10T10:43:03.611355+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:43:03.611355+0000","last_undegraded":"2024-09-10T10:43:03.611355+0000","last_fullsized":"2024-09-10T10:43:03.611355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.540824+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:43:30.540824+0000","last_peered":"2024-09-10T10:43:30.540824+0000","last_clean":"2024-09-10T10:43:30.540824+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:43:30.540824+0000","last_undegraded":"2024-09-10T10:43:30.540824+0000","last_fullsized":"2024-09-10T10:43:30.540824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.494495+0000","last_change":"2024-09-10T10:43:00.290136+0000","last_active":"2024-09-10T10:43:03.494495+0000","last_peered":"2024-09-10T10:43:03.494495+0000","last_clean":"2024-09-10T10:43:03.494495+0000","last_became_active":"2024-09-10T10:43:00.289490+0000","last_became_peered":"2024-09-10T10:43:00.289490+0000","last_unstale":"2024-09-10T10:43:03.494495+0000","last_undegraded":"2024-09-10T10:43:03.494495+0000","last_fullsized":"2024-09-10T10:43:03.494495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.771225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612180+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:43:03.612180+0000","last_peered":"2024-09-10T10:43:03.612180+0000","last_clean":"2024-09-10T10:43:03.612180+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:43:03.612180+0000","last_undegraded":"2024-09-10T10:43:03.612180+0000","last_fullsized":"2024-09-10T10:43:03.612180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.549618+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:43:31.549618+0000","last_peered":"2024-09-10T10:43:31.549618+0000","last_clean":"2024-09-10T10:43:31.549618+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:43:31.549618+0000","last_undegraded":"2024-09-10T10:43:31.549618+0000","last_fullsized":"2024-09-10T10:43:31.549618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612550+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:43:03.612550+0000","last_peered":"2024-09-10T10:43:03.612550+0000","last_clean":"2024-09-10T10:43:03.612550+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:43:03.612550+0000","last_undegraded":"2024-09-10T10:43:03.612550+0000","last_fullsized":"2024-09-10T10:43:03.612550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288663+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:43:03.288663+0000","last_peered":"2024-09-10T10:43:03.288663+0000","last_clean":"2024-09-10T10:43:03.288663+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:43:03.288663+0000","last_undegraded":"2024-09-10T10:43:03.288663+0000","last_fullsized":"2024-09-10T10:43:03.288663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:28.526524+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:43:28.526524+0000","last_peered":"2024-09-10T10:43:28.526524+0000","last_clean":"2024-09-10T10:43:28.526524+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:43:28.526524+0000","last_undegraded":"2024-09-10T10:43:28.526524+0000","last_fullsized":"2024-09-10T10:43:28.526524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.291256+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:43:03.291256+0000","last_peered":"2024-09-10T10:43:03.291256+0000","last_clean":"2024-09-10T10:43:03.291256+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:43:03.291256+0000","last_undegraded":"2024-09-10T10:43:03.291256+0000","last_fullsized":"2024-09-10T10:43:03.291256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290206+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:43:03.290206+0000","last_peered":"2024-09-10T10:43:03.290206+0000","last_clean":"2024-09-10T10:43:03.290206+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:43:03.290206+0000","last_undegraded":"2024-09-10T10:43:03.290206+0000","last_fullsized":"2024-09-10T10:43:03.290206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551504+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:43:31.551504+0000","last_peered":"2024-09-10T10:43:31.551504+0000","last_clean":"2024-09-10T10:43:31.551504+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:43:31.551504+0000","last_undegraded":"2024-09-10T10:43:31.551504+0000","last_fullsized":"2024-09-10T10:43:31.551504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611879+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:43:03.611879+0000","last_peered":"2024-09-10T10:43:03.611879+0000","last_clean":"2024-09-10T10:43:03.611879+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:43:03.611879+0000","last_undegraded":"2024-09-10T10:43:03.611879+0000","last_fullsized":"2024-09-10T10:43:03.611879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612048+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:43:03.612048+0000","last_peered":"2024-09-10T10:43:03.612048+0000","last_clean":"2024-09-10T10:43:03.612048+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:43:03.612048+0000","last_undegraded":"2024-09-10T10:43:03.612048+0000","last_fullsized":"2024-09-10T10:43:03.612048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'14","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:44.257478+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:43:44.257478+0000","last_peered":"2024-09-10T10:43:44.257478+0000","last_clean":"2024-09-10T10:43:44.257478+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:43:44.257478+0000","last_undegraded":"2024-09-10T10:43:44.257478+0000","last_fullsized":"2024-09-10T10:43:44.257478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610779+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:43:03.610779+0000","last_peered":"2024-09-10T10:43:03.610779+0000","last_clean":"2024-09-10T10:43:03.610779+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:43:03.610779+0000","last_undegraded":"2024-09-10T10:43:03.610779+0000","last_fullsized":"2024-09-10T10:43:03.610779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611473+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:43:03.611473+0000","last_peered":"2024-09-10T10:43:03.611473+0000","last_clean":"2024-09-10T10:43:03.611473+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:43:03.611473+0000","last_undegraded":"2024-09-10T10:43:03.611473+0000","last_fullsized":"2024-09-10T10:43:03.611473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551505+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:43:31.551505+0000","last_peered":"2024-09-10T10:43:31.551505+0000","last_clean":"2024-09-10T10:43:31.551505+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:43:31.551505+0000","last_undegraded":"2024-09-10T10:43:31.551505+0000","last_fullsized":"2024-09-10T10:43:31.551505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611491+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:43:03.611491+0000","last_peered":"2024-09-10T10:43:03.611491+0000","last_clean":"2024-09-10T10:43:03.611491+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:43:03.611491+0000","last_undegraded":"2024-09-10T10:43:03.611491+0000","last_fullsized":"2024-09-10T10:43:03.611491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611913+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:43:03.611913+0000","last_peered":"2024-09-10T10:43:03.611913+0000","last_clean":"2024-09-10T10:43:03.611913+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:43:03.611913+0000","last_undegraded":"2024-09-10T10:43:03.611913+0000","last_fullsized":"2024-09-10T10:43:03.611913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612519+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:43:03.612519+0000","last_peered":"2024-09-10T10:43:03.612519+0000","last_clean":"2024-09-10T10:43:03.612519+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:43:03.612519+0000","last_undegraded":"2024-09-10T10:43:03.612519+0000","last_fullsized":"2024-09-10T10:43:03.612519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288755+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:43:03.288755+0000","last_peered":"2024-09-10T10:43:03.288755+0000","last_clean":"2024-09-10T10:43:03.288755+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:43:03.288755+0000","last_undegraded":"2024-09-10T10:43:03.288755+0000","last_fullsized":"2024-09-10T10:43:03.288755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611686+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:43:03.611686+0000","last_peered":"2024-09-10T10:43:03.611686+0000","last_clean":"2024-09-10T10:43:03.611686+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:43:03.611686+0000","last_undegraded":"2024-09-10T10:43:03.611686+0000","last_fullsized":"2024-09-10T10:43:03.611686+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:43.579653+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:43:43.579653+0000","last_peered":"2024-09-10T10:43:43.579653+0000","last_clean":"2024-09-10T10:43:43.579653+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:43:43.579653+0000","last_undegraded":"2024-09-10T10:43:43.579653+0000","last_fullsized":"2024-09-10T10:43:43.579653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611662+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:43:03.611662+0000","last_peered":"2024-09-10T10:43:03.611662+0000","last_clean":"2024-09-10T10:43:03.611662+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:43:03.611662+0000","last_undegraded":"2024-09-10T10:43:03.611662+0000","last_fullsized":"2024-09-10T10:43:03.611662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612738+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:43:03.612738+0000","last_peered":"2024-09-10T10:43:03.612738+0000","last_clean":"2024-09-10T10:43:03.612738+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:43:03.612738+0000","last_undegraded":"2024-09-10T10:43:03.612738+0000","last_fullsized":"2024-09-10T10:43:03.612738+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:42.578336+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:43:42.578336+0000","last_peered":"2024-09-10T10:43:42.578336+0000","last_clean":"2024-09-10T10:43:42.578336+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:43:42.578336+0000","last_undegraded":"2024-09-10T10:43:42.578336+0000","last_fullsized":"2024-09-10T10:43:42.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611830+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:43:03.611830+0000","last_peered":"2024-09-10T10:43:03.611830+0000","last_clean":"2024-09-10T10:43:03.611830+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:43:03.611830+0000","last_undegraded":"2024-09-10T10:43:03.611830+0000","last_fullsized":"2024-09-10T10:43:03.611830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610712+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:43:03.610712+0000","last_peered":"2024-09-10T10:43:03.610712+0000","last_clean":"2024-09-10T10:43:03.610712+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:43:03.610712+0000","last_undegraded":"2024-09-10T10:43:03.610712+0000","last_fullsized":"2024-09-10T10:43:03.610712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:29.535103+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:43:29.535103+0000","last_peered":"2024-09-10T10:43:29.535103+0000","last_clean":"2024-09-10T10:43:29.535103+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:43:29.535103+0000","last_undegraded":"2024-09-10T10:43:29.535103+0000","last_fullsized":"2024-09-10T10:43:29.535103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.613078+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:43:03.613078+0000","last_peered":"2024-09-10T10:43:03.613078+0000","last_clean":"2024-09-10T10:43:03.613078+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:43:03.613078+0000","last_undegraded":"2024-09-10T10:43:03.613078+0000","last_fullsized":"2024-09-10T10:43:03.613078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611579+0000","last_change":"2024-09-10T10:42:53.233215+0000","last_active":"2024-09-10T10:43:03.611579+0000","last_peered":"2024-09-10T10:43:03.611579+0000","last_clean":"2024-09-10T10:43:03.611579+0000","last_became_active":"2024-09-10T10:42:53.232897+0000","last_became_peered":"2024-09-10T10:42:53.232897+0000","last_unstale":"2024-09-10T10:43:03.611579+0000","last_undegraded":"2024-09-10T10:43:03.611579+0000","last_fullsized":"2024-09-10T10:43:03.611579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:32.458081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:32.560847+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:43:32.560847+0000","last_peered":"2024-09-10T10:43:32.560847+0000","last_clean":"2024-09-10T10:43:32.560847+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:43:32.560847+0000","last_undegraded":"2024-09-10T10:43:32.560847+0000","last_fullsized":"2024-09-10T10:43:32.560847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612631+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:43:03.612631+0000","last_peered":"2024-09-10T10:43:03.612631+0000","last_clean":"2024-09-10T10:43:03.612631+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:43:03.612631+0000","last_undegraded":"2024-09-10T10:43:03.612631+0000","last_fullsized":"2024-09-10T10:43:03.612631+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612438+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:43:03.612438+0000","last_peered":"2024-09-10T10:43:03.612438+0000","last_clean":"2024-09-10T10:43:03.612438+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:43:03.612438+0000","last_undegraded":"2024-09-10T10:43:03.612438+0000","last_fullsized":"2024-09-10T10:43:03.612438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612208+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:43:03.612208+0000","last_peered":"2024-09-10T10:43:03.612208+0000","last_clean":"2024-09-10T10:43:03.612208+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:43:03.612208+0000","last_undegraded":"2024-09-10T10:43:03.612208+0000","last_fullsized":"2024-09-10T10:43:03.612208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:44.596798+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:43:44.596798+0000","last_peered":"2024-09-10T10:43:44.596798+0000","last_clean":"2024-09-10T10:43:44.596798+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:43:44.596798+0000","last_undegraded":"2024-09-10T10:43:44.596798+0000","last_fullsized":"2024-09-10T10:43:44.596798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427262+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:43:03.427262+0000","last_peered":"2024-09-10T10:43:03.427262+0000","last_clean":"2024-09-10T10:43:03.427262+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:43:03.427262+0000","last_undegraded":"2024-09-10T10:43:03.427262+0000","last_fullsized":"2024-09-10T10:43:03.427262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611967+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:43:03.611967+0000","last_peered":"2024-09-10T10:43:03.611967+0000","last_clean":"2024-09-10T10:43:03.611967+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:43:03.611967+0000","last_undegraded":"2024-09-10T10:43:03.611967+0000","last_fullsized":"2024-09-10T10:43:03.611967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.550868+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:43:31.550868+0000","last_peered":"2024-09-10T10:43:31.550868+0000","last_clean":"2024-09-10T10:43:31.550868+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:43:31.550868+0000","last_undegraded":"2024-09-10T10:43:31.550868+0000","last_fullsized":"2024-09-10T10:43:31.550868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289972+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:43:03.289972+0000","last_peered":"2024-09-10T10:43:03.289972+0000","last_clean":"2024-09-10T10:43:03.289972+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:43:03.289972+0000","last_undegraded":"2024-09-10T10:43:03.289972+0000","last_fullsized":"2024-09-10T10:43:03.289972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612185+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:43:03.612185+0000","last_peered":"2024-09-10T10:43:03.612185+0000","last_clean":"2024-09-10T10:43:03.612185+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:43:03.612185+0000","last_undegraded":"2024-09-10T10:43:03.612185+0000","last_fullsized":"2024-09-10T10:43:03.612185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551110+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:43:31.551110+0000","last_peered":"2024-09-10T10:43:31.551110+0000","last_clean":"2024-09-10T10:43:31.551110+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:43:31.551110+0000","last_undegraded":"2024-09-10T10:43:31.551110+0000","last_fullsized":"2024-09-10T10:43:31.551110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612319+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:43:03.612319+0000","last_peered":"2024-09-10T10:43:03.612319+0000","last_clean":"2024-09-10T10:43:03.612319+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:43:03.612319+0000","last_undegraded":"2024-09-10T10:43:03.612319+0000","last_fullsized":"2024-09-10T10:43:03.612319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611708+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:43:03.611708+0000","last_peered":"2024-09-10T10:43:03.611708+0000","last_clean":"2024-09-10T10:43:03.611708+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:43:03.611708+0000","last_undegraded":"2024-09-10T10:43:03.611708+0000","last_fullsized":"2024-09-10T10:43:03.611708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.531214+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:43:30.531214+0000","last_peered":"2024-09-10T10:43:30.531214+0000","last_clean":"2024-09-10T10:43:30.531214+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:43:30.531214+0000","last_undegraded":"2024-09-10T10:43:30.531214+0000","last_fullsized":"2024-09-10T10:43:30.531214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612182+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:43:03.612182+0000","last_peered":"2024-09-10T10:43:03.612182+0000","last_clean":"2024-09-10T10:43:03.612182+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:43:03.612182+0000","last_undegraded":"2024-09-10T10:43:03.612182+0000","last_fullsized":"2024-09-10T10:43:03.612182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612381+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:43:03.612381+0000","last_peered":"2024-09-10T10:43:03.612381+0000","last_clean":"2024-09-10T10:43:03.612381+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:43:03.612381+0000","last_undegraded":"2024-09-10T10:43:03.612381+0000","last_fullsized":"2024-09-10T10:43:03.612381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.551186+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:43:31.551186+0000","last_peered":"2024-09-10T10:43:31.551186+0000","last_clean":"2024-09-10T10:43:31.551186+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:43:31.551186+0000","last_undegraded":"2024-09-10T10:43:31.551186+0000","last_fullsized":"2024-09-10T10:43:31.551186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611587+0000","last_change":"2024-09-10T10:43:00.289800+0000","last_active":"2024-09-10T10:43:03.611587+0000","last_peered":"2024-09-10T10:43:03.611587+0000","last_clean":"2024-09-10T10:43:03.611587+0000","last_became_active":"2024-09-10T10:43:00.289440+0000","last_became_peered":"2024-09-10T10:43:00.289440+0000","last_unstale":"2024-09-10T10:43:03.611587+0000","last_undegraded":"2024-09-10T10:43:03.611587+0000","last_fullsized":"2024-09-10T10:43:03.611587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:02.145265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612508+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:43:03.612508+0000","last_peered":"2024-09-10T10:43:03.612508+0000","last_clean":"2024-09-10T10:43:03.612508+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:43:03.612508+0000","last_undegraded":"2024-09-10T10:43:03.612508+0000","last_fullsized":"2024-09-10T10:43:03.612508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:41.577579+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:43:41.577579+0000","last_peered":"2024-09-10T10:43:41.577579+0000","last_clean":"2024-09-10T10:43:41.577579+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:43:41.577579+0000","last_undegraded":"2024-09-10T10:43:41.577579+0000","last_fullsized":"2024-09-10T10:43:41.577579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612605+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:43:03.612605+0000","last_peered":"2024-09-10T10:43:03.612605+0000","last_clean":"2024-09-10T10:43:03.612605+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:43:03.612605+0000","last_undegraded":"2024-09-10T10:43:03.612605+0000","last_fullsized":"2024-09-10T10:43:03.612605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611858+0000","last_change":"2024-09-10T10:42:53.255587+0000","last_active":"2024-09-10T10:43:03.611858+0000","last_peered":"2024-09-10T10:43:03.611858+0000","last_clean":"2024-09-10T10:43:03.611858+0000","last_became_active":"2024-09-10T10:42:53.254961+0000","last_became_peered":"2024-09-10T10:42:53.254961+0000","last_unstale":"2024-09-10T10:43:03.611858+0000","last_undegraded":"2024-09-10T10:43:03.611858+0000","last_fullsized":"2024-09-10T10:43:03.611858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:04.319949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.550871+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:43:31.550871+0000","last_peered":"2024-09-10T10:43:31.550871+0000","last_clean":"2024-09-10T10:43:31.550871+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:43:31.550871+0000","last_undegraded":"2024-09-10T10:43:31.550871+0000","last_fullsized":"2024-09-10T10:43:31.550871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612342+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:43:03.612342+0000","last_peered":"2024-09-10T10:43:03.612342+0000","last_clean":"2024-09-10T10:43:03.612342+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:43:03.612342+0000","last_undegraded":"2024-09-10T10:43:03.612342+0000","last_fullsized":"2024-09-10T10:43:03.612342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611076+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:43:03.611076+0000","last_peered":"2024-09-10T10:43:03.611076+0000","last_clean":"2024-09-10T10:43:03.611076+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:43:03.611076+0000","last_undegraded":"2024-09-10T10:43:03.611076+0000","last_fullsized":"2024-09-10T10:43:03.611076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:32.558874+0000","last_change":"2024-09-10T10:42:51.225741+0000","last_active":"2024-09-10T10:43:32.558874+0000","last_peered":"2024-09-10T10:43:32.558874+0000","last_clean":"2024-09-10T10:43:32.558874+0000","last_became_active":"2024-09-10T10:42:51.225363+0000","last_became_peered":"2024-09-10T10:42:51.225363+0000","last_unstale":"2024-09-10T10:43:32.558874+0000","last_undegraded":"2024-09-10T10:43:32.558874+0000","last_fullsized":"2024-09-10T10:43:32.558874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:28.548670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612699+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:43:03.612699+0000","last_peered":"2024-09-10T10:43:03.612699+0000","last_clean":"2024-09-10T10:43:03.612699+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:43:03.612699+0000","last_undegraded":"2024-09-10T10:43:03.612699+0000","last_fullsized":"2024-09-10T10:43:03.612699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289187+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:43:03.289187+0000","last_peered":"2024-09-10T10:43:03.289187+0000","last_clean":"2024-09-10T10:43:03.289187+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:43:03.289187+0000","last_undegraded":"2024-09-10T10:43:03.289187+0000","last_fullsized":"2024-09-10T10:43:03.289187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:45.585016+0000","last_change":"2024-09-10T10:42:51.225026+0000","last_active":"2024-09-10T10:43:45.585016+0000","last_peered":"2024-09-10T10:43:45.585016+0000","last_clean":"2024-09-10T10:43:45.585016+0000","last_became_active":"2024-09-10T10:42:51.224678+0000","last_became_peered":"2024-09-10T10:42:51.224678+0000","last_unstale":"2024-09-10T10:43:45.585016+0000","last_undegraded":"2024-09-10T10:43:45.585016+0000","last_fullsized":"2024-09-10T10:43:45.585016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:24.154989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56673,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":73,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612239+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:43:03.612239+0000","last_peered":"2024-09-10T10:43:03.612239+0000","last_clean":"2024-09-10T10:43:03.612239+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:43:03.612239+0000","last_undegraded":"2024-09-10T10:43:03.612239+0000","last_fullsized":"2024-09-10T10:43:03.612239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427397+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:43:03.427397+0000","last_peered":"2024-09-10T10:43:03.427397+0000","last_clean":"2024-09-10T10:43:03.427397+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:43:03.427397+0000","last_undegraded":"2024-09-10T10:43:03.427397+0000","last_fullsized":"2024-09-10T10:43:03.427397+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.541927+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:43:30.541927+0000","last_peered":"2024-09-10T10:43:30.541927+0000","last_clean":"2024-09-10T10:43:30.541927+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:43:30.541927+0000","last_undegraded":"2024-09-10T10:43:30.541927+0000","last_fullsized":"2024-09-10T10:43:30.541927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290945+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:43:03.290945+0000","last_peered":"2024-09-10T10:43:03.290945+0000","last_clean":"2024-09-10T10:43:03.290945+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:43:03.290945+0000","last_undegraded":"2024-09-10T10:43:03.290945+0000","last_fullsized":"2024-09-10T10:43:03.290945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290294+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:43:03.290294+0000","last_peered":"2024-09-10T10:43:03.290294+0000","last_clean":"2024-09-10T10:43:03.290294+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:43:03.290294+0000","last_undegraded":"2024-09-10T10:43:03.290294+0000","last_fullsized":"2024-09-10T10:43:03.290294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.550624+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:43:31.550624+0000","last_peered":"2024-09-10T10:43:31.550624+0000","last_clean":"2024-09-10T10:43:31.550624+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:43:31.550624+0000","last_undegraded":"2024-09-10T10:43:31.550624+0000","last_fullsized":"2024-09-10T10:43:31.550624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611135+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:43:03.611135+0000","last_peered":"2024-09-10T10:43:03.611135+0000","last_clean":"2024-09-10T10:43:03.611135+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:43:03.611135+0000","last_undegraded":"2024-09-10T10:43:03.611135+0000","last_fullsized":"2024-09-10T10:43:03.611135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427585+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:43:03.427585+0000","last_peered":"2024-09-10T10:43:03.427585+0000","last_clean":"2024-09-10T10:43:03.427585+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:43:03.427585+0000","last_undegraded":"2024-09-10T10:43:03.427585+0000","last_fullsized":"2024-09-10T10:43:03.427585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:45.583173+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:43:45.583173+0000","last_peered":"2024-09-10T10:43:45.583173+0000","last_clean":"2024-09-10T10:43:45.583173+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:43:45.583173+0000","last_undegraded":"2024-09-10T10:43:45.583173+0000","last_fullsized":"2024-09-10T10:43:45.583173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611849+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:43:03.611849+0000","last_peered":"2024-09-10T10:43:03.611849+0000","last_clean":"2024-09-10T10:43:03.611849+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:43:03.611849+0000","last_undegraded":"2024-09-10T10:43:03.611849+0000","last_fullsized":"2024-09-10T10:43:03.611849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290094+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:43:03.290094+0000","last_peered":"2024-09-10T10:43:03.290094+0000","last_clean":"2024-09-10T10:43:03.290094+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:43:03.290094+0000","last_undegraded":"2024-09-10T10:43:03.290094+0000","last_fullsized":"2024-09-10T10:43:03.290094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612036+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:43:03.612036+0000","last_peered":"2024-09-10T10:43:03.612036+0000","last_clean":"2024-09-10T10:43:03.612036+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:43:03.612036+0000","last_undegraded":"2024-09-10T10:43:03.612036+0000","last_fullsized":"2024-09-10T10:43:03.612036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:31.552174+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:43:31.552174+0000","last_peered":"2024-09-10T10:43:31.552174+0000","last_clean":"2024-09-10T10:43:31.552174+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:43:31.552174+0000","last_undegraded":"2024-09-10T10:43:31.552174+0000","last_fullsized":"2024-09-10T10:43:31.552174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612366+0000","last_change":"2024-09-10T10:43:00.294591+0000","last_active":"2024-09-10T10:43:03.612366+0000","last_peered":"2024-09-10T10:43:03.612366+0000","last_clean":"2024-09-10T10:43:03.612366+0000","last_became_active":"2024-09-10T10:43:00.294194+0000","last_became_peered":"2024-09-10T10:43:00.294194+0000","last_unstale":"2024-09-10T10:43:03.612366+0000","last_undegraded":"2024-09-10T10:43:03.612366+0000","last_fullsized":"2024-09-10T10:43:03.612366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:48:40.230579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610945+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:43:03.610945+0000","last_peered":"2024-09-10T10:43:03.610945+0000","last_clean":"2024-09-10T10:43:03.610945+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:43:03.610945+0000","last_undegraded":"2024-09-10T10:43:03.610945+0000","last_fullsized":"2024-09-10T10:43:03.610945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:29.535069+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:43:29.535069+0000","last_peered":"2024-09-10T10:43:29.535069+0000","last_clean":"2024-09-10T10:43:29.535069+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:43:29.535069+0000","last_undegraded":"2024-09-10T10:43:29.535069+0000","last_fullsized":"2024-09-10T10:43:29.535069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:30.541280+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:43:30.541280+0000","last_peered":"2024-09-10T10:43:30.541280+0000","last_clean":"2024-09-10T10:43:30.541280+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:43:30.541280+0000","last_undegraded":"2024-09-10T10:43:30.541280+0000","last_fullsized":"2024-09-10T10:43:30.541280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611973+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:43:03.611973+0000","last_peered":"2024-09-10T10:43:03.611973+0000","last_clean":"2024-09-10T10:43:03.611973+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:43:03.611973+0000","last_undegraded":"2024-09-10T10:43:03.611973+0000","last_fullsized":"2024-09-10T10:43:03.611973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611661+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:43:03.611661+0000","last_peered":"2024-09-10T10:43:03.611661+0000","last_clean":"2024-09-10T10:43:03.611661+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:43:03.611661+0000","last_undegraded":"2024-09-10T10:43:03.611661+0000","last_fullsized":"2024-09-10T10:43:03.611661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:45.583784+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:43:45.583784+0000","last_peered":"2024-09-10T10:43:45.583784+0000","last_clean":"2024-09-10T10:43:45.583784+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:43:45.583784+0000","last_undegraded":"2024-09-10T10:43:45.583784+0000","last_fullsized":"2024-09-10T10:43:45.583784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288839+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:43:03.288839+0000","last_peered":"2024-09-10T10:43:03.288839+0000","last_clean":"2024-09-10T10:43:03.288839+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:43:03.288839+0000","last_undegraded":"2024-09-10T10:43:03.288839+0000","last_fullsized":"2024-09-10T10:43:03.288839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612373+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:43:03.612373+0000","last_peered":"2024-09-10T10:43:03.612373+0000","last_clean":"2024-09-10T10:43:03.612373+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:43:03.612373+0000","last_undegraded":"2024-09-10T10:43:03.612373+0000","last_fullsized":"2024-09-10T10:43:03.612373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612084+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:43:03.612084+0000","last_peered":"2024-09-10T10:43:03.612084+0000","last_clean":"2024-09-10T10:43:03.612084+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:43:03.612084+0000","last_undegraded":"2024-09-10T10:43:03.612084+0000","last_fullsized":"2024-09-10T10:43:03.612084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612019+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:43:03.612019+0000","last_peered":"2024-09-10T10:43:03.612019+0000","last_clean":"2024-09-10T10:43:03.612019+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:43:03.612019+0000","last_undegraded":"2024-09-10T10:43:03.612019+0000","last_fullsized":"2024-09-10T10:43:03.612019+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611359+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:43:03.611359+0000","last_peered":"2024-09-10T10:43:03.611359+0000","last_clean":"2024-09-10T10:43:03.611359+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:43:03.611359+0000","last_undegraded":"2024-09-10T10:43:03.611359+0000","last_fullsized":"2024-09-10T10:43:03.611359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289920+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:43:03.289920+0000","last_peered":"2024-09-10T10:43:03.289920+0000","last_clean":"2024-09-10T10:43:03.289920+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:43:03.289920+0000","last_undegraded":"2024-09-10T10:43:03.289920+0000","last_fullsized":"2024-09-10T10:43:03.289920+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611390+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:43:03.611390+0000","last_peered":"2024-09-10T10:43:03.611390+0000","last_clean":"2024-09-10T10:43:03.611390+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:43:03.611390+0000","last_undegraded":"2024-09-10T10:43:03.611390+0000","last_fullsized":"2024-09-10T10:43:03.611390+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611935+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:43:03.611935+0000","last_peered":"2024-09-10T10:43:03.611935+0000","last_clean":"2024-09-10T10:43:03.611935+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:43:03.611935+0000","last_undegraded":"2024-09-10T10:43:03.611935+0000","last_fullsized":"2024-09-10T10:43:03.611935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612155+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:43:03.612155+0000","last_peered":"2024-09-10T10:43:03.612155+0000","last_clean":"2024-09-10T10:43:03.612155+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:43:03.612155+0000","last_undegraded":"2024-09-10T10:43:03.612155+0000","last_fullsized":"2024-09-10T10:43:03.612155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611962+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:43:03.611962+0000","last_peered":"2024-09-10T10:43:03.611962+0000","last_clean":"2024-09-10T10:43:03.611962+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:43:03.611962+0000","last_undegraded":"2024-09-10T10:43:03.611962+0000","last_fullsized":"2024-09-10T10:43:03.611962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612637+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:43:03.612637+0000","last_peered":"2024-09-10T10:43:03.612637+0000","last_clean":"2024-09-10T10:43:03.612637+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:43:03.612637+0000","last_undegraded":"2024-09-10T10:43:03.612637+0000","last_fullsized":"2024-09-10T10:43:03.612637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610995+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:43:03.610995+0000","last_peered":"2024-09-10T10:43:03.610995+0000","last_clean":"2024-09-10T10:43:03.610995+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:43:03.610995+0000","last_undegraded":"2024-09-10T10:43:03.610995+0000","last_fullsized":"2024-09-10T10:43:03.610995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427569+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:43:03.427569+0000","last_peered":"2024-09-10T10:43:03.427569+0000","last_clean":"2024-09-10T10:43:03.427569+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:43:03.427569+0000","last_undegraded":"2024-09-10T10:43:03.427569+0000","last_fullsized":"2024-09-10T10:43:03.427569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611274+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:43:03.611274+0000","last_peered":"2024-09-10T10:43:03.611274+0000","last_clean":"2024-09-10T10:43:03.611274+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:43:03.611274+0000","last_undegraded":"2024-09-10T10:43:03.611274+0000","last_fullsized":"2024-09-10T10:43:03.611274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612398+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:43:03.612398+0000","last_peered":"2024-09-10T10:43:03.612398+0000","last_clean":"2024-09-10T10:43:03.612398+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:43:03.612398+0000","last_undegraded":"2024-09-10T10:43:03.612398+0000","last_fullsized":"2024-09-10T10:43:03.612398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612530+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:43:03.612530+0000","last_peered":"2024-09-10T10:43:03.612530+0000","last_clean":"2024-09-10T10:43:03.612530+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:43:03.612530+0000","last_undegraded":"2024-09-10T10:43:03.612530+0000","last_fullsized":"2024-09-10T10:43:03.612530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612414+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:43:03.612414+0000","last_peered":"2024-09-10T10:43:03.612414+0000","last_clean":"2024-09-10T10:43:03.612414+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:43:03.612414+0000","last_undegraded":"2024-09-10T10:43:03.612414+0000","last_fullsized":"2024-09-10T10:43:03.612414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612699+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:43:03.612699+0000","last_peered":"2024-09-10T10:43:03.612699+0000","last_clean":"2024-09-10T10:43:03.612699+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:43:03.612699+0000","last_undegraded":"2024-09-10T10:43:03.612699+0000","last_fullsized":"2024-09-10T10:43:03.612699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427021+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:43:03.427021+0000","last_peered":"2024-09-10T10:43:03.427021+0000","last_clean":"2024-09-10T10:43:03.427021+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:43:03.427021+0000","last_undegraded":"2024-09-10T10:43:03.427021+0000","last_fullsized":"2024-09-10T10:43:03.427021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289692+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:43:03.289692+0000","last_peered":"2024-09-10T10:43:03.289692+0000","last_clean":"2024-09-10T10:43:03.289692+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:43:03.289692+0000","last_undegraded":"2024-09-10T10:43:03.289692+0000","last_fullsized":"2024-09-10T10:43:03.289692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289723+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:43:03.289723+0000","last_peered":"2024-09-10T10:43:03.289723+0000","last_clean":"2024-09-10T10:43:03.289723+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:43:03.289723+0000","last_undegraded":"2024-09-10T10:43:03.289723+0000","last_fullsized":"2024-09-10T10:43:03.289723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611299+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:43:03.611299+0000","last_peered":"2024-09-10T10:43:03.611299+0000","last_clean":"2024-09-10T10:43:03.611299+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:43:03.611299+0000","last_undegraded":"2024-09-10T10:43:03.611299+0000","last_fullsized":"2024-09-10T10:43:03.611299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.291084+0000","last_change":"2024-09-10T10:42:53.252841+0000","last_active":"2024-09-10T10:43:03.291084+0000","last_peered":"2024-09-10T10:43:03.291084+0000","last_clean":"2024-09-10T10:43:03.291084+0000","last_became_active":"2024-09-10T10:42:53.252500+0000","last_became_peered":"2024-09-10T10:42:53.252500+0000","last_unstale":"2024-09-10T10:43:03.291084+0000","last_undegraded":"2024-09-10T10:43:03.291084+0000","last_fullsized":"2024-09-10T10:43:03.291084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:28.502382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612116+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:43:03.612116+0000","last_peered":"2024-09-10T10:43:03.612116+0000","last_clean":"2024-09-10T10:43:03.612116+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:43:03.612116+0000","last_undegraded":"2024-09-10T10:43:03.612116+0000","last_fullsized":"2024-09-10T10:43:03.612116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612215+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:43:03.612215+0000","last_peered":"2024-09-10T10:43:03.612215+0000","last_clean":"2024-09-10T10:43:03.612215+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:43:03.612215+0000","last_undegraded":"2024-09-10T10:43:03.612215+0000","last_fullsized":"2024-09-10T10:43:03.612215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.427716+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:43:03.427716+0000","last_peered":"2024-09-10T10:43:03.427716+0000","last_clean":"2024-09-10T10:43:03.427716+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:43:03.427716+0000","last_undegraded":"2024-09-10T10:43:03.427716+0000","last_fullsized":"2024-09-10T10:43:03.427716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.610880+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:43:03.610880+0000","last_peered":"2024-09-10T10:43:03.610880+0000","last_clean":"2024-09-10T10:43:03.610880+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:43:03.610880+0000","last_undegraded":"2024-09-10T10:43:03.610880+0000","last_fullsized":"2024-09-10T10:43:03.610880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611704+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:43:03.611704+0000","last_peered":"2024-09-10T10:43:03.611704+0000","last_clean":"2024-09-10T10:43:03.611704+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:43:03.611704+0000","last_undegraded":"2024-09-10T10:43:03.611704+0000","last_fullsized":"2024-09-10T10:43:03.611704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.611687+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:43:03.611687+0000","last_peered":"2024-09-10T10:43:03.611687+0000","last_clean":"2024-09-10T10:43:03.611687+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:43:03.611687+0000","last_undegraded":"2024-09-10T10:43:03.611687+0000","last_fullsized":"2024-09-10T10:43:03.611687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.288785+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:43:03.288785+0000","last_peered":"2024-09-10T10:43:03.288785+0000","last_clean":"2024-09-10T10:43:03.288785+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:43:03.288785+0000","last_undegraded":"2024-09-10T10:43:03.288785+0000","last_fullsized":"2024-09-10T10:43:03.288785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290977+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:43:03.290977+0000","last_peered":"2024-09-10T10:43:03.290977+0000","last_clean":"2024-09-10T10:43:03.290977+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:43:03.290977+0000","last_undegraded":"2024-09-10T10:43:03.290977+0000","last_fullsized":"2024-09-10T10:43:03.290977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.289933+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:43:03.289933+0000","last_peered":"2024-09-10T10:43:03.289933+0000","last_clean":"2024-09-10T10:43:03.289933+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:43:03.289933+0000","last_undegraded":"2024-09-10T10:43:03.289933+0000","last_fullsized":"2024-09-10T10:43:03.289933+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.290097+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:43:03.290097+0000","last_peered":"2024-09-10T10:43:03.290097+0000","last_clean":"2024-09-10T10:43:03.290097+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:43:03.290097+0000","last_undegraded":"2024-09-10T10:43:03.290097+0000","last_fullsized":"2024-09-10T10:43:03.290097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-10T10:43:03.612605+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:43:03.612605+0000","last_peered":"2024-09-10T10:43:03.612605+0000","last_clean":"2024-09-10T10:43:03.612605+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:43:03.612605+0000","last_undegraded":"2024-09-10T10:43:03.612605+0000","last_fullsized":"2024-09-10T10:43:03.612605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63881,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":174,"num_read_kb":159,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":481508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1124997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":485141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":481423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1077922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":476549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1073111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":481016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":476257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40544,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704608,"statfs":{"total":95995035648,"available":95953518592,"internally_reserved":0,"allocated":1064960,"data_stored":534865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":534252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":478398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:43:48.842 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:43:48.842 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T10:43:48.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T10:43:49.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:48 smithi143 ceph-mon[31057]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:43:49.091 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:49.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:48 smithi178 ceph-mon[34813]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:43:49.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:48 smithi144 ceph-mon[34230]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:43:49.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 -- 172.21.15.143:0/2445499851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0102690 msgr2=0x7f6be0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:49.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 --2- 172.21.15.143:0/2445499851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0102690 0x7f6be0108f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009ca0 tx=0x7f6bd802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 -- 172.21.15.143:0/2445499851 shutdown_connections 2024-09-10T10:43:49.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 --2- 172.21.15.143:0/2445499851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0102690 0x7f6be0108f20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 --2- 172.21.15.143:0/2445499851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6be0101cd0 0x7f6be0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 --2- 172.21.15.143:0/2445499851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be0100ad0 0x7f6be0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 -- 172.21.15.143:0/2445499851 >> 172.21.15.143:0/2445499851 conn(0x7f6be00fc280 msgr2=0x7f6be00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 -- 172.21.15.143:0/2445499851 shutdown_connections 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.582+0000 7f6be5004640 1 -- 172.21.15.143:0/2445499851 wait complete. 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 Processor -- start 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 -- start start 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 0x7f6be006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be0101cd0 0x7f6be019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6be0102690 0x7f6be01a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010c5e0 con 0x7f6be0100ad0 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010c460 con 0x7f6be0101cd0 2024-09-10T10:43:49.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6be5004640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010c760 con 0x7f6be0102690 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6bded9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 0x7f6be006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6bded9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 0x7f6be006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47166/0 (socket says 172.21.15.143:47166) 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6bded9d640 1 -- 172.21.15.143:0/711998432 learned_addr learned my addr 172.21.15.143:0/711998432 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6bdf59e640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6be0102690 0x7f6be01a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.583+0000 7f6bde59c640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be0101cd0 0x7f6be019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bded9d640 1 -- 172.21.15.143:0/711998432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6be0102690 msgr2=0x7f6be01a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bded9d640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6be0102690 0x7f6be01a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bded9d640 1 -- 172.21.15.143:0/711998432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be0101cd0 msgr2=0x7f6be019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bded9d640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be0101cd0 0x7f6be019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bded9d640 1 -- 172.21.15.143:0/711998432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be01a1c30 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bded9d640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 0x7f6be006aad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400d950 tx=0x7f6bd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4017020 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be01a1ec0 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bd4009c30 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd400f6b0 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.584+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be01a23c0 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.585+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be0107f20 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.586+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6bd401a020 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.587+0000 7f6bc7fff640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6bc80a2c40 0x7f6bc80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.587+0000 7f6bde59c640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6bc80a2c40 0x7f6bc80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.587+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f6bd40cefe0 con 0x7f6be0100ad0 2024-09-10T10:43:49.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.588+0000 7f6bde59c640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6bc80a2c40 0x7f6bc80a5100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6be006b050 tx=0x7f6bd000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:49.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.590+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bd4021070 con 0x7f6be0100ad0 2024-09-10T10:43:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.838+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6be002d7c0 con 0x7f6be0100ad0 2024-09-10T10:43:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.840+0000 7f6bc7fff640 1 -- 172.21.15.143:0/711998432 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f6bd409bba0 con 0x7f6be0100ad0 2024-09-10T10:43:49.842 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-09-10T10:43:49.842 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDlIuBmdGgIMhAADJ+fedUY/sm4jzBt5xAVKg== 2024-09-10T10:43:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6bc80a2c40 msgr2=0x7f6bc80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6bc80a2c40 0x7f6bc80a5100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6be006b050 tx=0x7f6bd000b390 comp rx=0 tx=0).stop 2024-09-10T10:43:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 msgr2=0x7f6be006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 0x7f6be006aad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400d950 tx=0x7f6bd400de10 comp rx=0 tx=0).stop 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 shutdown_connections 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6bc80a2c40 0x7f6bc80a5100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6be0102690 0x7f6be01a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be0101cd0 0x7f6be019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 --2- 172.21.15.143:0/711998432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6be0100ad0 0x7f6be006aad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 >> 172.21.15.143:0/711998432 conn(0x7f6be00fc280 msgr2=0x7f6be01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 shutdown_connections 2024-09-10T10:43:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:49.842+0000 7f6be5004640 1 -- 172.21.15.143:0/711998432 wait complete. 2024-09-10T10:43:50.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:49 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:50.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:49 smithi143 ceph-mon[31057]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T10:43:50.066 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:49 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:50.066 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:49 smithi144 ceph-mon[34230]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T10:43:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:49 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:50.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:49 smithi178 ceph-mon[34813]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T10:43:51.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:50 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/711998432' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:43:51.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:50 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/711998432' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:43:51.171 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:43:51.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T10:43:51.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T10:43:51.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:50 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/711998432' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:43:51.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:50 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/711998432' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:43:51.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:50 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/711998432' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:43:51.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:50 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/711998432' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:43:51.213 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T10:43:51.213 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T10:43:51.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T10:43:51.531 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:51.956 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:51 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.039+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3975935229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100480 msgr2=0x7f2f88100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.039+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3975935229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100480 0x7f2f88100860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009900 tx=0x7f2f8002a410 comp rx=0 tx=0).stop 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.039+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3975935229 shutdown_connections 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.039+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3975935229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f881055b0 0x7f2f8810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.039+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3975935229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f88100e30 0x7f2f88104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3975935229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100480 0x7f2f88100860 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3975935229 >> 172.21.15.143:0/3975935229 conn(0x7f2f88077210 msgr2=0x7f2f880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3975935229 shutdown_connections 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3975935229 wait complete. 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 Processor -- start 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 -- start start 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f88100480 0x7f2f88199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.040+0000 7f2f8d27a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 0x7f2f881997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f8d27a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f881055b0 0x7f2f88199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f8d27a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88110d40 con 0x7f2f88100e30 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f8d27a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88110bc0 con 0x7f2f881055b0 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f8d27a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88110ec0 con 0x7f2f88100480 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 0x7f2f881997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 0x7f2f881997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47174/0 (socket says 172.21.15.143:47174) 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 -- 172.21.15.143:0/3105027206 learned_addr learned my addr 172.21.15.143:0/3105027206 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f877fe640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f881055b0 0x7f2f88199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f86ffd640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f88100480 0x7f2f88199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 -- 172.21.15.143:0/3105027206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f88100480 msgr2=0x7f2f88199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f88100480 0x7f2f88199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 -- 172.21.15.143:0/3105027206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f881055b0 msgr2=0x7f2f88199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f881055b0 0x7f2f88199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 -- 172.21.15.143:0/3105027206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f881a4640 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f867fc640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 0x7f2f881997d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000d770 tx=0x7f2f7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70017070 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881a4930 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f700054d0 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70004cf0 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.041+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a4e70 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.042+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f88100860 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.043+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2f70004e90 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.044+0000 7f2f7ffff640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f740a2c10 0x7f2f740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.044+0000 7f2f86ffd640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f740a2c10 0x7f2f740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.044+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f2f700cedd0 con 0x7f2f88100e30 2024-09-10T10:43:52.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.045+0000 7f2f86ffd640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f740a2c10 0x7f2f740a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f80031440 tx=0x7f2f80005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:52.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.048+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f7009b340 con 0x7f2f88100e30 2024-09-10T10:43:52.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-10T10:43:52.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:51 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:51 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.260+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2f88108790 con 0x7f2f88100e30 2024-09-10T10:43:52.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.266+0000 7f2f7ffff640 1 -- 172.21.15.143:0/3105027206 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263209 (secure 0 0 0) 0x7f2f7009f640 con 0x7f2f88100e30 2024-09-10T10:43:52.270 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:52.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.268+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f740a2c10 msgr2=0x7f2f740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.268+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f740a2c10 0x7f2f740a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2f80031440 tx=0x7f2f80005b50 comp rx=0 tx=0).stop 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.268+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 msgr2=0x7f2f881997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.268+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 0x7f2f881997d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000d770 tx=0x7f2f7000dc30 comp rx=0 tx=0).stop 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 shutdown_connections 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f740a2c10 0x7f2f740a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f881055b0 0x7f2f88199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f88100e30 0x7f2f881997d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 --2- 172.21.15.143:0/3105027206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f88100480 0x7f2f88199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 >> 172.21.15.143:0/3105027206 conn(0x7f2f88077210 msgr2=0x7f2f88103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 shutdown_connections 2024-09-10T10:43:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:52.269+0000 7f2f8d27a640 1 -- 172.21.15.143:0/3105027206 wait complete. 2024-09-10T10:43:52.955 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":1095035473},{"type":"v1","addr":"172.21.15.143:6801","nonce":1095035473}]},"active_addr":"172.21.15.143:6801/1095035473","active_change":"2024-09-10T10:35:59.338398+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.143:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2758892857}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1417438014}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2940924984}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":507656575}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3424083089}]}]} 2024-09-10T10:43:52.962 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T10:43:52.962 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T10:43:52.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:53.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:53.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:53.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:53.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:52 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3105027206' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:43:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:52 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3105027206' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:43:53.209 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:53.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:53.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:53.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:43:53.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:52 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3105027206' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 -- 172.21.15.143:0/1252341408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c101de0 msgr2=0x7f661c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 --2- 172.21.15.143:0/1252341408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c101de0 0x7f661c1021c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6618009900 tx=0x7f661802a410 comp rx=0 tx=0).stop 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 -- 172.21.15.143:0/1252341408 shutdown_connections 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 --2- 172.21.15.143:0/1252341408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c106f10 0x7f661c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 --2- 172.21.15.143:0/1252341408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f661c102790 0x7f661c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 --2- 172.21.15.143:0/1252341408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c101de0 0x7f661c1021c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 -- 172.21.15.143:0/1252341408 >> 172.21.15.143:0/1252341408 conn(0x7f661c0fd8c0 msgr2=0x7f661c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 -- 172.21.15.143:0/1252341408 shutdown_connections 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.700+0000 7f66240a2640 1 -- 172.21.15.143:0/1252341408 wait complete. 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 Processor -- start 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 -- start start 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f661c101de0 0x7f661c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c102790 0x7f661c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 0x7f661c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f661c111eb0 con 0x7f661c106f10 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f661c111d30 con 0x7f661c101de0 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f66240a2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f661c112030 con 0x7f661c102790 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f662263f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 0x7f661c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f662263f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 0x7f661c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47202/0 (socket says 172.21.15.143:47202) 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f662263f640 1 -- 172.21.15.143:0/1098538939 learned_addr learned my addr 172.21.15.143:0/1098538939 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f6621e3e640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f661c101de0 0x7f661c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.701+0000 7f662163d640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c102790 0x7f661c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662263f640 1 -- 172.21.15.143:0/1098538939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c102790 msgr2=0x7f661c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662263f640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c102790 0x7f661c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662263f640 1 -- 172.21.15.143:0/1098538939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f661c101de0 msgr2=0x7f661c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662263f640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f661c101de0 0x7f661c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662263f640 1 -- 172.21.15.143:0/1098538939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c19d3a0 con 0x7f661c106f10 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662163d640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c102790 0x7f661c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f662263f640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 0x7f661c19ccc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f661400eaa0 tx=0x7f661400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661400cdb0 con 0x7f661c106f10 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6614005020 con 0x7f661c106f10 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c19d690 con 0x7f661c106f10 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66140116d0 con 0x7f661c106f10 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.702+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c1a4f70 con 0x7f661c106f10 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.703+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661c1021c0 con 0x7f661c106f10 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.704+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f66140051d0 con 0x7f661c106f10 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.705+0000 7f6612ffd640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66080a2c10 0x7f66080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.705+0000 7f6621e3e640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66080a2c10 0x7f66080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.705+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f66140cf990 con 0x7f661c106f10 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.705+0000 7f6621e3e640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66080a2c10 0x7f66080a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6618005ab0 tx=0x7f6618005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.708+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6614016030 con 0x7f661c106f10 2024-09-10T10:43:53.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.877+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f661c10a150 con 0x7f661c106f10 2024-09-10T10:43:53.878 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:53 smithi143 ceph-mon[31057]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-10T10:43:53.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.878+0000 7f6612ffd640 1 -- 172.21.15.143:0/1098538939 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7f661409c550 con 0x7f661c106f10 2024-09-10T10:43:53.880 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:53.880 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66080a2c10 msgr2=0x7f66080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66080a2c10 0x7f66080a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6618005ab0 tx=0x7f6618005a20 comp rx=0 tx=0).stop 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 msgr2=0x7f661c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 0x7f661c19ccc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f661400eaa0 tx=0x7f661400ef60 comp rx=0 tx=0).stop 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 shutdown_connections 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f66080a2c10 0x7f66080a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f661c106f10 0x7f661c19ccc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c102790 0x7f661c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 --2- 172.21.15.143:0/1098538939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f661c101de0 0x7f661c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 >> 172.21.15.143:0/1098538939 conn(0x7f661c0fd8c0 msgr2=0x7f661c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 shutdown_connections 2024-09-10T10:43:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:53.881+0000 7f66240a2640 1 -- 172.21.15.143:0/1098538939 wait complete. 2024-09-10T10:43:54.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:53 smithi178 ceph-mon[34813]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-10T10:43:54.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:53 smithi144 ceph-mon[34230]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-10T10:43:54.615 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T10:43:54.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:54.861 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:54.982 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:54 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1098538939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:55.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:54 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1098538939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:55.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:54 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1098538939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 -- 172.21.15.143:0/2206282240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 msgr2=0x7ff100106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 --2- 172.21.15.143:0/2206282240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff100106860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009930 tx=0x7ff0f402a420 comp rx=0 tx=0).stop 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 -- 172.21.15.143:0/2206282240 shutdown_connections 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 --2- 172.21.15.143:0/2206282240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff100106fc0 0x7ff10010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 --2- 172.21.15.143:0/2206282240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff100106860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 --2- 172.21.15.143:0/2206282240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 0x7ff100100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 -- 172.21.15.143:0/2206282240 >> 172.21.15.143:0/2206282240 conn(0x7ff100077210 msgr2=0x7ff1000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 -- 172.21.15.143:0/2206282240 shutdown_connections 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 -- 172.21.15.143:0/2206282240 wait complete. 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.326+0000 7ff108316640 1 Processor -- start 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 -- start start 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 0x7ff100198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff1001988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff100106fc0 0x7ff10019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111e70 con 0x7ff100100d20 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111cf0 con 0x7ff100106fc0 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff108316640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111ff0 con 0x7ff100100370 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff1001988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff1001988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47216/0 (socket says 172.21.15.143:47216) 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 -- 172.21.15.143:0/493717480 learned_addr learned my addr 172.21.15.143:0/493717480 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1060b2640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 0x7ff100198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1068b3640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff100106fc0 0x7ff10019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 -- 172.21.15.143:0/493717480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 msgr2=0x7ff100198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 0x7ff100198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 -- 172.21.15.143:0/493717480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff100106fc0 msgr2=0x7ff10019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff100106fc0 0x7ff10019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 -- 172.21.15.143:0/493717480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10019d320 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1060b2640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 0x7ff100198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.327+0000 7ff1058b1640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff1001988b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009230 tx=0x7ff0f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.328+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f403f030 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.328+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10019d5b0 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.328+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f4003aa0 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.328+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4035070 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.328+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001a4ef0 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.329+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff100103430 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.330+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff0f4042030 con 0x7ff100100d20 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.331+0000 7ff0ff7fe640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff0e80a2c10 0x7ff0e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.331+0000 7ff1060b2640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff0e80a2c10 0x7ff0e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.331+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7ff0f40ef9e0 con 0x7ff100100d20 2024-09-10T10:43:55.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.331+0000 7ff1060b2640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff0e80a2c10 0x7ff0e80a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009fa0 tx=0x7ff0f8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.335+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0f40bc5a0 con 0x7ff100100d20 2024-09-10T10:43:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.505+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff100100750 con 0x7ff100100d20 2024-09-10T10:43:55.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.507+0000 7ff0ff7fe640 1 -- 172.21.15.143:0/493717480 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24307 (secure 0 0 0) 0x7ff100100750 con 0x7ff100100d20 2024-09-10T10:43:55.509 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:43:55.509 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:43:03.101055+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff0e80a2c10 msgr2=0x7ff0e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff0e80a2c10 0x7ff0e80a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009fa0 tx=0x7ff0f8009570 comp rx=0 tx=0).stop 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 msgr2=0x7ff1001988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff1001988b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009230 tx=0x7ff0f4004270 comp rx=0 tx=0).stop 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 shutdown_connections 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff0e80a2c10 0x7ff0e80a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff100106fc0 0x7ff10019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff100100d20 0x7ff1001988b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 --2- 172.21.15.143:0/493717480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff100100370 0x7ff100198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 >> 172.21.15.143:0/493717480 conn(0x7ff100077210 msgr2=0x7ff1000fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 shutdown_connections 2024-09-10T10:43:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:55.510+0000 7ff108316640 1 -- 172.21.15.143:0/493717480 wait complete. 2024-09-10T10:43:56.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:55 smithi178 ceph-mon[34813]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-10T10:43:56.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:55 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/493717480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:56.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:55 smithi144 ceph-mon[34230]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-10T10:43:56.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:55 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/493717480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:56.202 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:55 smithi143 ceph-mon[31057]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-10T10:43:56.203 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:55 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/493717480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:56.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T10:43:56.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T10:43:56.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T10:43:56.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T10:43:56.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T10:43:56.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T10:43:56.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T10:43:56.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T10:43:56.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T10:43:56.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T10:43:56.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T10:43:56.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T10:43:56.877 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:56.977 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:56.982 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.513 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.514 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.529 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.529 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.546 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.546 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.548 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.548 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.723+0000 7fb179a7c640 1 -- 172.21.15.143:0/286392376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741036e0 msgr2=0x7fb174107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.723+0000 7fb179a7c640 1 --2- 172.21.15.143:0/286392376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741036e0 0x7fb174107b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb168009930 tx=0x7fb16802a420 comp rx=0 tx=0).stop 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.723+0000 7fb179a7c640 1 -- 172.21.15.143:0/286392376 shutdown_connections 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.723+0000 7fb179a7c640 1 --2- 172.21.15.143:0/286392376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1741082a0 0x7fb17410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.723+0000 7fb179a7c640 1 --2- 172.21.15.143:0/286392376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741036e0 0x7fb174107b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.724+0000 7fb179a7c640 1 --2- 172.21.15.143:0/286392376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb174102d30 0x7fb174103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.724+0000 7fb179a7c640 1 -- 172.21.15.143:0/286392376 >> 172.21.15.143:0/286392376 conn(0x7fb1740fc7d0 msgr2=0x7fb1740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.724+0000 7fb179a7c640 1 -- 172.21.15.143:0/286392376 shutdown_connections 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.724+0000 7fb179a7c640 1 -- 172.21.15.143:0/286392376 wait complete. 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.724+0000 7fb179a7c640 1 Processor -- start 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.724+0000 7fb179a7c640 1 -- start start 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174102d30 0x7fb174100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1741036e0 0x7fb174100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741082a0 0x7fb174101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb174111f60 con 0x7fb1741082a0 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb174111de0 con 0x7fb1741036e0 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb1741120e0 con 0x7fb174102d30 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741082a0 0x7fb174101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741082a0 0x7fb174101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34588/0 (socket says 172.21.15.143:34588) 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 -- 172.21.15.143:0/2068470404 learned_addr learned my addr 172.21.15.143:0/2068470404 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb172ffd640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1741036e0 0x7fb174100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb1737fe640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174102d30 0x7fb174100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 -- 172.21.15.143:0/2068470404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174102d30 msgr2=0x7fb174100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174102d30 0x7fb174100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 -- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1741036e0 msgr2=0x7fb174100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1741036e0 0x7fb174100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 -- 172.21.15.143:0/2068470404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1741a69c0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb173fff640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741082a0 0x7fb174101410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00eaa0 tx=0x7fb16c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00cdb0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb16c005020 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741a6cb0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0116d0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.725+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741a71c0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.726+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1740628c0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.727+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb16c0051d0 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.728+0000 7fb170ff9640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.728+0000 7fb1737fe640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.728+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fb16c0cf860 con 0x7fb1741082a0 2024-09-10T10:43:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.728+0000 7fb170ff9640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] conn(0x7fb15c0ad940 0x7fb15c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.728+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 --> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb16c014ec0 con 0x7fb15c0ad940 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.728+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb16c0cfcb0 con 0x7fb1741082a0 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.729+0000 7fb1737fe640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb16400bbe0 tx=0x7fb16400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.729+0000 7fb172ffd640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] conn(0x7fb15c0ad940 0x7fb15c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.729+0000 7fb172ffd640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] conn(0x7fb15c0ad940 0x7fb15c0afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.730+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== osd.1 v2:172.21.15.144:6800/3635948930 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb16c014ec0 con 0x7fb15c0ad940 2024-09-10T10:43:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.743+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 --> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb174062d40 con 0x7fb15c0ad940 2024-09-10T10:43:57.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb170ff9640 1 -- 172.21.15.143:0/2068470404 <== osd.1 v2:172.21.15.144:6800/3635948930 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb174062d40 con 0x7fb15c0ad940 2024-09-10T10:43:57.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] conn(0x7fb15c0ad940 msgr2=0x7fb15c0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb179a7c640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] conn(0x7fb15c0ad940 0x7fb15c0afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb15c0a2c10 msgr2=0x7fb15c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb179a7c640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb16400bbe0 tx=0x7fb16400b390 comp rx=0 tx=0).stop 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741082a0 msgr2=0x7fb174101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb179a7c640 1 --2- 172.21.15.143:0/2068470404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1741082a0 0x7fb174101410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00eaa0 tx=0x7fb16c00ef60 comp rx=0 tx=0).stop 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.744+0000 7fb173fff640 1 -- 172.21.15.143:0/2068470404 reap_dead start 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.745+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 shutdown_connections 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.745+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 >> 172.21.15.143:0/2068470404 conn(0x7fb1740fc7d0 msgr2=0x7fb174105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.745+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 shutdown_connections 2024-09-10T10:43:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:57.745+0000 7fb179a7c640 1 -- 172.21.15.143:0/2068470404 wait complete. 2024-09-10T10:43:57.750 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:58.477 INFO:teuthology.orchestra.run.smithi143.stdout:55834574926 2024-09-10T10:43:58.478 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T10:43:58.703 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:58 smithi143 ceph-mon[31057]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T10:43:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:58 smithi178 ceph-mon[34813]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T10:43:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:58 smithi144 ceph-mon[34230]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 -- 172.21.15.143:0/2214958298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 msgr2=0x7fd5b0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 --2- 172.21.15.143:0/2214958298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b0101200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009900 tx=0x7fd5a002a480 comp rx=0 tx=0).stop 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 -- 172.21.15.143:0/2214958298 shutdown_connections 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 --2- 172.21.15.143:0/2214958298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5b0102ef0 0x7fd5b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 --2- 172.21.15.143:0/2214958298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0102570 0x7fd5b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 --2- 172.21.15.143:0/2214958298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b0101200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 -- 172.21.15.143:0/2214958298 >> 172.21.15.143:0/2214958298 conn(0x7fd5b00fc7d0 msgr2=0x7fd5b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 -- 172.21.15.143:0/2214958298 shutdown_connections 2024-09-10T10:43:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.482+0000 7fd5b6286640 1 -- 172.21.15.143:0/2214958298 wait complete. 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 Processor -- start 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 -- start start 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0102570 0x7fd5b01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5b0102ef0 0x7fd5b01a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0110e30 con 0x7fd5b0100e20 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0110cb0 con 0x7fd5b0102570 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b6286640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0110fb0 con 0x7fd5b0102ef0 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34626/0 (socket says 172.21.15.143:34626) 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 -- 172.21.15.143:0/2308056572 learned_addr learned my addr 172.21.15.143:0/2308056572 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:59.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5a7fff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0102570 0x7fd5b01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5b4823640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5b0102ef0 0x7fd5b01a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 -- 172.21.15.143:0/2308056572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5b0102ef0 msgr2=0x7fd5b01a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5b0102ef0 0x7fd5b01a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 -- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0102570 msgr2=0x7fd5b01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0102570 0x7fd5b01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.483+0000 7fd5affff640 1 -- 172.21.15.143:0/2308056572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01a4860 con 0x7fd5b0100e20 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5a7fff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0102570 0x7fd5b01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5affff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b01992a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006010 tx=0x7fd5a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a003f030 con 0x7fd5b0100e20 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5b6286640 1 -- 172.21.15.143:0/2308056572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01a4a30 con 0x7fd5b0100e20 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5a0003cb0 con 0x7fd5b0100e20 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0035070 con 0x7fd5b0100e20 2024-09-10T10:43:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.484+0000 7fd5b6286640 1 -- 172.21.15.143:0/2308056572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01a4ee0 con 0x7fd5b0100e20 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.485+0000 7fd5b6286640 1 -- 172.21.15.143:0/2308056572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5b0062940 con 0x7fd5b0100e20 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.486+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd5a0042030 con 0x7fd5b0100e20 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.487+0000 7fd5adffb640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.487+0000 7fd5a7fff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.488+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fd5a00efd20 con 0x7fd5b0100e20 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.488+0000 7fd5adffb640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] conn(0x7fd59c0ad940 0x7fd59c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.488+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 --> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd59c0b0450 con 0x7fd59c0ad940 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.488+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fd5a00f00d0 con 0x7fd5b0100e20 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.488+0000 7fd5a7fff640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd5b006aac0 tx=0x7fd59800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.488+0000 7fd5b4823640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] conn(0x7fd59c0ad940 0x7fd59c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.489+0000 7fd5b4823640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] conn(0x7fd59c0ad940 0x7fd59c0afd80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.489+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== osd.7 v2:172.21.15.144:6816/3746149853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd59c0b0450 con 0x7fd59c0ad940 2024-09-10T10:43:59.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.520+0000 7fd5b6286640 1 -- 172.21.15.143:0/2308056572 --> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5b0062d70 con 0x7fd59c0ad940 2024-09-10T10:43:59.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.520+0000 7fd5adffb640 1 -- 172.21.15.143:0/2308056572 <== osd.7 v2:172.21.15.144:6816/3746149853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd5b0062d70 con 0x7fd59c0ad940 2024-09-10T10:43:59.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] conn(0x7fd59c0ad940 msgr2=0x7fd59c0afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5a6ffd640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] conn(0x7fd59c0ad940 0x7fd59c0afd80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd59c0a2c10 msgr2=0x7fd59c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5a6ffd640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd5b006aac0 tx=0x7fd59800b390 comp rx=0 tx=0).stop 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 msgr2=0x7fd5b01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5a6ffd640 1 --2- 172.21.15.143:0/2308056572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5b0100e20 0x7fd5b01992a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006010 tx=0x7fd5a00042f0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.521+0000 7fd5b4823640 1 -- 172.21.15.143:0/2308056572 reap_dead start 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.522+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 shutdown_connections 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.522+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 >> 172.21.15.143:0/2308056572 conn(0x7fd5b00fc7d0 msgr2=0x7fd5b00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.523+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 shutdown_connections 2024-09-10T10:43:59.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.523+0000 7fd5a6ffd640 1 -- 172.21.15.143:0/2308056572 wait complete. 2024-09-10T10:43:59.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 -- 172.21.15.143:0/3582435692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 msgr2=0x7f80d010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 --2- 172.21.15.143:0/3582435692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d010eaa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009ca0 tx=0x7f80c802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 -- 172.21.15.143:0/3582435692 shutdown_connections 2024-09-10T10:43:59.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 --2- 172.21.15.143:0/3582435692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d010eaa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 --2- 172.21.15.143:0/3582435692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d01027a0 0x7f80d01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 --2- 172.21.15.143:0/3582435692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d0101df0 0x7f80d01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 -- 172.21.15.143:0/3582435692 >> 172.21.15.143:0/3582435692 conn(0x7f80d00fd8b0 msgr2=0x7f80d00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 -- 172.21.15.143:0/3582435692 shutdown_connections 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 -- 172.21.15.143:0/3582435692 wait complete. 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.364+0000 7f80d7a07640 1 Processor -- start 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 -- start start 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d0101df0 0x7f80d0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d01027a0 0x7f80d0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f80d0111e30 con 0x7f80d0106f20 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80d0111cb0 con 0x7f80d01027a0 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d7a07640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80d0111fb0 con 0x7f80d0101df0 2024-09-10T10:43:59.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34610/0 (socket says 172.21.15.143:34610) 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 -- 172.21.15.143:0/2211544916 learned_addr learned my addr 172.21.15.143:0/2211544916 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d57a3640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d0101df0 0x7f80d0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d4fa2640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d01027a0 0x7f80d0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 -- 172.21.15.143:0/2211544916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d0101df0 msgr2=0x7f80d0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d0101df0 0x7f80d0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 -- 172.21.15.143:0/2211544916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d01027a0 msgr2=0x7f80d0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d01027a0 0x7f80d0198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 -- 172.21.15.143:0/2211544916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d019d300 con 0x7f80d0106f20 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.365+0000 7f80d5fa4640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d019cc20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009dc0 tx=0x7f80c8032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.366+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003cf0 con 0x7f80d0106f20 2024-09-10T10:43:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.366+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80c80044f0 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.366+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d019d590 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.366+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8030bf0 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.366+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d01a4ed0 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.367+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80d0062900 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.368+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f80c8004040 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.368+0000 7f80c67fc640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f80bc0a2c90 0x7f80bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80d57a3640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f80bc0a2c90 0x7f80bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f80c80f0ad0 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80c67fc640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] conn(0x7f80bc0ad9c0 0x7f80bc0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 --> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80c800b150 con 0x7f80bc0ad9c0 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f80c80f0f20 con 0x7f80d0106f20 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80d4fa2640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] conn(0x7f80bc0ad9c0 0x7f80bc0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80d57a3640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f80bc0a2c90 0x7f80bc0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00bbe0 tx=0x7f80cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.369+0000 7f80d4fa2640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] conn(0x7f80bc0ad9c0 0x7f80bc0afe00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.370+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== osd.0 v2:172.21.15.143:6802/3953963737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80c800b150 con 0x7f80bc0ad9c0 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.385+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 --> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80d0062d00 con 0x7f80bc0ad9c0 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.385+0000 7f80c67fc640 1 -- 172.21.15.143:0/2211544916 <== osd.0 v2:172.21.15.143:6802/3953963737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f80d0062d00 con 0x7f80bc0ad9c0 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] conn(0x7f80bc0ad9c0 msgr2=0x7f80bc0afe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] conn(0x7f80bc0ad9c0 0x7f80bc0afe00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f80bc0a2c90 msgr2=0x7f80bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f80bc0a2c90 0x7f80bc0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00bbe0 tx=0x7f80cc00b390 comp rx=0 tx=0).stop 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 msgr2=0x7f80d019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 --2- 172.21.15.143:0/2211544916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f80d0106f20 0x7f80d019cc20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009dc0 tx=0x7f80c8032ff0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d5fa4640 1 -- 172.21.15.143:0/2211544916 reap_dead start 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 shutdown_connections 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 >> 172.21.15.143:0/2211544916 conn(0x7f80d00fd8b0 msgr2=0x7f80d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 shutdown_connections 2024-09-10T10:43:59.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:43:59.386+0000 7f80d7a07640 1 -- 172.21.15.143:0/2211544916 wait complete. 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T10:43:59.623 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-10T10:43:59.624 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-09-10T10:43:59.624 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:43:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T10:43:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-09-10T10:43:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:43:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T10:43:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-10T10:43:59.981 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-10T10:43:59.981 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-10T10:43:59.981 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-10T10:43:59.981 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-10T10:43:59.981 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:43:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T10:44:00.489 INFO:teuthology.orchestra.run.smithi143.stdout:34359738451 2024-09-10T10:44:00.489 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-10T10:44:00.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T10:44:00.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T10:44:00.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-10T10:44:00.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-09-10T10:44:00.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T10:44:00.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:00 smithi143 ceph-mon[31057]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-09-10T10:44:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T10:44:00.926 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:00 smithi178 ceph-mon[34813]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T10:44:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-10T10:44:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-09-10T10:44:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T10:44:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:00 smithi144 ceph-mon[34230]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:44:01.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 -- 172.21.15.143:0/3448588522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3720102d40 msgr2=0x7f3720103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 --2- 172.21.15.143:0/3448588522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3720102d40 0x7f3720103120 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f371c009ca0 tx=0x7f371c02a950 comp rx=0 tx=0).stop 2024-09-10T10:44:01.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 -- 172.21.15.143:0/3448588522 shutdown_connections 2024-09-10T10:44:01.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 --2- 172.21.15.143:0/3448588522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f372010c170 0x7f372010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 --2- 172.21.15.143:0/3448588522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3720103660 0x7f372010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 --2- 172.21.15.143:0/3448588522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3720102d40 0x7f3720103120 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.220+0000 7f3726b1a640 1 -- 172.21.15.143:0/3448588522 >> 172.21.15.143:0/3448588522 conn(0x7f37200fc7a0 msgr2=0x7f37200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.221+0000 7f3726b1a640 1 -- 172.21.15.143:0/3448588522 shutdown_connections 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.221+0000 7f3726b1a640 1 -- 172.21.15.143:0/3448588522 wait complete. 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.221+0000 7f3726b1a640 1 Processor -- start 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 -- start start 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3720102d40 0x7f37201009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3720103660 0x7f3720100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f372010c170 0x7f3720101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3720111e20 con 0x7f372010c170 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3720111ca0 con 0x7f3720102d40 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3726b1a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3720111fa0 con 0x7f3720103660 2024-09-10T10:44:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f372010c170 0x7f3720101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f372010c170 0x7f3720101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34690/0 (socket says 172.21.15.143:34690) 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 -- 172.21.15.143:0/2612713584 learned_addr learned my addr 172.21.15.143:0/2612713584 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37248b6640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3720102d40 0x7f37201009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3717fff640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3720103660 0x7f3720100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 -- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3720103660 msgr2=0x7f3720100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3720103660 0x7f3720100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 -- 172.21.15.143:0/2612713584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3720102d40 msgr2=0x7f37201009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3720102d40 0x7f37201009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 -- 172.21.15.143:0/2612713584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37201a6830 con 0x7f372010c170 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f37250b7640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f372010c170 0x7f3720101420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f371800e970 tx=0x7f371800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.222+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371800ccc0 con 0x7f372010c170 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.223+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37201a6b20 con 0x7f372010c170 2024-09-10T10:44:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.223+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371800ce60 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.223+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718010430 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.223+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37201a70b0 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.224+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37200628c0 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.225+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3718004f50 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.225+0000 7f3715ffb640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f37080a2c10 0x7f37080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.225+0000 7f37248b6640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f37080a2c10 0x7f37080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.225+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f37180cfcf0 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.226+0000 7f3715ffb640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] conn(0x7f37080adde0 0x7f37080b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.226+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 --> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37080b0950 con 0x7f37080adde0 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.226+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f37180d0140 con 0x7f372010c170 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.226+0000 7f37248b6640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f37080a2c10 0x7f37080a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f371c004ad0 tx=0x7f371c004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.226+0000 7f3717fff640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] conn(0x7f37080adde0 0x7f37080b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.226+0000 7f3717fff640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] conn(0x7f37080adde0 0x7f37080b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.227+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== osd.2 v2:172.21.15.178:6800/3522332994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37080b0950 con 0x7f37080adde0 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.240+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 --> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3720062cc0 con 0x7f37080adde0 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3715ffb640 1 -- 172.21.15.143:0/2612713584 <== osd.2 v2:172.21.15.178:6800/3522332994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3720062cc0 con 0x7f37080adde0 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] conn(0x7f37080adde0 msgr2=0x7f37080b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] conn(0x7f37080adde0 0x7f37080b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f37080a2c10 msgr2=0x7f37080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f37080a2c10 0x7f37080a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f371c004ad0 tx=0x7f371c004a60 comp rx=0 tx=0).stop 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f372010c170 msgr2=0x7f3720101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 --2- 172.21.15.143:0/2612713584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f372010c170 0x7f3720101420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f371800e970 tx=0x7f371800ed70 comp rx=0 tx=0).stop 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f37250b7640 1 -- 172.21.15.143:0/2612713584 reap_dead start 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 shutdown_connections 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.241+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 >> 172.21.15.143:0/2612713584 conn(0x7f37200fc7a0 msgr2=0x7f37200fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.242+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 shutdown_connections 2024-09-10T10:44:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.242+0000 7f3726b1a640 1 -- 172.21.15.143:0/2612713584 wait complete. 2024-09-10T10:44:01.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:01 smithi143 ceph-mon[31057]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:44:01.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 -- 172.21.15.143:0/1032276405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc102d40 msgr2=0x7ff2bc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 --2- 172.21.15.143:0/1032276405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc102d40 0x7ff2bc103120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002a480 comp rx=0 tx=0).stop 2024-09-10T10:44:01.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 -- 172.21.15.143:0/1032276405 shutdown_connections 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 --2- 172.21.15.143:0/1032276405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc10c170 0x7ff2bc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 --2- 172.21.15.143:0/1032276405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc103660 0x7ff2bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 --2- 172.21.15.143:0/1032276405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc102d40 0x7ff2bc103120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.084+0000 7ff2c4678640 1 -- 172.21.15.143:0/1032276405 >> 172.21.15.143:0/1032276405 conn(0x7ff2bc0fc7a0 msgr2=0x7ff2bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 -- 172.21.15.143:0/1032276405 shutdown_connections 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 -- 172.21.15.143:0/1032276405 wait complete. 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 Processor -- start 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 -- start start 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc102d40 0x7ff2bc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc103660 0x7ff2bc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc10c170 0x7ff2bc19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc111e20 con 0x7ff2bc103660 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc111ca0 con 0x7ff2bc10c170 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c4678640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc111fa0 con 0x7ff2bc102d40 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.085+0000 7ff2c1c13640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc103660 0x7ff2bc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc103660 0x7ff2bc198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34648/0 (socket says 172.21.15.143:34648) 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 -- 172.21.15.143:0/2229737411 learned_addr learned my addr 172.21.15.143:0/2229737411 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c2414640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc102d40 0x7ff2bc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c2c15640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc10c170 0x7ff2bc19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 -- 172.21.15.143:0/2229737411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc102d40 msgr2=0x7ff2bc198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc102d40 0x7ff2bc198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 -- 172.21.15.143:0/2229737411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc10c170 msgr2=0x7ff2bc19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc10c170 0x7ff2bc19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 -- 172.21.15.143:0/2229737411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc19d2d0 con 0x7ff2bc103660 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c2414640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2bc102d40 0x7ff2bc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:01.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c1c13640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc103660 0x7ff2bc198860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800c970 tx=0x7ff2b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8007bb0 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc19d5c0 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2b8007d50 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8004e10 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.086+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc1008d0 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.087+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff2bc062980 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.088+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff2b8005000 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2af7fe640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2a80a2c10 0x7ff2a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2c2414640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2a80a2c10 0x7ff2a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff2b80cf7a0 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2af7fe640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] conn(0x7ff2a80adde0 0x7ff2a80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 --> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2b800fe50 con 0x7ff2a80adde0 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff2b80cfbf0 con 0x7ff2bc103660 2024-09-10T10:44:01.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2c2c15640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] conn(0x7ff2a80adde0 0x7ff2a80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.089+0000 7ff2c2414640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2a80a2c10 0x7ff2a80a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0031440 tx=0x7ff2b00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.090+0000 7ff2c2c15640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] conn(0x7ff2a80adde0 0x7ff2a80b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.090+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== osd.9 v2:172.21.15.143:6826/4014720297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2b800fe50 con 0x7ff2a80adde0 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.104+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 --> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2bc062e00 con 0x7ff2a80adde0 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2af7fe640 1 -- 172.21.15.143:0/2229737411 <== osd.9 v2:172.21.15.143:6826/4014720297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff2bc062e00 con 0x7ff2a80adde0 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] conn(0x7ff2a80adde0 msgr2=0x7ff2a80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] conn(0x7ff2a80adde0 0x7ff2a80b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2a80a2c10 msgr2=0x7ff2a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2a80a2c10 0x7ff2a80a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0031440 tx=0x7ff2b00095f0 comp rx=0 tx=0).stop 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc103660 msgr2=0x7ff2bc198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 --2- 172.21.15.143:0/2229737411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2bc103660 0x7ff2bc198860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800c970 tx=0x7ff2b800ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c2c15640 1 -- 172.21.15.143:0/2229737411 reap_dead start 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 shutdown_connections 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.105+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 >> 172.21.15.143:0/2229737411 conn(0x7ff2bc0fc7a0 msgr2=0x7ff2bc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.106+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 shutdown_connections 2024-09-10T10:44:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:00.106+0000 7ff2c4678640 1 -- 172.21.15.143:0/2229737411 wait complete. 2024-09-10T10:44:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:01 smithi178 ceph-mon[34813]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:44:01.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:01 smithi144 ceph-mon[34230]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:44:02.519 INFO:teuthology.orchestra.run.smithi143.stdout:270582939685 2024-09-10T10:44:02.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T10:44:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:02 smithi143 ceph-mon[31057]: pgmap v320: 129 pgs: 3 peering, 126 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 383 B/s wr, 10 op/s 2024-09-10T10:44:02.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:02 smithi143 ceph-mon[31057]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-10T10:44:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:02 smithi178 ceph-mon[34813]: pgmap v320: 129 pgs: 3 peering, 126 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 383 B/s wr, 10 op/s 2024-09-10T10:44:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:02 smithi178 ceph-mon[34813]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-10T10:44:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:02 smithi144 ceph-mon[34230]: pgmap v320: 129 pgs: 3 peering, 126 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 383 B/s wr, 10 op/s 2024-09-10T10:44:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:02 smithi144 ceph-mon[34230]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-10T10:44:03.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 -- 172.21.15.143:0/2862231322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824101420 msgr2=0x7f082410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 --2- 172.21.15.143:0/2862231322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824101420 0x7f082410d950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f081c009cd0 tx=0x7f081c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 -- 172.21.15.143:0/2862231322 shutdown_connections 2024-09-10T10:44:03.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 --2- 172.21.15.143:0/2862231322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824101420 0x7f082410d950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 --2- 172.21.15.143:0/2862231322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0824100a80 0x7f0824100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 --2- 172.21.15.143:0/2862231322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0824106a80 0x7f0824106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 -- 172.21.15.143:0/2862231322 >> 172.21.15.143:0/2862231322 conn(0x7f08240fc7a0 msgr2=0x7f08240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 -- 172.21.15.143:0/2862231322 shutdown_connections 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.100+0000 7f0829220640 1 -- 172.21.15.143:0/2862231322 wait complete. 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 Processor -- start 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 -- start start 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824100a80 0x7f0824196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0824101420 0x7f0824197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0824106a80 0x7f0824197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0824110d30 con 0x7f0824100a80 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0824110bb0 con 0x7f0824106a80 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.101+0000 7f0829220640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0824110eb0 con 0x7f0824101420 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824100a80 0x7f0824196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824100a80 0x7f0824196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34668/0 (socket says 172.21.15.143:34668) 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 -- 172.21.15.143:0/2505771416 learned_addr learned my addr 172.21.15.143:0/2505771416 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f082359e640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0824106a80 0x7f0824197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f082259c640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0824101420 0x7f0824197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 -- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0824101420 msgr2=0x7f0824197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0824101420 0x7f0824197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 -- 172.21.15.143:0/2505771416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0824106a80 msgr2=0x7f0824197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0824106a80 0x7f0824197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 -- 172.21.15.143:0/2505771416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08241a6840 con 0x7f0824100a80 2024-09-10T10:44:03.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f082259c640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0824101420 0x7f0824197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:03.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0822d9d640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824100a80 0x7f0824196fd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f081400d950 tx=0x7f081400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814017070 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0814009c30 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08241a6b30 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400f620 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.102+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08241a7070 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.103+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0824062940 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.105+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f08140109a0 con 0x7f0824100a80 2024-09-10T10:44:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.105+0000 7f080bfff640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f080c0a2c10 0x7f080c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.105+0000 7f082259c640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f080c0a2c10 0x7f080c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f08140cf540 con 0x7f0824100a80 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f080bfff640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] conn(0x7f080c0adde0 0x7f080c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 --> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f080c0b0950 con 0x7f080c0adde0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f08140cf990 con 0x7f0824100a80 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f082259c640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f080c0a2c10 0x7f080c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0824199130 tx=0x7f0818005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f082359e640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] conn(0x7f080c0adde0 0x7f080c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.106+0000 7f082359e640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] conn(0x7f080c0adde0 0x7f080c0b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.107+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== osd.5 v2:172.21.15.178:6808/1007276569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f080c0b0950 con 0x7f080c0adde0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.121+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 --> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0824062dc0 con 0x7f080c0adde0 2024-09-10T10:44:03.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.121+0000 7f080bfff640 1 -- 172.21.15.143:0/2505771416 <== osd.5 v2:172.21.15.178:6808/1007276569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0824062dc0 con 0x7f080c0adde0 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] conn(0x7f080c0adde0 msgr2=0x7f080c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] conn(0x7f080c0adde0 0x7f080c0b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f080c0a2c10 msgr2=0x7f080c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f080c0a2c10 0x7f080c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0824199130 tx=0x7f0818005e50 comp rx=0 tx=0).stop 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824100a80 msgr2=0x7f0824196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 --2- 172.21.15.143:0/2505771416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824100a80 0x7f0824196fd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f081400d950 tx=0x7f081400de10 comp rx=0 tx=0).stop 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f082359e640 1 -- 172.21.15.143:0/2505771416 reap_dead start 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 shutdown_connections 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 >> 172.21.15.143:0/2505771416 conn(0x7f08240fc7a0 msgr2=0x7f08240fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 shutdown_connections 2024-09-10T10:44:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:01.122+0000 7f0829220640 1 -- 172.21.15.143:0/2505771416 wait complete. 2024-09-10T10:44:03.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 -- 172.21.15.143:0/4039896200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 msgr2=0x7fa9c8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 --2- 172.21.15.143:0/4039896200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8100ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009930 tx=0x7fa9bc02a420 comp rx=0 tx=0).stop 2024-09-10T10:44:03.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 -- 172.21.15.143:0/4039896200 shutdown_connections 2024-09-10T10:44:03.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 --2- 172.21.15.143:0/4039896200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c8101430 0x7fa9c810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 --2- 172.21.15.143:0/4039896200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8100ef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 --2- 172.21.15.143:0/4039896200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c8106a90 0x7fa9c8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 -- 172.21.15.143:0/4039896200 >> 172.21.15.143:0/4039896200 conn(0x7fa9c80fc7d0 msgr2=0x7fa9c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 -- 172.21.15.143:0/4039896200 shutdown_connections 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.740+0000 7fa9ce657640 1 -- 172.21.15.143:0/4039896200 wait complete. 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 Processor -- start 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 -- start start 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c8101430 0x7fa9c8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c8106a90 0x7fa9c81a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8110d00 con 0x7fa9c8100a90 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8110b80 con 0x7fa9c8106a90 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ce657640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8110e80 con 0x7fa9c8101430 2024-09-10T10:44:03.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34718/0 (socket says 172.21.15.143:34718) 2024-09-10T10:44:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9c7fff640 1 -- 172.21.15.143:0/1746052609 learned_addr learned my addr 172.21.15.143:0/1746052609 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9c77fe640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c8101430 0x7fa9c8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.741+0000 7fa9ccbf4640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c8106a90 0x7fa9c81a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c7fff640 1 -- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c8101430 msgr2=0x7fa9c8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c7fff640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c8101430 0x7fa9c8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c7fff640 1 -- 172.21.15.143:0/1746052609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c8106a90 msgr2=0x7fa9c81a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c7fff640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c8106a90 0x7fa9c81a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c7fff640 1 -- 172.21.15.143:0/1746052609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c81a47d0 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c7fff640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8199210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800c890 tx=0x7fa9b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004220 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c81a49a0 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9b80043c0 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004a00 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.742+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c81a4e60 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.743+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9c8062900 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.744+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa9b8004ba0 con 0x7fa9c8100a90 2024-09-10T10:44:03.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c57fa640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b00a2c10 0x7fa9b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c77fe640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b00a2c10 0x7fa9b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa9b80cf5b0 con 0x7fa9c8100a90 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c57fa640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] conn(0x7fa9b00adde0 0x7fa9b00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 --> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9b8010e50 con 0x7fa9b00adde0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa9b80cfa00 con 0x7fa9c8100a90 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9c77fe640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b00a2c10 0x7fa9b00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009900 tx=0x7fa9bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.745+0000 7fa9ccbf4640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] conn(0x7fa9b00adde0 0x7fa9b00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.746+0000 7fa9ccbf4640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] conn(0x7fa9b00adde0 0x7fa9b00b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.746+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== osd.8 v2:172.21.15.178:6816/3485367257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9b8010e50 con 0x7fa9b00adde0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 --> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9c8062d80 con 0x7fa9b00adde0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9c57fa640 1 -- 172.21.15.143:0/1746052609 <== osd.8 v2:172.21.15.178:6816/3485367257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9c8062d80 con 0x7fa9b00adde0 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] conn(0x7fa9b00adde0 msgr2=0x7fa9b00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] conn(0x7fa9b00adde0 0x7fa9b00b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b00a2c10 msgr2=0x7fa9b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b00a2c10 0x7fa9b00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009900 tx=0x7fa9bc000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 msgr2=0x7fa9c8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.760+0000 7fa9ce657640 1 --2- 172.21.15.143:0/1746052609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9c8100a90 0x7fa9c8199210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800c890 tx=0x7fa9b800cd50 comp rx=0 tx=0).stop 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.761+0000 7fa9ccbf4640 1 -- 172.21.15.143:0/1746052609 reap_dead start 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.761+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 shutdown_connections 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.761+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 >> 172.21.15.143:0/1746052609 conn(0x7fa9c80fc7d0 msgr2=0x7fa9c80fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.761+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 shutdown_connections 2024-09-10T10:44:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:03.761+0000 7fa9ce657640 1 -- 172.21.15.143:0/1746052609 wait complete. 2024-09-10T10:44:03.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:03 smithi178 ceph-mon[34813]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T10:44:03.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:03 smithi178 ceph-mon[34813]: Cluster is now healthy 2024-09-10T10:44:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:03 smithi144 ceph-mon[34230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T10:44:03.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:03 smithi144 ceph-mon[34230]: Cluster is now healthy 2024-09-10T10:44:04.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:03 smithi143 ceph-mon[31057]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T10:44:04.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:03 smithi143 ceph-mon[31057]: Cluster is now healthy 2024-09-10T10:44:04.213 INFO:teuthology.orchestra.run.smithi143.stdout:210453397551 2024-09-10T10:44:04.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T10:44:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 -- 172.21.15.143:0/2190623300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08100e20 msgr2=0x7fdf08101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 --2- 172.21.15.143:0/2190623300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08100e20 0x7fdf08101200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009900 tx=0x7fdf0402a480 comp rx=0 tx=0).stop 2024-09-10T10:44:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 -- 172.21.15.143:0/2190623300 shutdown_connections 2024-09-10T10:44:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 --2- 172.21.15.143:0/2190623300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08102ef0 0x7fdf0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 --2- 172.21.15.143:0/2190623300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf08102570 0x7fdf081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 --2- 172.21.15.143:0/2190623300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08100e20 0x7fdf08101200 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 -- 172.21.15.143:0/2190623300 >> 172.21.15.143:0/2190623300 conn(0x7fdf080fc7d0 msgr2=0x7fdf080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 -- 172.21.15.143:0/2190623300 shutdown_connections 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 -- 172.21.15.143:0/2190623300 wait complete. 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 Processor -- start 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.390+0000 7fdf10025640 1 -- start start 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf08100e20 0x7fdf08199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08102570 0x7fdf08199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08102ef0 0x7fdf081a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08110e30 con 0x7fdf08102570 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08110cb0 con 0x7fdf08102ef0 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08110fb0 con 0x7fdf08100e20 2024-09-10T10:44:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08102570 0x7fdf08199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08102570 0x7fdf08199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34740/0 (socket says 172.21.15.143:34740) 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 -- 172.21.15.143:0/1773762273 learned_addr learned my addr 172.21.15.143:0/1773762273 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0e5c2640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08102ef0 0x7fdf081a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0ddc1640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf08100e20 0x7fdf08199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 -- 172.21.15.143:0/1773762273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf08100e20 msgr2=0x7fdf08199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf08100e20 0x7fdf08199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 -- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08102ef0 msgr2=0x7fdf081a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:04.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08102ef0 0x7fdf081a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 -- 172.21.15.143:0/1773762273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf081a47c0 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0e5c2640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08102ef0 0x7fdf081a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf0d5c0640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08102570 0x7fdf08199740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d950 tx=0x7fdef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8017070 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdef8009c30 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.391+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081a4990 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.392+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef800f620 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.392+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a4ea0 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.393+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf08062980 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.394+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdef8010460 con 0x7fdf08102570 2024-09-10T10:44:04.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.394+0000 7fdefeffd640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdef40a2c10 0x7fdef40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.394+0000 7fdf0ddc1640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdef40a2c10 0x7fdef40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.394+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdef80cf240 con 0x7fdf08102570 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.395+0000 7fdefeffd640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] conn(0x7fdef40adde0 0x7fdef40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.395+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 --> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdef8004470 con 0x7fdef40adde0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.395+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fdef80d6050 con 0x7fdf08102570 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.395+0000 7fdf0ddc1640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdef40a2c10 0x7fdef40a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005ab0 tx=0x7fdf04005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.395+0000 7fdf0e5c2640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] conn(0x7fdef40adde0 0x7fdef40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.395+0000 7fdf0e5c2640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] conn(0x7fdef40adde0 0x7fdef40b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.396+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== osd.10 v2:172.21.15.144:6824/2444988938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdef8004470 con 0x7fdef40adde0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.410+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 --> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf08062e30 con 0x7fdef40adde0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.411+0000 7fdefeffd640 1 -- 172.21.15.143:0/1773762273 <== osd.10 v2:172.21.15.144:6824/2444988938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdf08062e30 con 0x7fdef40adde0 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.411+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] conn(0x7fdef40adde0 msgr2=0x7fdef40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.411+0000 7fdf10025640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] conn(0x7fdef40adde0 0x7fdef40b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.411+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdef40a2c10 msgr2=0x7fdef40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.411+0000 7fdf10025640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdef40a2c10 0x7fdef40a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005ab0 tx=0x7fdf04005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08102570 msgr2=0x7fdf08199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf10025640 1 --2- 172.21.15.143:0/1773762273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf08102570 0x7fdf08199740 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d950 tx=0x7fdef800de10 comp rx=0 tx=0).stop 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf0e5c2640 1 -- 172.21.15.143:0/1773762273 reap_dead start 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 shutdown_connections 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 >> 172.21.15.143:0/1773762273 conn(0x7fdf080fc7d0 msgr2=0x7fdf080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 shutdown_connections 2024-09-10T10:44:04.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:04.412+0000 7fdf10025640 1 -- 172.21.15.143:0/1773762273 wait complete. 2024-09-10T10:44:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:04 smithi178 ceph-mon[34813]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 383 B/s wr, 10 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:04 smithi144 ceph-mon[34230]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 383 B/s wr, 10 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:05.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:04 smithi143 ceph-mon[31057]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 383 B/s wr, 10 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:05.274 INFO:teuthology.orchestra.run.smithi143.stdout:292057776161 2024-09-10T10:44:05.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T10:44:05.710 INFO:teuthology.orchestra.run.smithi143.stdout:77309411402 2024-09-10T10:44:05.710 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T10:44:05.875 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:05 smithi143 ceph-mon[31057]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:05.875 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:05 smithi143 ceph-mon[31057]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T10:44:05.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:05 smithi178 ceph-mon[34813]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:05.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:05 smithi178 ceph-mon[34813]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T10:44:05.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:05 smithi144 ceph-mon[34230]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:05.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:05 smithi144 ceph-mon[34230]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T10:44:06.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 -- 172.21.15.143:0/136188525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c1036e0 msgr2=0x7f228c10bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 --2- 172.21.15.143:0/136188525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c1036e0 0x7f228c10bbd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2278009900 tx=0x7f227802a480 comp rx=0 tx=0).stop 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 -- 172.21.15.143:0/136188525 shutdown_connections 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 --2- 172.21.15.143:0/136188525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f228c10c1a0 0x7f228c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 --2- 172.21.15.143:0/136188525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c1036e0 0x7f228c10bbd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 --2- 172.21.15.143:0/136188525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f228c102dc0 0x7f228c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 -- 172.21.15.143:0/136188525 >> 172.21.15.143:0/136188525 conn(0x7f228c0fc7a0 msgr2=0x7f228c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 -- 172.21.15.143:0/136188525 shutdown_connections 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.217+0000 7f229311c640 1 -- 172.21.15.143:0/136188525 wait complete. 2024-09-10T10:44:06.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 Processor -- start 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 -- start start 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c102dc0 0x7f228c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f228c1036e0 0x7f228c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f228c10c1a0 0x7f228c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f228c111e20 con 0x7f228c102dc0 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f228c111ca0 con 0x7f228c1036e0 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f229311c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f228c111fa0 con 0x7f228c10c1a0 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c102dc0 0x7f228c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c102dc0 0x7f228c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34772/0 (socket says 172.21.15.143:34772) 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 -- 172.21.15.143:0/3342098771 learned_addr learned my addr 172.21.15.143:0/3342098771 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 -- 172.21.15.143:0/3342098771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f228c10c1a0 msgr2=0x7f228c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f228c10c1a0 0x7f228c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 -- 172.21.15.143:0/3342098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f228c1036e0 msgr2=0x7f228c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f228c1036e0 0x7f228c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.218+0000 7f2290eb8640 1 -- 172.21.15.143:0/3342098771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c19d370 con 0x7f228c102dc0 2024-09-10T10:44:06.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.219+0000 7f2290eb8640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c102dc0 0x7f228c1983c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f228000c970 tx=0x7f228000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.219+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280007bb0 con 0x7f228c102dc0 2024-09-10T10:44:06.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.219+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2280007d50 con 0x7f228c102dc0 2024-09-10T10:44:06.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.219+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228000f450 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.219+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c19d660 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.219+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c1a4f40 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.220+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f228c062980 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.221+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2280004ee0 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f2289ffb640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f227c0a2c10 0x7f227c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f228bfff640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f227c0a2c10 0x7f227c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f22800cf740 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f2289ffb640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] conn(0x7f227c0adde0 0x7f227c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 --> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2280012ec0 con 0x7f227c0adde0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f22800cfb90 con 0x7f228c102dc0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.222+0000 7f22916b9640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] conn(0x7f227c0adde0 0x7f227c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.223+0000 7f228bfff640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f227c0a2c10 0x7f227c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22780096c0 tx=0x7f2278031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.223+0000 7f22916b9640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] conn(0x7f227c0adde0 0x7f227c0b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.223+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== osd.3 v2:172.21.15.143:6810/3687852925 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2280012ec0 con 0x7f227c0adde0 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 --> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f228c062d80 con 0x7f227c0adde0 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f2289ffb640 1 -- 172.21.15.143:0/3342098771 <== osd.3 v2:172.21.15.143:6810/3687852925 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f228c062d80 con 0x7f227c0adde0 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] conn(0x7f227c0adde0 msgr2=0x7f227c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] conn(0x7f227c0adde0 0x7f227c0b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f227c0a2c10 msgr2=0x7f227c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f227c0a2c10 0x7f227c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22780096c0 tx=0x7f2278031400 comp rx=0 tx=0).stop 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c102dc0 msgr2=0x7f228c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 --2- 172.21.15.143:0/3342098771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f228c102dc0 0x7f228c1983c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f228000c970 tx=0x7f228000ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f22916b9640 1 -- 172.21.15.143:0/3342098771 reap_dead start 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 shutdown_connections 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 >> 172.21.15.143:0/3342098771 conn(0x7f228c0fc7a0 msgr2=0x7f228c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 shutdown_connections 2024-09-10T10:44:06.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:06.237+0000 7f229311c640 1 -- 172.21.15.143:0/3342098771 wait complete. 2024-09-10T10:44:07.053 INFO:teuthology.orchestra.run.smithi143.stdout:158913790010 2024-09-10T10:44:07.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T10:44:07.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 -- 172.21.15.143:0/977834750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0100de0 msgr2=0x7f28a0104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 --2- 172.21.15.143:0/977834750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0100de0 0x7f28a0104e30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2888009900 tx=0x7f288802a410 comp rx=0 tx=0).stop 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 -- 172.21.15.143:0/977834750 shutdown_connections 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 --2- 172.21.15.143:0/977834750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a0105560 0x7f28a010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 --2- 172.21.15.143:0/977834750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0100de0 0x7f28a0104e30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 --2- 172.21.15.143:0/977834750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a0100430 0x7f28a0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 -- 172.21.15.143:0/977834750 >> 172.21.15.143:0/977834750 conn(0x7f28a0077230 msgr2=0x7f28a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 -- 172.21.15.143:0/977834750 shutdown_connections 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.874+0000 7f28a540e640 1 -- 172.21.15.143:0/977834750 wait complete. 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 Processor -- start 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 -- start start 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a0100430 0x7f28a0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a0100de0 0x7f28a0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0105560 0x7f28a0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0110cf0 con 0x7f28a0105560 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0110b70 con 0x7f28a0100de0 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f28a540e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0110e70 con 0x7f28a0100430 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0105560 0x7f28a0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0105560 0x7f28a0199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34760/0 (socket says 172.21.15.143:34760) 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 -- 172.21.15.143:0/3899914330 learned_addr learned my addr 172.21.15.143:0/3899914330 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289effd640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a0100430 0x7f28a0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f2897fff640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a0100de0 0x7f28a0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 -- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a0100430 msgr2=0x7f28a0199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a0100430 0x7f28a0199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 -- 172.21.15.143:0/3899914330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a0100de0 msgr2=0x7f28a0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a0100de0 0x7f28a0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.875+0000 7f289f7fe640 1 -- 172.21.15.143:0/3899914330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a01a45f0 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f2897fff640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a0100de0 0x7f28a0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f289f7fe640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0105560 0x7f28a0199cc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f289800eaa0 tx=0x7f289800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289800cdb0 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2898005020 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a01a48e0 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28980116d0 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.876+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a01a4df0 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.877+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28a0062900 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.879+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f28980051d0 con 0x7f28a0105560 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.880+0000 7f289cff9640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f288c0a2c10 0x7f288c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.880+0000 7f289effd640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f288c0a2c10 0x7f288c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.880+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f28980cfa50 con 0x7f28a0105560 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.880+0000 7f289cff9640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] conn(0x7f288c0adde0 0x7f288c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.880+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 --> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2898014ec0 con 0x7f288c0adde0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.880+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f28980cfea0 con 0x7f28a0105560 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.881+0000 7f289effd640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f288c0a2c10 0x7f288c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f28900058c0 tx=0x7f289000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.881+0000 7f2897fff640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] conn(0x7f288c0adde0 0x7f288c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.881+0000 7f2897fff640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] conn(0x7f288c0adde0 0x7f288c0b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.882+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== osd.11 v2:172.21.15.178:6824/2309120918 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2898014ec0 con 0x7f288c0adde0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.897+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 --> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28a0062d80 con 0x7f288c0adde0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.898+0000 7f289cff9640 1 -- 172.21.15.143:0/3899914330 <== osd.11 v2:172.21.15.178:6824/2309120918 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f28a0062d80 con 0x7f288c0adde0 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.898+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] conn(0x7f288c0adde0 msgr2=0x7f288c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.898+0000 7f28a540e640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] conn(0x7f288c0adde0 0x7f288c0b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.898+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f288c0a2c10 msgr2=0x7f288c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.898+0000 7f28a540e640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f288c0a2c10 0x7f288c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f28900058c0 tx=0x7f289000a2d0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0105560 msgr2=0x7f28a0199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f28a540e640 1 --2- 172.21.15.143:0/3899914330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28a0105560 0x7f28a0199cc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f289800eaa0 tx=0x7f289800ef60 comp rx=0 tx=0).stop 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f289f7fe640 1 -- 172.21.15.143:0/3899914330 reap_dead start 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 shutdown_connections 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 >> 172.21.15.143:0/3899914330 conn(0x7f28a0077230 msgr2=0x7f28a0103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 shutdown_connections 2024-09-10T10:44:07.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:05.899+0000 7f28a540e640 1 -- 172.21.15.143:0/3899914330 wait complete. 2024-09-10T10:44:07.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:07 smithi143 ceph-mon[31057]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T10:44:07.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:07 smithi143 ceph-mon[31057]: Cluster is now healthy 2024-09-10T10:44:07.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:07 smithi178 ceph-mon[34813]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T10:44:07.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:07 smithi178 ceph-mon[34813]: Cluster is now healthy 2024-09-10T10:44:07.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:07 smithi144 ceph-mon[34230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T10:44:07.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:07 smithi144 ceph-mon[34230]: Cluster is now healthy 2024-09-10T10:44:08.028 INFO:teuthology.orchestra.run.smithi143.stdout:240518168619 2024-09-10T10:44:08.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T10:44:08.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 -- 172.21.15.143:0/1030614958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102570 msgr2=0x7f8f481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 --2- 172.21.15.143:0/1030614958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102570 0x7f8f481029b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009900 tx=0x7f8f3402a480 comp rx=0 tx=0).stop 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 -- 172.21.15.143:0/1030614958 shutdown_connections 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 --2- 172.21.15.143:0/1030614958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f48102ef0 0x7f8f4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 --2- 172.21.15.143:0/1030614958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102570 0x7f8f481029b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 --2- 172.21.15.143:0/1030614958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f48100e20 0x7f8f48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 -- 172.21.15.143:0/1030614958 >> 172.21.15.143:0/1030614958 conn(0x7f8f480fc7d0 msgr2=0x7f8f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 -- 172.21.15.143:0/1030614958 shutdown_connections 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.531+0000 7f8f4f141640 1 -- 172.21.15.143:0/1030614958 wait complete. 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 Processor -- start 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 -- start start 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f48100e20 0x7f8f481973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f48102570 0x7f8f48197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102ef0 0x7f8f4819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48110e50 con 0x7f8f48102ef0 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48110cd0 con 0x7f8f48100e20 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4f141640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48110fd0 con 0x7f8f48102570 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4d6de640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102ef0 0x7f8f4819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4d6de640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102ef0 0x7f8f4819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47942/0 (socket says 172.21.15.143:47942) 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4d6de640 1 -- 172.21.15.143:0/4005011285 learned_addr learned my addr 172.21.15.143:0/4005011285 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.532+0000 7f8f4cedd640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f48100e20 0x7f8f481973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f47fff640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f48102570 0x7f8f48197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4d6de640 1 -- 172.21.15.143:0/4005011285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f48102570 msgr2=0x7f8f48197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4d6de640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f48102570 0x7f8f48197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4d6de640 1 -- 172.21.15.143:0/4005011285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f48100e20 msgr2=0x7f8f481973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4d6de640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f48100e20 0x7f8f481973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4d6de640 1 -- 172.21.15.143:0/4005011285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4819c3a0 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f47fff640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f48102570 0x7f8f48197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4d6de640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102ef0 0x7f8f4819bcc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000cad0 tx=0x7f8f4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000ae20 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4819c690 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f40005020 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000f6b0 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.533+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a3f70 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.534+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f48062940 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.535+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8f400051d0 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f45ffb640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f380a2c10 0x7f8f380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f400cfa50 con 0x7f8f48102ef0 2024-09-10T10:44:08.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f4cedd640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f380a2c10 0x7f8f380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f45ffb640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] conn(0x7f8f380adde0 0x7f8f380b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 --> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f380b0950 con 0x7f8f380adde0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8f400cfe00 con 0x7f8f48102ef0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f47fff640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] conn(0x7f8f380adde0 0x7f8f380b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f4cedd640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f380a2c10 0x7f8f380a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009890 tx=0x7f8f3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.536+0000 7f8f47fff640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] conn(0x7f8f380adde0 0x7f8f380b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.537+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== osd.6 v2:172.21.15.143:6818/148519296 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f380b0950 con 0x7f8f380adde0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.551+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 --> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f48062dc0 con 0x7f8f380adde0 2024-09-10T10:44:08.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.551+0000 7f8f45ffb640 1 -- 172.21.15.143:0/4005011285 <== osd.6 v2:172.21.15.143:6818/148519296 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f48062dc0 con 0x7f8f380adde0 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] conn(0x7f8f380adde0 msgr2=0x7f8f380b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] conn(0x7f8f380adde0 0x7f8f380b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f380a2c10 msgr2=0x7f8f380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f380a2c10 0x7f8f380a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009890 tx=0x7f8f3c008040 comp rx=0 tx=0).stop 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102ef0 msgr2=0x7f8f4819bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 --2- 172.21.15.143:0/4005011285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f48102ef0 0x7f8f4819bcc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000cad0 tx=0x7f8f4000cf90 comp rx=0 tx=0).stop 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4d6de640 1 -- 172.21.15.143:0/4005011285 reap_dead start 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 shutdown_connections 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 >> 172.21.15.143:0/4005011285 conn(0x7f8f480fc7d0 msgr2=0x7f8f480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 shutdown_connections 2024-09-10T10:44:08.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:08.552+0000 7f8f4f141640 1 -- 172.21.15.143:0/4005011285 wait complete. 2024-09-10T10:44:08.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:08 smithi143 ceph-mon[31057]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:08.826 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:08 smithi178 ceph-mon[34813]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:08.837 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:08 smithi144 ceph-mon[34230]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:09.513 INFO:teuthology.orchestra.run.smithi143.stdout:188978561078 2024-09-10T10:44:09.514 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.135+0000 7f8d78809640 1 -- 172.21.15.143:0/1601410068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 msgr2=0x7f8d741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.135+0000 7f8d78809640 1 --2- 172.21.15.143:0/1601410068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d741021c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009ca0 tx=0x7f8d6402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 -- 172.21.15.143:0/1601410068 shutdown_connections 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 --2- 172.21.15.143:0/1601410068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d74106f10 0x7f8d7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 --2- 172.21.15.143:0/1601410068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d74102790 0x7f8d741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 --2- 172.21.15.143:0/1601410068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d741021c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 -- 172.21.15.143:0/1601410068 >> 172.21.15.143:0/1601410068 conn(0x7f8d740fd8c0 msgr2=0x7f8d740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 -- 172.21.15.143:0/1601410068 shutdown_connections 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 -- 172.21.15.143:0/1601410068 wait complete. 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 Processor -- start 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 -- start start 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d74198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.136+0000 7f8d78809640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d74102790 0x7f8d741988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d78809640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d74106f10 0x7f8d7419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d78809640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74111e20 con 0x7f8d74101de0 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d78809640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74111ca0 con 0x7f8d74102790 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d78809640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d74111fa0 con 0x7f8d74106f10 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d74198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d74198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47970/0 (socket says 172.21.15.143:47970) 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 -- 172.21.15.143:0/1903313524 learned_addr learned my addr 172.21.15.143:0/1903313524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d72d9d640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d74106f10 0x7f8d7419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d71d9b640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d74102790 0x7f8d741988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 -- 172.21.15.143:0/1903313524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d74106f10 msgr2=0x7f8d7419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d74106f10 0x7f8d7419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 -- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d74102790 msgr2=0x7f8d741988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d74102790 0x7f8d741988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 -- 172.21.15.143:0/1903313524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7419d330 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d72d9d640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d74106f10 0x7f8d7419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d7259c640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d74198380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d6402ade0 tx=0x7f8d64032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64003cf0 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d640044f0 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7419d5c0 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64030c40 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.137+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d741a4f00 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.138+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d74062980 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8d64004040 con 0x7f8d74101de0 2024-09-10T10:44:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d637fe640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8d5c0a2c10 0x7f8d5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d71d9b640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8d5c0a2c10 0x7f8d5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8d640f0d50 con 0x7f8d74101de0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d637fe640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] conn(0x7f8d5c0adde0 0x7f8d5c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 --> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d64050ea0 con 0x7f8d5c0adde0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.140+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8d640f11a0 con 0x7f8d74101de0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.141+0000 7f8d71d9b640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8d5c0a2c10 0x7f8d5c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005fd0 tx=0x7f8d68005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.141+0000 7f8d72d9d640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] conn(0x7f8d5c0adde0 0x7f8d5c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.141+0000 7f8d72d9d640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] conn(0x7f8d5c0adde0 0x7f8d5c0b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.142+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== osd.4 v2:172.21.15.144:6808/2906236694 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d64050ea0 con 0x7f8d5c0adde0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 --> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d74062e00 con 0x7f8d5c0adde0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d637fe640 1 -- 172.21.15.143:0/1903313524 <== osd.4 v2:172.21.15.144:6808/2906236694 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d74062e00 con 0x7f8d5c0adde0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] conn(0x7f8d5c0adde0 msgr2=0x7f8d5c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] conn(0x7f8d5c0adde0 0x7f8d5c0b0280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8d5c0a2c10 msgr2=0x7f8d5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8d5c0a2c10 0x7f8d5c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005fd0 tx=0x7f8d68005850 comp rx=0 tx=0).stop 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 msgr2=0x7f8d74198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d78809640 1 --2- 172.21.15.143:0/1903313524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d74101de0 0x7f8d74198380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d6402ade0 tx=0x7f8d64032ff0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.156+0000 7f8d72d9d640 1 -- 172.21.15.143:0/1903313524 reap_dead start 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.157+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 shutdown_connections 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.157+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 >> 172.21.15.143:0/1903313524 conn(0x7f8d740fd8c0 msgr2=0x7f8d740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.157+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 shutdown_connections 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:09.157+0000 7f8d78809640 1 -- 172.21.15.143:0/1903313524 wait complete. 2024-09-10T10:44:09.875 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:09.981 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:10.315 INFO:teuthology.orchestra.run.smithi143.stdout:103079215173 2024-09-10T10:44:10.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T10:44:10.773 INFO:teuthology.orchestra.run.smithi143.stdout:322122547226 2024-09-10T10:44:10.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T10:44:10.795 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:10 smithi143 ceph-mon[31057]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 6.1 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:10 smithi178 ceph-mon[34813]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 6.1 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:10 smithi144 ceph-mon[34230]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 6.1 KiB/s, 1 keys/s, 1 objects/s recovering 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 -- 172.21.15.143:0/1329195702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc106f50 msgr2=0x7fb0bc10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 --2- 172.21.15.143:0/1329195702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc106f50 0x7fb0bc10eaf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009cd0 tx=0x7fb0b402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 -- 172.21.15.143:0/1329195702 shutdown_connections 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 --2- 172.21.15.143:0/1329195702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc106f50 0x7fb0bc10eaf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 --2- 172.21.15.143:0/1329195702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0bc100ca0 0x7fb0bc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 --2- 172.21.15.143:0/1329195702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 -- 172.21.15.143:0/1329195702 >> 172.21.15.143:0/1329195702 conn(0x7fb0bc077210 msgr2=0x7fb0bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 -- 172.21.15.143:0/1329195702 shutdown_connections 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.214+0000 7fb0c2925640 1 -- 172.21.15.143:0/1329195702 wait complete. 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 Processor -- start 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 -- start start 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc100ca0 0x7fb0bc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0bc106f50 0x7fb0bc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc111f30 con 0x7fb0bc1002f0 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc111db0 con 0x7fb0bc100ca0 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0c2925640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1120b0 con 0x7fb0bc106f50 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47994/0 (socket says 172.21.15.143:47994) 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.215+0000 7fb0bbfff640 1 -- 172.21.15.143:0/2392103836 learned_addr learned my addr 172.21.15.143:0/2392103836 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bb7fe640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc100ca0 0x7fb0bc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bbfff640 1 -- 172.21.15.143:0/2392103836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0bc106f50 msgr2=0x7fb0bc19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0c0ec2640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0bc106f50 0x7fb0bc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bbfff640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0bc106f50 0x7fb0bc19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bbfff640 1 -- 172.21.15.143:0/2392103836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc100ca0 msgr2=0x7fb0bc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bbfff640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc100ca0 0x7fb0bc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bbfff640 1 -- 172.21.15.143:0/2392103836 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc19d4a0 con 0x7fb0bc1002f0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bb7fe640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc100ca0 0x7fb0bc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0bbfff640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1984f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00b4a0 tx=0x7fb0ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac014070 con 0x7fb0bc1002f0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0ac010ac0 con 0x7fb0bc1002f0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc19d790 con 0x7fb0bc1002f0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac017c30 con 0x7fb0bc1002f0 2024-09-10T10:44:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.216+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1a5070 con 0x7fb0bc1002f0 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.217+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc100750 con 0x7fb0bc1002f0 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.218+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb0ac017070 con 0x7fb0bc1002f0 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.219+0000 7fb0b97fa640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.219+0000 7fb0bb7fe640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.219+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb0ac0cec20 con 0x7fb0bc1002f0 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.219+0000 7fb0bb7fe640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006fd0 tx=0x7fb0b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.222+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0ac09b7d0 con 0x7fb0bc1002f0 2024-09-10T10:44:11.237 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:11.238 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:11.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.409+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb0bc199850 con 0x7fb0bc1002f0 2024-09-10T10:44:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.409+0000 7fb0b97fa640 1 -- 172.21.15.143:0/2392103836 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb0ac09b0d0 con 0x7fb0bc1002f0 2024-09-10T10:44:11.411 INFO:teuthology.orchestra.run.smithi143.stdout:34359738453 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0a40a2c10 msgr2=0x7fb0a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006fd0 tx=0x7fb0b0008040 comp rx=0 tx=0).stop 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 msgr2=0x7fb0bc1984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1984f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00b4a0 tx=0x7fb0ac00b960 comp rx=0 tx=0).stop 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 shutdown_connections 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0a40a2c10 0x7fb0a40a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0bc106f50 0x7fb0bc19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0bc100ca0 0x7fb0bc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 --2- 172.21.15.143:0/2392103836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0bc1002f0 0x7fb0bc1984f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 >> 172.21.15.143:0/2392103836 conn(0x7fb0bc077210 msgr2=0x7fb0bc105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 shutdown_connections 2024-09-10T10:44:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:11.412+0000 7fb0c2925640 1 -- 172.21.15.143:0/2392103836 wait complete. 2024-09-10T10:44:11.767 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:11 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2392103836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:44:11.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:11 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2392103836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:44:11.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:11 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2392103836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:44:12.401 INFO:teuthology.orchestra.run.smithi143.stdout:137438953536 2024-09-10T10:44:12.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T10:44:12.636 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:12 smithi143 ceph-mon[31057]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-10T10:44:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 -- 172.21.15.143:0/2167028843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 msgr2=0x7ff40c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 --2- 172.21.15.143:0/2167028843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c106e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff408009930 tx=0x7ff40802a420 comp rx=0 tx=0).stop 2024-09-10T10:44:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 -- 172.21.15.143:0/2167028843 shutdown_connections 2024-09-10T10:44:12.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 --2- 172.21.15.143:0/2167028843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff40c101420 0x7ff40c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 --2- 172.21.15.143:0/2167028843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff40c100a80 0x7ff40c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 --2- 172.21.15.143:0/2167028843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c106e60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 -- 172.21.15.143:0/2167028843 >> 172.21.15.143:0/2167028843 conn(0x7ff40c0fc7a0 msgr2=0x7ff40c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 -- 172.21.15.143:0/2167028843 shutdown_connections 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.839+0000 7ff414135640 1 -- 172.21.15.143:0/2167028843 wait complete. 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 Processor -- start 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 -- start start 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff40c100a80 0x7ff40c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff40c101420 0x7ff40c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c110d30 con 0x7ff40c106a80 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c110bb0 con 0x7ff40c100a80 2024-09-10T10:44:12.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff414135640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c110eb0 con 0x7ff40c101420 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48022/0 (socket says 172.21.15.143:48022) 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 -- 172.21.15.143:0/1309394731 learned_addr learned my addr 172.21.15.143:0/1309394731 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4116d0640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff40c101420 0x7ff40c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff411ed1640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff40c100a80 0x7ff40c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 -- 172.21.15.143:0/1309394731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff40c101420 msgr2=0x7ff40c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff40c101420 0x7ff40c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 -- 172.21.15.143:0/1309394731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff40c100a80 msgr2=0x7ff40c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff40c100a80 0x7ff40c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 -- 172.21.15.143:0/1309394731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c1a67a0 con 0x7ff40c106a80 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.840+0000 7ff4126d2640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c1979b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff40400cad0 tx=0x7ff40400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.841+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400ad60 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.841+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c1a6a90 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.841+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff404005020 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.841+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400f6b0 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.841+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c1a6f80 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.842+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c100ee0 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.843+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff4040051d0 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.843+0000 7ff402ffd640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff3f80a2c10 0x7ff3f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.844+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff4040cf460 con 0x7ff40c106a80 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.844+0000 7ff411ed1640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff3f80a2c10 0x7ff3f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.844+0000 7ff411ed1640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff3f80a2c10 0x7ff3f80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff408006010 tx=0x7ff408000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:12.847+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff404014030 con 0x7ff40c106a80 2024-09-10T10:44:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:12 smithi178 ceph-mon[34813]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-10T10:44:12.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:12 smithi144 ceph-mon[34230]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 5.6 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-10T10:44:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.022+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff40c02d7c0 con 0x7ff40c106a80 2024-09-10T10:44:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.022+0000 7ff402ffd640 1 -- 172.21.15.143:0/1309394731 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff40409bf00 con 0x7ff40c106a80 2024-09-10T10:44:13.024 INFO:teuthology.orchestra.run.smithi143.stdout:240518168621 2024-09-10T10:44:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff3f80a2c10 msgr2=0x7ff3f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff3f80a2c10 0x7ff3f80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff408006010 tx=0x7ff408000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 msgr2=0x7ff40c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c1979b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff40400cad0 tx=0x7ff40400ced0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 shutdown_connections 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff3f80a2c10 0x7ff3f80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40c106a80 0x7ff40c1979b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff40c101420 0x7ff40c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 --2- 172.21.15.143:0/1309394731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff40c100a80 0x7ff40c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 >> 172.21.15.143:0/1309394731 conn(0x7ff40c0fc7a0 msgr2=0x7ff40c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.024+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 shutdown_connections 2024-09-10T10:44:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.025+0000 7ff414135640 1 -- 172.21.15.143:0/1309394731 wait complete. 2024-09-10T10:44:13.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738453 for osd.0 2024-09-10T10:44:13.368 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:13.708 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:13 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1309394731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:44:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.786+0000 7f93ff471640 1 -- 172.21.15.143:0/2423056225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 msgr2=0x7f93f810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.786+0000 7f93ff471640 1 --2- 172.21.15.143:0/2423056225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f810ec50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009cd0 tx=0x7f93f002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- 172.21.15.143:0/2423056225 shutdown_connections 2024-09-10T10:44:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 --2- 172.21.15.143:0/2423056225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f810ec50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 --2- 172.21.15.143:0/2423056225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93f8103760 0x7f93f8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 --2- 172.21.15.143:0/2423056225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8102db0 0x7f93f8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- 172.21.15.143:0/2423056225 >> 172.21.15.143:0/2423056225 conn(0x7f93f80fc7d0 msgr2=0x7f93f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- 172.21.15.143:0/2423056225 shutdown_connections 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- 172.21.15.143:0/2423056225 wait complete. 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 Processor -- start 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- start start 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93f8102db0 0x7f93f819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8103760 0x7f93f819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8111f60 con 0x7f93f81082e0 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8111de0 con 0x7f93f8103760 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.787+0000 7f93ff471640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81120e0 con 0x7f93f8102db0 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48044/0 (socket says 172.21.15.143:48044) 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 -- 172.21.15.143:0/689320696 learned_addr learned my addr 172.21.15.143:0/689320696 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fd20d640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93f8102db0 0x7f93f819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fca0c640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8103760 0x7f93f819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 -- 172.21.15.143:0/689320696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93f8102db0 msgr2=0x7f93f819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93f8102db0 0x7f93f819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 -- 172.21.15.143:0/689320696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8103760 msgr2=0x7f93f819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8103760 0x7f93f819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 -- 172.21.15.143:0/689320696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819f6f0 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fca0c640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8103760 0x7f93f819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93fda0e640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f819f010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93f00097b0 tx=0x7f93f0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004240 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93f00305a0 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819f980 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0045650 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.788+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a72c0 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.789+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f8103190 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.790+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f93f0043020 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.791+0000 7f93ee7fc640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93e40a2c10 0x7f93e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.791+0000 7f93fd20d640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93e40a2c10 0x7f93e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.791+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93f002e030 con 0x7f93f81082e0 2024-09-10T10:44:13.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.791+0000 7f93fd20d640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93e40a2c10 0x7f93e40a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93f40058c0 tx=0x7f93f400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:13.795+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93f004a060 con 0x7f93f81082e0 2024-09-10T10:44:13.826 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.826 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.827 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.827 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.835 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.871 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:13 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1309394731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:44:13.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:13 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1309394731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:44:14.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.010+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f93f810c2d0 con 0x7f93f81082e0 2024-09-10T10:44:14.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.011+0000 7f93ee7fc640 1 -- 172.21.15.143:0/689320696 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f93f00bd050 con 0x7f93f81082e0 2024-09-10T10:44:14.013 INFO:teuthology.orchestra.run.smithi143.stdout:55834574929 2024-09-10T10:44:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93e40a2c10 msgr2=0x7f93e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93e40a2c10 0x7f93e40a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93f40058c0 tx=0x7f93f400a4c0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 msgr2=0x7f93f819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f819f010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93f00097b0 tx=0x7f93f0003ba0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 shutdown_connections 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93e40a2c10 0x7f93e40a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93f81082e0 0x7f93f819f010 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f8103760 0x7f93f819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 --2- 172.21.15.143:0/689320696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93f8102db0 0x7f93f819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 >> 172.21.15.143:0/689320696 conn(0x7f93f80fc7d0 msgr2=0x7f93f80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 shutdown_connections 2024-09-10T10:44:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.013+0000 7f93ff471640 1 -- 172.21.15.143:0/689320696 wait complete. 2024-09-10T10:44:14.806 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:14 smithi143 ceph-mon[31057]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 5.1 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-10T10:44:14.807 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:14 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/689320696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:44:14.833 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574929 for osd.1 2024-09-10T10:44:14.833 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:14 smithi178 ceph-mon[34813]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 5.1 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-10T10:44:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:14 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/689320696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:44:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:14 smithi144 ceph-mon[34230]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 5.1 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-10T10:44:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:14 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/689320696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 -- 172.21.15.143:0/3793177797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398106a90 msgr2=0x7fb398106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/3793177797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398106a90 0x7fb398106e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb388009930 tx=0x7fb38802a420 comp rx=0 tx=0).stop 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 -- 172.21.15.143:0/3793177797 shutdown_connections 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/3793177797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398101430 0x7fb39810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/3793177797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb398100a90 0x7fb398100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/3793177797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398106a90 0x7fb398106e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 -- 172.21.15.143:0/3793177797 >> 172.21.15.143:0/3793177797 conn(0x7fb3980fc7d0 msgr2=0x7fb3980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 -- 172.21.15.143:0/3793177797 shutdown_connections 2024-09-10T10:44:15.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.990+0000 7fb39c9dd640 1 -- 172.21.15.143:0/3793177797 wait complete. 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 Processor -- start 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 -- start start 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 0x7fb398199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb398101430 0x7fb3981997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398106a90 0x7fb3981a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb398110d80 con 0x7fb398100a90 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb398110c00 con 0x7fb398106a90 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.991+0000 7fb39c9dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb398110f00 con 0x7fb398101430 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 0x7fb398199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 0x7fb398199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48066/0 (socket says 172.21.15.143:48066) 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 -- 172.21.15.143:0/334994757 learned_addr learned my addr 172.21.15.143:0/334994757 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb38dd9b640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb398101430 0x7fb3981997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb396d9d640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398106a90 0x7fb3981a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 -- 172.21.15.143:0/334994757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb398101430 msgr2=0x7fb3981997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb398101430 0x7fb3981997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 -- 172.21.15.143:0/334994757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398106a90 msgr2=0x7fb3981a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398106a90 0x7fb3981a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 -- 172.21.15.143:0/334994757 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3981a4840 con 0x7fb398100a90 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb396d9d640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398106a90 0x7fb3981a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39659c640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 0x7fb398199280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb388004770 tx=0x7fb3880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38803f070 con 0x7fb398100a90 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb38803b460 con 0x7fb398100a90 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3981a4a10 con 0x7fb398100a90 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388035070 con 0x7fb398100a90 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.992+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3981a4e60 con 0x7fb398100a90 2024-09-10T10:44:15.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.993+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb398100ef0 con 0x7fb398100a90 2024-09-10T10:44:15.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.994+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb38803b610 con 0x7fb398100a90 2024-09-10T10:44:15.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.995+0000 7fb38ffff640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb3840a2c10 0x7fb3840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.995+0000 7fb38dd9b640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb3840a2c10 0x7fb3840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.995+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb3880ef8c0 con 0x7fb398100a90 2024-09-10T10:44:15.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.996+0000 7fb38dd9b640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb3840a2c10 0x7fb3840a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb39806b2e0 tx=0x7fb38000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:14.999+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3880bc360 con 0x7fb398100a90 2024-09-10T10:44:15.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.171+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb398108a70 con 0x7fb398100a90 2024-09-10T10:44:15.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.171+0000 7fb38ffff640 1 -- 172.21.15.143:0/334994757 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb3880bbc60 con 0x7fb398100a90 2024-09-10T10:44:15.174 INFO:teuthology.orchestra.run.smithi143.stdout:188978561079 2024-09-10T10:44:15.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb3840a2c10 msgr2=0x7fb3840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb3840a2c10 0x7fb3840a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb39806b2e0 tx=0x7fb38000b390 comp rx=0 tx=0).stop 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 msgr2=0x7fb398199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 0x7fb398199280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb388004770 tx=0x7fb3880042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 shutdown_connections 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb3840a2c10 0x7fb3840a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb398106a90 0x7fb3981a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb398101430 0x7fb3981997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 --2- 172.21.15.143:0/334994757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb398100a90 0x7fb398199280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.173+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 >> 172.21.15.143:0/334994757 conn(0x7fb3980fc7d0 msgr2=0x7fb3980fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.174+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 shutdown_connections 2024-09-10T10:44:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:15.174+0000 7fb39c9dd640 1 -- 172.21.15.143:0/334994757 wait complete. 2024-09-10T10:44:15.762 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:15 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/334994757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:44:15.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:15 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/334994757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:44:15.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:15 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/334994757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:44:16.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/15212674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 msgr2=0x7f9dc410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/15212674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc410bb50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009900 tx=0x7f9db402a480 comp rx=0 tx=0).stop 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/15212674 shutdown_connections 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/15212674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 0x7f9dc410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/15212674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc410bb50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/15212674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc4102d40 0x7f9dc4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.086+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/15212674 >> 172.21.15.143:0/15212674 conn(0x7f9dc40fc7a0 msgr2=0x7f9dc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/15212674 shutdown_connections 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/15212674 wait complete. 2024-09-10T10:44:16.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 Processor -- start 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 -- start start 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc4102d40 0x7f9dc419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 0x7f9dc419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111e20 con 0x7f9dc4103660 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111ca0 con 0x7f9dc4102d40 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dcb3d2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111fa0 con 0x7f9dc410c170 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dc896d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dc896d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48096/0 (socket says 172.21.15.143:48096) 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.087+0000 7f9dc896d640 1 -- 172.21.15.143:0/4044065584 learned_addr learned my addr 172.21.15.143:0/4044065584 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc916e640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc4102d40 0x7f9dc419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc996f640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 0x7f9dc419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc896d640 1 -- 172.21.15.143:0/4044065584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 msgr2=0x7f9dc419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc896d640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 0x7f9dc419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc896d640 1 -- 172.21.15.143:0/4044065584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc4102d40 msgr2=0x7f9dc419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc896d640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc4102d40 0x7f9dc419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc896d640 1 -- 172.21.15.143:0/4044065584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc419f530 con 0x7f9dc4103660 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc996f640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 0x7f9dc419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dc896d640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc419aac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9db4005940 tx=0x7f9db40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db403f030 con 0x7f9dc4103660 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc419f7c0 con 0x7f9dc4103660 2024-09-10T10:44:16.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9db4003cb0 con 0x7f9dc4103660 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4035070 con 0x7f9dc4103660 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.088+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc41a7100 con 0x7f9dc4103660 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.089+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc4103120 con 0x7f9dc4103660 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.090+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9db4042030 con 0x7f9dc4103660 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.091+0000 7f9dba7fc640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9db00a2c10 0x7f9db00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.091+0000 7f9dc916e640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9db00a2c10 0x7f9db00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.091+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9db40f0050 con 0x7f9dc4103660 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.092+0000 7f9dc916e640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9db00a2c10 0x7f9db00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00058c0 tx=0x7f9dc000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.094+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9db40bcaf0 con 0x7f9dc4103660 2024-09-10T10:44:16.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.264+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9dc41a72d0 con 0x7f9dc4103660 2024-09-10T10:44:16.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.264+0000 7f9dba7fc640 1 -- 172.21.15.143:0/4044065584 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9db40bc3f0 con 0x7f9dc4103660 2024-09-10T10:44:16.267 INFO:teuthology.orchestra.run.smithi143.stdout:210453397554 2024-09-10T10:44:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9db00a2c10 msgr2=0x7f9db00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9db00a2c10 0x7f9db00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00058c0 tx=0x7f9dc000a4c0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 msgr2=0x7f9dc419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc419aac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9db4005940 tx=0x7f9db40042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 shutdown_connections 2024-09-10T10:44:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9db00a2c10 0x7f9db00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dc410c170 0x7f9dc419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4103660 0x7f9dc419aac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 --2- 172.21.15.143:0/4044065584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc4102d40 0x7f9dc419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 >> 172.21.15.143:0/4044065584 conn(0x7f9dc40fc7a0 msgr2=0x7f9dc4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 shutdown_connections 2024-09-10T10:44:16.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:16.267+0000 7f9dcb3d2640 1 -- 172.21.15.143:0/4044065584 wait complete. 2024-09-10T10:44:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:16 smithi178 ceph-mon[34813]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T10:44:16.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:16 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4044065584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:44:16.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:16 smithi144 ceph-mon[34230]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T10:44:16.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:16 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4044065584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:44:17.040 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:16 smithi143 ceph-mon[31057]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T10:44:17.040 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:16 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4044065584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:44:17.114 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561078 got 188978561079 for osd.6 2024-09-10T10:44:17.114 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 -- 172.21.15.143:0/2850533281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8101410 msgr2=0x7f84a810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/2850533281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8101410 0x7f84a810d940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009cd0 tx=0x7f84a002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 -- 172.21.15.143:0/2850533281 shutdown_connections 2024-09-10T10:44:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/2850533281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8101410 0x7f84a810d940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/2850533281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8100a70 0x7f84a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/2850533281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8106a70 0x7f84a8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 -- 172.21.15.143:0/2850533281 >> 172.21.15.143:0/2850533281 conn(0x7f84a80fc790 msgr2=0x7f84a80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 -- 172.21.15.143:0/2850533281 shutdown_connections 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 -- 172.21.15.143:0/2850533281 wait complete. 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 Processor -- start 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.654+0000 7f84ac8e7640 1 -- start start 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84ac8e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 0x7f84a8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84ac8e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8101410 0x7f84a8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84ac8e7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8106a70 0x7f84a81a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84ac8e7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8110d20 con 0x7f84a8106a70 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84ac8e7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8110ba0 con 0x7f84a8100a70 2024-09-10T10:44:17.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84ac8e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8110ea0 con 0x7f84a8101410 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 0x7f84a8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 0x7f84a8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:47228/0 (socket says 172.21.15.143:47228) 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 -- 172.21.15.143:0/1897495847 learned_addr learned my addr 172.21.15.143:0/1897495847 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f849ffff640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8101410 0x7f84a8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a6d9d640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8106a70 0x7f84a81a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 -- 172.21.15.143:0/1897495847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8101410 msgr2=0x7f84a8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8101410 0x7f84a8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 -- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8106a70 msgr2=0x7f84a81a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8106a70 0x7f84a81a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 -- 172.21.15.143:0/1897495847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a81a47d0 con 0x7f84a8100a70 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f849ffff640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8101410 0x7f84a8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.655+0000 7f84a659c640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 0x7f84a8199210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f849800d950 tx=0x7f849800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.656+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498017070 con 0x7f84a8100a70 2024-09-10T10:44:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.656+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a8197000 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.656+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8498009c30 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.656+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849800f620 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.656+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a81974f0 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.657+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a8100ed0 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.658+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f84980109a0 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.658+0000 7f849f7fe640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f84940a2c10 0x7f84940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.658+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f84980cf040 con 0x7f84a8100a70 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.658+0000 7f849ffff640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f84940a2c10 0x7f84940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.659+0000 7f849ffff640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f84940a2c10 0x7f84940a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f84a806b2e0 tx=0x7f8490012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.662+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8498009de0 con 0x7f84a8100a70 2024-09-10T10:44:17.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.844+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f84a8109000 con 0x7f84a8100a70 2024-09-10T10:44:17.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.844+0000 7f849f7fe640 1 -- 172.21.15.143:0/1897495847 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f849809b490 con 0x7f84a8100a70 2024-09-10T10:44:17.847 INFO:teuthology.orchestra.run.smithi143.stdout:158913790014 2024-09-10T10:44:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f84940a2c10 msgr2=0x7f84940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f84940a2c10 0x7f84940a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f84a806b2e0 tx=0x7f8490012040 comp rx=0 tx=0).stop 2024-09-10T10:44:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 msgr2=0x7f84a8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 0x7f84a8199210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f849800d950 tx=0x7f849800de10 comp rx=0 tx=0).stop 2024-09-10T10:44:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 shutdown_connections 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f84940a2c10 0x7f84940a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84a8106a70 0x7f84a81a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a8101410 0x7f84a8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 --2- 172.21.15.143:0/1897495847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a8100a70 0x7f84a8199210 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 >> 172.21.15.143:0/1897495847 conn(0x7f84a80fc790 msgr2=0x7f84a80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 shutdown_connections 2024-09-10T10:44:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.847+0000 7f84ac8e7640 1 -- 172.21.15.143:0/1897495847 wait complete. 2024-09-10T10:44:18.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397551 got 210453397554 for osd.7 2024-09-10T10:44:18.146 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:18.635 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:18 smithi143 ceph-mon[31057]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-10T10:44:18.635 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3540316889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:44:18.635 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1897495847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:44:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:18 smithi178 ceph-mon[34813]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-10T10:44:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:18 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3540316889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:44:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:18 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1897495847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:44:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:18 smithi144 ceph-mon[34230]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-10T10:44:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:18 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3540316889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:44:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:18 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1897495847' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:44:19.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 -- 172.21.15.143:0/3428190660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 msgr2=0x7f0ba01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3428190660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba01021d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009900 tx=0x7f0b9002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 -- 172.21.15.143:0/3428190660 shutdown_connections 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3428190660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba0106f20 0x7f0ba010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3428190660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba01027a0 0x7f0ba01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3428190660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba01021d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 -- 172.21.15.143:0/3428190660 >> 172.21.15.143:0/3428190660 conn(0x7f0ba00fd8f0 msgr2=0x7f0ba00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 -- 172.21.15.143:0/3428190660 shutdown_connections 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.988+0000 7f0ba5458640 1 -- 172.21.15.143:0/3428190660 wait complete. 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.989+0000 7f0ba5458640 1 Processor -- start 2024-09-10T10:44:19.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.989+0000 7f0ba5458640 1 -- start start 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.989+0000 7f0ba5458640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.989+0000 7f0ba5458640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba01027a0 0x7f0ba01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0ba5458640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba0106f20 0x7f0ba019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0ba5458640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0111e30 con 0x7f0ba0101df0 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0ba5458640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0111cb0 con 0x7f0ba0106f20 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0ba5458640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0111fb0 con 0x7f0ba01027a0 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba0198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41664/0 (socket says 172.21.15.143:41664) 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 -- 172.21.15.143:0/3616562107 learned_addr learned my addr 172.21.15.143:0/3616562107 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9f7fe640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba0106f20 0x7f0ba019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9e7fc640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba01027a0 0x7f0ba01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 -- 172.21.15.143:0/3616562107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba01027a0 msgr2=0x7f0ba01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba01027a0 0x7f0ba01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 -- 172.21.15.143:0/3616562107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba0106f20 msgr2=0x7f0ba019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba0106f20 0x7f0ba019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 -- 172.21.15.143:0/3616562107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba019d220 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9f7fe640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba0106f20 0x7f0ba019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b9effd640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba0198270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006010 tx=0x7f0b90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9003f030 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba019d4b0 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b900035f0 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90035070 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.990+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a4df0 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.991+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba01021d0 con 0x7f0ba0101df0 2024-09-10T10:44:19.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.993+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0b90003b10 con 0x7f0ba0101df0 2024-09-10T10:44:19.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.993+0000 7f0b87fff640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0b880a2c10 0x7f0b880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.994+0000 7f0b9e7fc640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0b880a2c10 0x7f0b880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.994+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0b900f0370 con 0x7f0ba0101df0 2024-09-10T10:44:19.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.994+0000 7f0b9e7fc640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0b880a2c10 0x7f0b880a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b94005e10 tx=0x7f0b94005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:19.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:18.997+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b900bce10 con 0x7f0ba0101df0 2024-09-10T10:44:19.176 INFO:teuthology.orchestra.run.smithi143.stdout:270582939689 2024-09-10T10:44:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.172+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f0ba010a100 con 0x7f0ba0101df0 2024-09-10T10:44:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.172+0000 7f0b87fff640 1 -- 172.21.15.143:0/3616562107 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b900bc710 con 0x7f0ba0101df0 2024-09-10T10:44:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0b880a2c10 msgr2=0x7f0b880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0b880a2c10 0x7f0b880a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b94005e10 tx=0x7f0b94005da0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 msgr2=0x7f0ba0198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba0198270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006010 tx=0x7f0b90004270 comp rx=0 tx=0).stop 2024-09-10T10:44:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 shutdown_connections 2024-09-10T10:44:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0b880a2c10 0x7f0b880a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ba0106f20 0x7f0ba019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ba01027a0 0x7f0ba01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 --2- 172.21.15.143:0/3616562107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ba0101df0 0x7f0ba0198270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.175+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 >> 172.21.15.143:0/3616562107 conn(0x7f0ba00fd8f0 msgr2=0x7f0ba00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.176+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 shutdown_connections 2024-09-10T10:44:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.176+0000 7f0ba5458640 1 -- 172.21.15.143:0/3616562107 wait complete. 2024-09-10T10:44:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 -- 172.21.15.143:0/3680429103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054101450 msgr2=0x7fb054105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 --2- 172.21.15.143:0/3680429103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054101450 0x7fb054105920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb044009900 tx=0x7fb04402a470 comp rx=0 tx=0).stop 2024-09-10T10:44:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 -- 172.21.15.143:0/3680429103 shutdown_connections 2024-09-10T10:44:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 --2- 172.21.15.143:0/3680429103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054105f40 0x7fb0541063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 --2- 172.21.15.143:0/3680429103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054101450 0x7fb054105920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 --2- 172.21.15.143:0/3680429103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb054100b30 0x7fb054100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 -- 172.21.15.143:0/3680429103 >> 172.21.15.143:0/3680429103 conn(0x7fb0540fc7a0 msgr2=0x7fb0540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 -- 172.21.15.143:0/3680429103 shutdown_connections 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.258+0000 7fb05bee1640 1 -- 172.21.15.143:0/3680429103 wait complete. 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 Processor -- start 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 -- start start 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054100b30 0x7fb0541a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb054101450 0x7fb0541a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 0x7fb05419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb054118410 con 0x7fb054105f40 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb054118290 con 0x7fb054100b30 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05bee1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb054118590 con 0x7fb054101450 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 0x7fb05419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 0x7fb05419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41680/0 (socket says 172.21.15.143:41680) 2024-09-10T10:44:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 -- 172.21.15.143:0/2503688584 learned_addr learned my addr 172.21.15.143:0/2503688584 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb059c7d640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054100b30 0x7fb0541a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 -- 172.21.15.143:0/2503688584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb054101450 msgr2=0x7fb0541a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb054101450 0x7fb0541a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 -- 172.21.15.143:0/2503688584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054100b30 msgr2=0x7fb0541a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054100b30 0x7fb0541a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 -- 172.21.15.143:0/2503688584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05419ef60 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb059c7d640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054100b30 0x7fb0541a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb05a47e640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 0x7fb05419e7c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eaa0 tx=0x7fb04c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00cdb0 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.259+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb04c005020 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.260+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05419f250 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.260+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0116d0 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.260+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541ab4a0 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.260+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05406b070 con 0x7fb054105f40 2024-09-10T10:44:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.262+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb04c0051d0 con 0x7fb054105f40 2024-09-10T10:44:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.262+0000 7fb04affd640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0400a2c10 0x7fb0400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.262+0000 7fb059c7d640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0400a2c10 0x7fb0400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.262+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb04c0cfb30 con 0x7fb054105f40 2024-09-10T10:44:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.263+0000 7fb059c7d640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0400a2c10 0x7fb0400a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0500058c0 tx=0x7fb050005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.266+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb04c016030 con 0x7fb054105f40 2024-09-10T10:44:19.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.436+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb0541ab730 con 0x7fb054105f40 2024-09-10T10:44:19.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.436+0000 7fb04affd640 1 -- 172.21.15.143:0/2503688584 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb04c09c5d0 con 0x7fb054105f40 2024-09-10T10:44:19.438 INFO:teuthology.orchestra.run.smithi143.stdout:103079215175 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.438+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0400a2c10 msgr2=0x7fb0400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.438+0000 7fb05bee1640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0400a2c10 0x7fb0400a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0500058c0 tx=0x7fb050005850 comp rx=0 tx=0).stop 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.438+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 msgr2=0x7fb05419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.438+0000 7fb05bee1640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 0x7fb05419e7c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00eaa0 tx=0x7fb04c00ef60 comp rx=0 tx=0).stop 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 shutdown_connections 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb0400a2c10 0x7fb0400a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054105f40 0x7fb05419e7c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb054101450 0x7fb0541a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 --2- 172.21.15.143:0/2503688584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb054100b30 0x7fb0541a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 >> 172.21.15.143:0/2503688584 conn(0x7fb0540fc7a0 msgr2=0x7fb05410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:19.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 shutdown_connections 2024-09-10T10:44:19.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:19.439+0000 7fb05bee1640 1 -- 172.21.15.143:0/2503688584 wait complete. 2024-09-10T10:44:19.711 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:19 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3616562107' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:44:19.711 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:19 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2503688584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:44:19.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:19 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3616562107' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:44:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:19 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2503688584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:44:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:19 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3616562107' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:44:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:19 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2503688584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:44:19.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168619 got 240518168621 for osd.8 2024-09-10T10:44:19.966 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 -- 172.21.15.143:0/1702344346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 msgr2=0x7ff71c10df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 --2- 172.21.15.143:0/1702344346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c10df00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009900 tx=0x7ff70c02a480 comp rx=0 tx=0).stop 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 -- 172.21.15.143:0/1702344346 shutdown_connections 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 --2- 172.21.15.143:0/1702344346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c10e440 0x7ff71c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 --2- 172.21.15.143:0/1702344346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c10df00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 --2- 172.21.15.143:0/1702344346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c074510 0x7ff71c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.142+0000 7ff72349e640 1 -- 172.21.15.143:0/1702344346 >> 172.21.15.143:0/1702344346 conn(0x7ff71c0fd8e0 msgr2=0x7ff71c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 -- 172.21.15.143:0/1702344346 shutdown_connections 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 -- 172.21.15.143:0/1702344346 wait complete. 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 Processor -- start 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 -- start start 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c074510 0x7ff71c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c10e440 0x7ff71c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c114010 con 0x7ff71c074e30 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c113e90 con 0x7ff71c10e440 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff72349e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c114190 con 0x7ff71c074510 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.143+0000 7ff720a39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41708/0 (socket says 172.21.15.143:41708) 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 -- 172.21.15.143:0/3773098435 learned_addr learned my addr 172.21.15.143:0/3773098435 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff72123a640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c074510 0x7ff71c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff721a3b640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c10e440 0x7ff71c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 -- 172.21.15.143:0/3773098435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c074510 msgr2=0x7ff71c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c074510 0x7ff71c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 -- 172.21.15.143:0/3773098435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c10e440 msgr2=0x7ff71c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c10e440 0x7ff71c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 -- 172.21.15.143:0/3773098435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c19f6a0 con 0x7ff71c074e30 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff720a39640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c19ac30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005940 tx=0x7ff70c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c03f030 con 0x7ff71c074e30 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff70c003aa0 con 0x7ff71c074e30 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c19f930 con 0x7ff71c074e30 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c035070 con 0x7ff71c074e30 2024-09-10T10:44:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.144+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a7270 con 0x7ff71c074e30 2024-09-10T10:44:20.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.145+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c0748f0 con 0x7ff71c074e30 2024-09-10T10:44:20.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.146+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff70c042030 con 0x7ff71c074e30 2024-09-10T10:44:20.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.148+0000 7ff7127fc640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff7080a2c10 0x7ff7080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.148+0000 7ff72123a640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff7080a2c10 0x7ff7080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.148+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff70c0efff0 con 0x7ff71c074e30 2024-09-10T10:44:20.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.148+0000 7ff72123a640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff7080a2c10 0x7ff7080a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff718005fd0 tx=0x7ff718009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.151+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff70c0bca90 con 0x7ff71c074e30 2024-09-10T10:44:20.169 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:20.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.325+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff71c1095c0 con 0x7ff71c074e30 2024-09-10T10:44:20.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.325+0000 7ff7127fc640 1 -- 172.21.15.143:0/3773098435 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff70c0bc390 con 0x7ff71c074e30 2024-09-10T10:44:20.327 INFO:teuthology.orchestra.run.smithi143.stdout:292057776164 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff7080a2c10 msgr2=0x7ff7080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff7080a2c10 0x7ff7080a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff718005fd0 tx=0x7ff718009470 comp rx=0 tx=0).stop 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 msgr2=0x7ff71c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c19ac30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005940 tx=0x7ff70c004270 comp rx=0 tx=0).stop 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 shutdown_connections 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff7080a2c10 0x7ff7080a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c10e440 0x7ff71c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff71c074e30 0x7ff71c19ac30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 --2- 172.21.15.143:0/3773098435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c074510 0x7ff71c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.327+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 >> 172.21.15.143:0/3773098435 conn(0x7ff71c0fd8e0 msgr2=0x7ff71c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.328+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 shutdown_connections 2024-09-10T10:44:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:20.328+0000 7ff72349e640 1 -- 172.21.15.143:0/3773098435 wait complete. 2024-09-10T10:44:20.723 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:20 smithi143 ceph-mon[31057]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:20.723 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:20 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3773098435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:44:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:20 smithi178 ceph-mon[34813]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:20 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3773098435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:44:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:20 smithi144 ceph-mon[34230]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:20 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3773098435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:44:21.100 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776164 for osd.10 2024-09-10T10:44:21.101 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3830946074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 msgr2=0x7f7a10106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3830946074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a10106e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009930 tx=0x7f7a0002a420 comp rx=0 tx=0).stop 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3830946074 shutdown_connections 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3830946074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 0x7f7a1010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3830946074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10100a90 0x7f7a10100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3830946074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a10106e70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3830946074 >> 172.21.15.143:0/3830946074 conn(0x7f7a100fc7d0 msgr2=0x7f7a100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3830946074 shutdown_connections 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3830946074 wait complete. 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.539+0000 7f7a15cfb640 1 Processor -- start 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 -- start start 2024-09-10T10:44:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10100a90 0x7f7a10198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 0x7f7a101988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a1019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10111e20 con 0x7f7a10106a90 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10111ca0 con 0x7f7a10101430 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a15cfb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10111fa0 con 0x7f7a10100a90 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a1019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a1019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41632/0 (socket says 172.21.15.143:41632) 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 -- 172.21.15.143:0/3540316889 learned_addr learned my addr 172.21.15.143:0/3540316889 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0f7fe640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10100a90 0x7f7a10198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:21.331 INFO:teuthology.orchestra.run.smithi143.stdout:322122547228 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0effd640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 0x7f7a101988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 -- 172.21.15.143:0/3540316889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10100a90 msgr2=0x7f7a10198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10100a90 0x7f7a10198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 -- 172.21.15.143:0/3540316889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 msgr2=0x7f7a101988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 0x7f7a101988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 -- 172.21.15.143:0/3540316889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1019d310 con 0x7f7a10106a90 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0effd640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 0x7f7a101988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.540+0000 7f7a0ffff640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a1019cc30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800eaa0 tx=0x7f7a0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.541+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0800cdb0 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.541+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a08005020 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.541+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a080116d0 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.541+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1019d600 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.541+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a4ee0 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.542+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10100ef0 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.543+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7a080051d0 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.543+0000 7f7a0cff9640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f79fc0a2c10 0x7f79fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.544+0000 7f7a0f7fe640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f79fc0a2c10 0x7f79fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.544+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7a080cf790 con 0x7f7a10106a90 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.544+0000 7f7a0f7fe640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f79fc0a2c10 0x7f79fc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000c00 tx=0x7f7a00032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:21.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.548+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a08016030 con 0x7f7a10106a90 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.728+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7a10109060 con 0x7f7a10106a90 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.728+0000 7f7a0cff9640 1 -- 172.21.15.143:0/3540316889 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7a0809c230 con 0x7f7a10106a90 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f79fc0a2c10 msgr2=0x7f79fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f79fc0a2c10 0x7f79fc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000c00 tx=0x7f7a00032c80 comp rx=0 tx=0).stop 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 msgr2=0x7f7a1019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a1019cc30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800eaa0 tx=0x7f7a0800ef60 comp rx=0 tx=0).stop 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 shutdown_connections 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f79fc0a2c10 0x7f79fc0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a10106a90 0x7f7a1019cc30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a10101430 0x7f7a101988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 --2- 172.21.15.143:0/3540316889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a10100a90 0x7f7a10198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 >> 172.21.15.143:0/3540316889 conn(0x7f7a100fc7d0 msgr2=0x7f7a100fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 shutdown_connections 2024-09-10T10:44:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:17.731+0000 7f7a15cfb640 1 -- 172.21.15.143:0/3540316889 wait complete. 2024-09-10T10:44:22.254 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547226 got 322122547228 for osd.11 2024-09-10T10:44:22.254 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:22.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 -- 172.21.15.143:0/1817131405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f88102d30 msgr2=0x7f5f88103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/1817131405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f88102d30 0x7f5f88103110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009930 tx=0x7f5f8402a440 comp rx=0 tx=0).stop 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 -- 172.21.15.143:0/1817131405 shutdown_connections 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/1817131405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f881082a0 0x7f5f8810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/1817131405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 0x7f5f88107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/1817131405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f88102d30 0x7f5f88103110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.388+0000 7f5f8fa91640 1 -- 172.21.15.143:0/1817131405 >> 172.21.15.143:0/1817131405 conn(0x7f5f880fc7d0 msgr2=0x7f5f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 -- 172.21.15.143:0/1817131405 shutdown_connections 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 -- 172.21.15.143:0/1817131405 wait complete. 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 Processor -- start 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 -- start start 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88102d30 0x7f5f88100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 0x7f5f88100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 0x7f5f88101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88111f90 con 0x7f5f881082a0 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88111e10 con 0x7f5f88102d30 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.389+0000 7f5f8fa91640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88112110 con 0x7f5f881036e0 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 0x7f5f88101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 0x7f5f88101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41720/0 (socket says 172.21.15.143:41720) 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 -- 172.21.15.143:0/3175961783 learned_addr learned my addr 172.21.15.143:0/3175961783 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8d02c640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 0x7f5f88100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8d82d640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88102d30 0x7f5f88100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 -- 172.21.15.143:0/3175961783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 msgr2=0x7f5f88100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 0x7f5f88100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 -- 172.21.15.143:0/3175961783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88102d30 msgr2=0x7f5f88100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88102d30 0x7f5f88100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 -- 172.21.15.143:0/3175961783 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f881a69d0 con 0x7f5f881082a0 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8d02c640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 0x7f5f88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8e02e640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 0x7f5f88101410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000cce0 tx=0x7f5f8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000f450 con 0x7f5f881082a0 2024-09-10T10:44:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8000faa0 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f881a6cc0 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80018b70 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.390+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f881a71b0 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.391+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8806a850 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.392+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5f80018d10 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.393+0000 7f5f7effd640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5f740a2c10 0x7f5f740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.393+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5f800cfa20 con 0x7f5f881082a0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.393+0000 7f5f8d82d640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5f740a2c10 0x7f5f740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.393+0000 7f5f8d82d640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5f740a2c10 0x7f5f740a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f84006010 tx=0x7f5f84000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.397+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f80014030 con 0x7f5f881082a0 2024-09-10T10:44:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.567+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5f8810a2b0 con 0x7f5f881082a0 2024-09-10T10:44:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.568+0000 7f5f7effd640 1 -- 172.21.15.143:0/3175961783 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f8009c4c0 con 0x7f5f881082a0 2024-09-10T10:44:22.569 INFO:teuthology.orchestra.run.smithi143.stdout:77309411406 2024-09-10T10:44:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5f740a2c10 msgr2=0x7f5f740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5f740a2c10 0x7f5f740a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f84006010 tx=0x7f5f84000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 msgr2=0x7f5f88101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 0x7f5f88101410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000cce0 tx=0x7f5f8000a620 comp rx=0 tx=0).stop 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 shutdown_connections 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5f740a2c10 0x7f5f740a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5f881082a0 0x7f5f88101410 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f881036e0 0x7f5f88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 --2- 172.21.15.143:0/3175961783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f88102d30 0x7f5f88100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 >> 172.21.15.143:0/3175961783 conn(0x7f5f880fc7d0 msgr2=0x7f5f88105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 shutdown_connections 2024-09-10T10:44:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:22.570+0000 7f5f8fa91640 1 -- 172.21.15.143:0/3175961783 wait complete. 2024-09-10T10:44:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:22 smithi178 ceph-mon[34813]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:44:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:22 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3175961783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:44:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:22 smithi144 ceph-mon[34230]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:44:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:22 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3175961783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:44:22.941 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:22 smithi143 ceph-mon[31057]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:44:22.941 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:22 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3175961783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:44:23.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411406 for osd.2 2024-09-10T10:44:23.279 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:23.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 -- 172.21.15.143:0/2918830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc101de0 msgr2=0x7f00bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 --2- 172.21.15.143:0/2918830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc101de0 0x7f00bc1021c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009900 tx=0x7f00b002a410 comp rx=0 tx=0).stop 2024-09-10T10:44:23.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 -- 172.21.15.143:0/2918830386 shutdown_connections 2024-09-10T10:44:23.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 --2- 172.21.15.143:0/2918830386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00bc106f10 0x7f00bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 --2- 172.21.15.143:0/2918830386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc102790 0x7f00bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 --2- 172.21.15.143:0/2918830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc101de0 0x7f00bc1021c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 -- 172.21.15.143:0/2918830386 >> 172.21.15.143:0/2918830386 conn(0x7f00bc0fd8c0 msgr2=0x7f00bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 -- 172.21.15.143:0/2918830386 shutdown_connections 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.438+0000 7f00c30b9640 1 -- 172.21.15.143:0/2918830386 wait complete. 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 Processor -- start 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 -- start start 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc101de0 0x7f00bc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00bc102790 0x7f00bc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 0x7f00bc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:23.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111e20 con 0x7f00bc106f10 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111ca0 con 0x7f00bc101de0 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c30b9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111fa0 con 0x7f00bc102790 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 0x7f00bc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 0x7f00bc19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41732/0 (socket says 172.21.15.143:41732) 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 -- 172.21.15.143:0/480525054 learned_addr learned my addr 172.21.15.143:0/480525054 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c0e55640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc101de0 0x7f00bc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 -- 172.21.15.143:0/480525054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00bc102790 msgr2=0x7f00bc198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00bbfff640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00bc102790 0x7f00bc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00bc102790 0x7f00bc198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 -- 172.21.15.143:0/480525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc101de0 msgr2=0x7f00bc198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc101de0 0x7f00bc198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 -- 172.21.15.143:0/480525054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc19d2c0 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c0e55640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc101de0 0x7f00bc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.439+0000 7f00c1656640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 0x7f00bc19cbe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f00b400eaa0 tx=0x7f00b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.440+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b400cdb0 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.440+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc19d5b0 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.440+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00b4005020 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.440+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40116d0 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.440+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a4e90 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.441+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc1021c0 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.442+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f00b40051d0 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.443+0000 7f00b9ffb640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00ac0a2c10 0x7f00ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.443+0000 7f00c0e55640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00ac0a2c10 0x7f00ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.443+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f00b40cfb10 con 0x7f00bc106f10 2024-09-10T10:44:23.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.443+0000 7f00c0e55640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00ac0a2c10 0x7f00ac0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f00b0000c00 tx=0x7f00b0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:23.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.446+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00b4016030 con 0x7f00bc106f10 2024-09-10T10:44:23.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.619+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f00bc109b60 con 0x7f00bc106f10 2024-09-10T10:44:23.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.620+0000 7f00b9ffb640 1 -- 172.21.15.143:0/480525054 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f00b409c5b0 con 0x7f00bc106f10 2024-09-10T10:44:23.621 INFO:teuthology.orchestra.run.smithi143.stdout:137438953538 2024-09-10T10:44:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00ac0a2c10 msgr2=0x7f00ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00ac0a2c10 0x7f00ac0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f00b0000c00 tx=0x7f00b0031400 comp rx=0 tx=0).stop 2024-09-10T10:44:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 msgr2=0x7f00bc19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 0x7f00bc19cbe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f00b400eaa0 tx=0x7f00b400ef60 comp rx=0 tx=0).stop 2024-09-10T10:44:23.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 shutdown_connections 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00ac0a2c10 0x7f00ac0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00bc106f10 0x7f00bc19cbe0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00bc102790 0x7f00bc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 --2- 172.21.15.143:0/480525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00bc101de0 0x7f00bc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 >> 172.21.15.143:0/480525054 conn(0x7f00bc0fd8c0 msgr2=0x7f00bc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 shutdown_connections 2024-09-10T10:44:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:23.622+0000 7f00c30b9640 1 -- 172.21.15.143:0/480525054 wait complete. 2024-09-10T10:44:23.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939685 got 270582939689 for osd.9 2024-09-10T10:44:23.900 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:24.432 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790010 got 158913790014 for osd.5 2024-09-10T10:44:24.432 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:24.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:24 smithi143 ceph-mon[31057]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:44:24.799 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:24 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/480525054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:44:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:24 smithi178 ceph-mon[34813]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:44:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:24 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/480525054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:44:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:24 smithi144 ceph-mon[34230]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:44:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:24 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/480525054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:44:25.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953536 got 137438953538 for osd.4 2024-09-10T10:44:25.157 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:25.731 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215173 got 103079215175 for osd.3 2024-09-10T10:44:25.731 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:25.732 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:44:25.732 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:44:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:26.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 -- 172.21.15.143:0/3328258280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108102dd0 msgr2=0x7f61081031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 --2- 172.21.15.143:0/3328258280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108102dd0 0x7f61081031b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009930 tx=0x7f60f802a420 comp rx=0 tx=0).stop 2024-09-10T10:44:26.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 -- 172.21.15.143:0/3328258280 shutdown_connections 2024-09-10T10:44:26.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 --2- 172.21.15.143:0/3328258280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6108108300 0x7f610810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 --2- 172.21.15.143:0/3328258280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6108103780 0x7f6108107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 --2- 172.21.15.143:0/3328258280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108102dd0 0x7f61081031b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 -- 172.21.15.143:0/3328258280 >> 172.21.15.143:0/3328258280 conn(0x7f61080fc7d0 msgr2=0x7f61080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 -- 172.21.15.143:0/3328258280 shutdown_connections 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.529+0000 7f610f17f640 1 -- 172.21.15.143:0/3328258280 wait complete. 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 Processor -- start 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 -- start start 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6108102dd0 0x7f6108100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 0x7f6108100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6108108300 0x7f61081014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6108111f60 con 0x7f6108103780 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6108111de0 con 0x7f6108102dd0 2024-09-10T10:44:26.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610f17f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61081120e0 con 0x7f6108108300 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f6107fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 0x7f6108100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f6107fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 0x7f6108100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45060/0 (socket says 172.21.15.143:45060) 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f6107fff640 1 -- 172.21.15.143:0/511679688 learned_addr learned my addr 172.21.15.143:0/511679688 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.530+0000 7f610cf1b640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6108102dd0 0x7f6108100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f610d71c640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6108108300 0x7f61081014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6107fff640 1 -- 172.21.15.143:0/511679688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6108108300 msgr2=0x7f61081014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6107fff640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6108108300 0x7f61081014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6107fff640 1 -- 172.21.15.143:0/511679688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6108102dd0 msgr2=0x7f6108100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6107fff640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6108102dd0 0x7f6108100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6107fff640 1 -- 172.21.15.143:0/511679688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61081a69a0 con 0x7f6108103780 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6107fff640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 0x7f6108100f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c970 tx=0x7f60fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc007bb0 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60fc007d50 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc00f450 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61081a6c90 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.531+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081a71c0 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.532+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61081031b0 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.533+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f60fc004eb0 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.534+0000 7f6105ffb640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f60f00a2c90 0x7f60f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.534+0000 7f610cf1b640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f60f00a2c90 0x7f60f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.535+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f60fc0cf200 con 0x7f6108103780 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.535+0000 7f610cf1b640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f60f00a2c90 0x7f60f00a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009900 tx=0x7f60f8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.539+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60fc014030 con 0x7f6108103780 2024-09-10T10:44:26.706 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:26 smithi143 ceph-mon[31057]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:26.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.704+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f610810a300 con 0x7f60f00a2c90 2024-09-10T10:44:26.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.712+0000 7f6105ffb640 1 -- 172.21.15.143:0/511679688 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340430 (secure 0 0 0) 0x7f60f80314a0 con 0x7f60f00a2c90 2024-09-10T10:44:26.717 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:26.718 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.716+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f60f00a2c90 msgr2=0x7f60f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.716+0000 7f610f17f640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f60f00a2c90 0x7f60f00a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009900 tx=0x7f60f8000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.716+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 msgr2=0x7f6108100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.716+0000 7f610f17f640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 0x7f6108100f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c970 tx=0x7f60fc00ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.716+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 shutdown_connections 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f60f00a2c90 0x7f60f00a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6108108300 0x7f61081014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6108103780 0x7f6108100f70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 --2- 172.21.15.143:0/511679688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6108102dd0 0x7f6108100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 >> 172.21.15.143:0/511679688 conn(0x7f61080fc7d0 msgr2=0x7f6108105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:26.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 shutdown_connections 2024-09-10T10:44:26.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:26.717+0000 7f610f17f640 1 -- 172.21.15.143:0/511679688 wait complete. 2024-09-10T10:44:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:26 smithi178 ceph-mon[34813]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:26 smithi144 ceph-mon[34230]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:27.399 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T10:44:25.441596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659886,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":652,"num_read_kb":449,"num_write":431,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375852,"kb_used_data":14036,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565972,"statfs":{"total":1151940427776,"available":1151555555328,"internally_reserved":0,"allocated":14372864,"data_stored":7948712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003255"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:19.636900+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:44:19.636900+0000","last_peered":"2024-09-10T10:44:19.636900+0000","last_clean":"2024-09-10T10:44:19.636900+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:44:19.636900+0000","last_undegraded":"2024-09-10T10:44:19.636900+0000","last_fullsized":"2024-09-10T10:44:19.636900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504790+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:00.504790+0000","last_peered":"2024-09-10T10:44:00.504790+0000","last_clean":"2024-09-10T10:44:00.504790+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:00.504790+0000","last_undegraded":"2024-09-10T10:44:00.504790+0000","last_fullsized":"2024-09-10T10:44:00.504790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500745+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500745+0000","last_peered":"2024-09-10T10:44:00.500745+0000","last_clean":"2024-09-10T10:44:00.500745+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500745+0000","last_undegraded":"2024-09-10T10:44:00.500745+0000","last_fullsized":"2024-09-10T10:44:00.500745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505003+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505003+0000","last_peered":"2024-09-10T10:44:00.505003+0000","last_clean":"2024-09-10T10:44:00.505003+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505003+0000","last_undegraded":"2024-09-10T10:44:00.505003+0000","last_fullsized":"2024-09-10T10:44:00.505003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501309+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:00.501309+0000","last_peered":"2024-09-10T10:44:00.501309+0000","last_clean":"2024-09-10T10:44:00.501309+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:00.501309+0000","last_undegraded":"2024-09-10T10:44:00.501309+0000","last_fullsized":"2024-09-10T10:44:00.501309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503391+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503391+0000","last_peered":"2024-09-10T10:44:00.503391+0000","last_clean":"2024-09-10T10:44:00.503391+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503391+0000","last_undegraded":"2024-09-10T10:44:00.503391+0000","last_fullsized":"2024-09-10T10:44:00.503391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506237+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:44:00.506237+0000","last_peered":"2024-09-10T10:44:00.506237+0000","last_clean":"2024-09-10T10:44:00.506237+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:44:00.506237+0000","last_undegraded":"2024-09-10T10:44:00.506237+0000","last_fullsized":"2024-09-10T10:44:00.506237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509720+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:00.509720+0000","last_peered":"2024-09-10T10:44:00.509720+0000","last_clean":"2024-09-10T10:44:00.509720+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:00.509720+0000","last_undegraded":"2024-09-10T10:44:00.509720+0000","last_fullsized":"2024-09-10T10:44:00.509720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505929+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505929+0000","last_peered":"2024-09-10T10:44:00.505929+0000","last_clean":"2024-09-10T10:44:00.505929+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505929+0000","last_undegraded":"2024-09-10T10:44:00.505929+0000","last_fullsized":"2024-09-10T10:44:00.505929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516100+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:00.516100+0000","last_peered":"2024-09-10T10:44:00.516100+0000","last_clean":"2024-09-10T10:44:00.516100+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:00.516100+0000","last_undegraded":"2024-09-10T10:44:00.516100+0000","last_fullsized":"2024-09-10T10:44:00.516100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506910+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:00.506910+0000","last_peered":"2024-09-10T10:44:00.506910+0000","last_clean":"2024-09-10T10:44:00.506910+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:00.506910+0000","last_undegraded":"2024-09-10T10:44:00.506910+0000","last_fullsized":"2024-09-10T10:44:00.506910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503914+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503914+0000","last_peered":"2024-09-10T10:44:00.503914+0000","last_clean":"2024-09-10T10:44:00.503914+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503914+0000","last_undegraded":"2024-09-10T10:44:00.503914+0000","last_fullsized":"2024-09-10T10:44:00.503914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505145+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:00.505145+0000","last_peered":"2024-09-10T10:44:00.505145+0000","last_clean":"2024-09-10T10:44:00.505145+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:00.505145+0000","last_undegraded":"2024-09-10T10:44:00.505145+0000","last_fullsized":"2024-09-10T10:44:00.505145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500867+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:44:00.500867+0000","last_peered":"2024-09-10T10:44:00.500867+0000","last_clean":"2024-09-10T10:44:00.500867+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:44:00.500867+0000","last_undegraded":"2024-09-10T10:44:00.500867+0000","last_fullsized":"2024-09-10T10:44:00.500867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.523722+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.523722+0000","last_peered":"2024-09-10T10:44:00.523722+0000","last_clean":"2024-09-10T10:44:00.523722+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.523722+0000","last_undegraded":"2024-09-10T10:44:00.523722+0000","last_fullsized":"2024-09-10T10:44:00.523722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502524+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:00.502524+0000","last_peered":"2024-09-10T10:44:00.502524+0000","last_clean":"2024-09-10T10:44:00.502524+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:00.502524+0000","last_undegraded":"2024-09-10T10:44:00.502524+0000","last_fullsized":"2024-09-10T10:44:00.502524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504899+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504899+0000","last_peered":"2024-09-10T10:44:00.504899+0000","last_clean":"2024-09-10T10:44:00.504899+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504899+0000","last_undegraded":"2024-09-10T10:44:00.504899+0000","last_fullsized":"2024-09-10T10:44:00.504899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500299+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500299+0000","last_peered":"2024-09-10T10:44:00.500299+0000","last_clean":"2024-09-10T10:44:00.500299+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500299+0000","last_undegraded":"2024-09-10T10:44:00.500299+0000","last_fullsized":"2024-09-10T10:44:00.500299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505392+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:00.505392+0000","last_peered":"2024-09-10T10:44:00.505392+0000","last_clean":"2024-09-10T10:44:00.505392+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:00.505392+0000","last_undegraded":"2024-09-10T10:44:00.505392+0000","last_fullsized":"2024-09-10T10:44:00.505392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510007+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510007+0000","last_peered":"2024-09-10T10:44:00.510007+0000","last_clean":"2024-09-10T10:44:00.510007+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510007+0000","last_undegraded":"2024-09-10T10:44:00.510007+0000","last_fullsized":"2024-09-10T10:44:00.510007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504053+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504053+0000","last_peered":"2024-09-10T10:44:00.504053+0000","last_clean":"2024-09-10T10:44:00.504053+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504053+0000","last_undegraded":"2024-09-10T10:44:00.504053+0000","last_fullsized":"2024-09-10T10:44:00.504053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500989+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:00.500989+0000","last_peered":"2024-09-10T10:44:00.500989+0000","last_clean":"2024-09-10T10:44:00.500989+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:00.500989+0000","last_undegraded":"2024-09-10T10:44:00.500989+0000","last_fullsized":"2024-09-10T10:44:00.500989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504293+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504293+0000","last_peered":"2024-09-10T10:44:00.504293+0000","last_clean":"2024-09-10T10:44:00.504293+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504293+0000","last_undegraded":"2024-09-10T10:44:00.504293+0000","last_fullsized":"2024-09-10T10:44:00.504293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500883+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500883+0000","last_peered":"2024-09-10T10:44:00.500883+0000","last_clean":"2024-09-10T10:44:00.500883+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500883+0000","last_undegraded":"2024-09-10T10:44:00.500883+0000","last_fullsized":"2024-09-10T10:44:00.500883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504671+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:00.504671+0000","last_peered":"2024-09-10T10:44:00.504671+0000","last_clean":"2024-09-10T10:44:00.504671+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:00.504671+0000","last_undegraded":"2024-09-10T10:44:00.504671+0000","last_fullsized":"2024-09-10T10:44:00.504671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500843+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:44:00.500843+0000","last_peered":"2024-09-10T10:44:00.500843+0000","last_clean":"2024-09-10T10:44:00.500843+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:44:00.500843+0000","last_undegraded":"2024-09-10T10:44:00.500843+0000","last_fullsized":"2024-09-10T10:44:00.500843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504281+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504281+0000","last_peered":"2024-09-10T10:44:00.504281+0000","last_clean":"2024-09-10T10:44:00.504281+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504281+0000","last_undegraded":"2024-09-10T10:44:00.504281+0000","last_fullsized":"2024-09-10T10:44:00.504281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502773+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:00.502773+0000","last_peered":"2024-09-10T10:44:00.502773+0000","last_clean":"2024-09-10T10:44:00.502773+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:00.502773+0000","last_undegraded":"2024-09-10T10:44:00.502773+0000","last_fullsized":"2024-09-10T10:44:00.502773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506283+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506283+0000","last_peered":"2024-09-10T10:44:00.506283+0000","last_clean":"2024-09-10T10:44:00.506283+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506283+0000","last_undegraded":"2024-09-10T10:44:00.506283+0000","last_fullsized":"2024-09-10T10:44:00.506283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503163+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503163+0000","last_peered":"2024-09-10T10:44:00.503163+0000","last_clean":"2024-09-10T10:44:00.503163+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503163+0000","last_undegraded":"2024-09-10T10:44:00.503163+0000","last_fullsized":"2024-09-10T10:44:00.503163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505442+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:00.505442+0000","last_peered":"2024-09-10T10:44:00.505442+0000","last_clean":"2024-09-10T10:44:00.505442+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:00.505442+0000","last_undegraded":"2024-09-10T10:44:00.505442+0000","last_fullsized":"2024-09-10T10:44:00.505442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501327+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501327+0000","last_peered":"2024-09-10T10:44:00.501327+0000","last_clean":"2024-09-10T10:44:00.501327+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501327+0000","last_undegraded":"2024-09-10T10:44:00.501327+0000","last_fullsized":"2024-09-10T10:44:00.501327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505290+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505290+0000","last_peered":"2024-09-10T10:44:00.505290+0000","last_clean":"2024-09-10T10:44:00.505290+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505290+0000","last_undegraded":"2024-09-10T10:44:00.505290+0000","last_fullsized":"2024-09-10T10:44:00.505290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505242+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:00.505242+0000","last_peered":"2024-09-10T10:44:00.505242+0000","last_clean":"2024-09-10T10:44:00.505242+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:00.505242+0000","last_undegraded":"2024-09-10T10:44:00.505242+0000","last_fullsized":"2024-09-10T10:44:00.505242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:18.636162+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:44:18.636162+0000","last_peered":"2024-09-10T10:44:18.636162+0000","last_clean":"2024-09-10T10:44:18.636162+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:44:18.636162+0000","last_undegraded":"2024-09-10T10:44:18.636162+0000","last_fullsized":"2024-09-10T10:44:18.636162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502584+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502584+0000","last_peered":"2024-09-10T10:44:00.502584+0000","last_clean":"2024-09-10T10:44:00.502584+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502584+0000","last_undegraded":"2024-09-10T10:44:00.502584+0000","last_fullsized":"2024-09-10T10:44:00.502584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515343+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:00.515343+0000","last_peered":"2024-09-10T10:44:00.515343+0000","last_clean":"2024-09-10T10:44:00.515343+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:00.515343+0000","last_undegraded":"2024-09-10T10:44:00.515343+0000","last_fullsized":"2024-09-10T10:44:00.515343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.629112+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:44:22.629112+0000","last_peered":"2024-09-10T10:44:22.629112+0000","last_clean":"2024-09-10T10:44:22.629112+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:44:22.629112+0000","last_undegraded":"2024-09-10T10:44:22.629112+0000","last_fullsized":"2024-09-10T10:44:22.629112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502695+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502695+0000","last_peered":"2024-09-10T10:44:00.502695+0000","last_clean":"2024-09-10T10:44:00.502695+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502695+0000","last_undegraded":"2024-09-10T10:44:00.502695+0000","last_fullsized":"2024-09-10T10:44:00.502695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502417+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:00.502417+0000","last_peered":"2024-09-10T10:44:00.502417+0000","last_clean":"2024-09-10T10:44:00.502417+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:00.502417+0000","last_undegraded":"2024-09-10T10:44:00.502417+0000","last_fullsized":"2024-09-10T10:44:00.502417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506043+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506043+0000","last_peered":"2024-09-10T10:44:00.506043+0000","last_clean":"2024-09-10T10:44:00.506043+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506043+0000","last_undegraded":"2024-09-10T10:44:00.506043+0000","last_fullsized":"2024-09-10T10:44:00.506043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515952+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.515952+0000","last_peered":"2024-09-10T10:44:00.515952+0000","last_clean":"2024-09-10T10:44:00.515952+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.515952+0000","last_undegraded":"2024-09-10T10:44:00.515952+0000","last_fullsized":"2024-09-10T10:44:00.515952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516146+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:00.516146+0000","last_peered":"2024-09-10T10:44:00.516146+0000","last_clean":"2024-09-10T10:44:00.516146+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:00.516146+0000","last_undegraded":"2024-09-10T10:44:00.516146+0000","last_fullsized":"2024-09-10T10:44:00.516146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503897+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503897+0000","last_peered":"2024-09-10T10:44:00.503897+0000","last_clean":"2024-09-10T10:44:00.503897+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503897+0000","last_undegraded":"2024-09-10T10:44:00.503897+0000","last_fullsized":"2024-09-10T10:44:00.503897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506228+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506228+0000","last_peered":"2024-09-10T10:44:00.506228+0000","last_clean":"2024-09-10T10:44:00.506228+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506228+0000","last_undegraded":"2024-09-10T10:44:00.506228+0000","last_fullsized":"2024-09-10T10:44:00.506228+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507045+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507045+0000","last_peered":"2024-09-10T10:44:00.507045+0000","last_clean":"2024-09-10T10:44:00.507045+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507045+0000","last_undegraded":"2024-09-10T10:44:00.507045+0000","last_fullsized":"2024-09-10T10:44:00.507045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507455+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:00.507455+0000","last_peered":"2024-09-10T10:44:00.507455+0000","last_clean":"2024-09-10T10:44:00.507455+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:00.507455+0000","last_undegraded":"2024-09-10T10:44:00.507455+0000","last_fullsized":"2024-09-10T10:44:00.507455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:19.649716+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:44:19.649716+0000","last_peered":"2024-09-10T10:44:19.649716+0000","last_clean":"2024-09-10T10:44:19.649716+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:44:19.649716+0000","last_undegraded":"2024-09-10T10:44:19.649716+0000","last_fullsized":"2024-09-10T10:44:19.649716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509606+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509606+0000","last_peered":"2024-09-10T10:44:00.509606+0000","last_clean":"2024-09-10T10:44:00.509606+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509606+0000","last_undegraded":"2024-09-10T10:44:00.509606+0000","last_fullsized":"2024-09-10T10:44:00.509606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506829+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:00.506829+0000","last_peered":"2024-09-10T10:44:00.506829+0000","last_clean":"2024-09-10T10:44:00.506829+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:00.506829+0000","last_undegraded":"2024-09-10T10:44:00.506829+0000","last_fullsized":"2024-09-10T10:44:00.506829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516333+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516333+0000","last_peered":"2024-09-10T10:44:00.516333+0000","last_clean":"2024-09-10T10:44:00.516333+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516333+0000","last_undegraded":"2024-09-10T10:44:00.516333+0000","last_fullsized":"2024-09-10T10:44:00.516333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500600+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500600+0000","last_peered":"2024-09-10T10:44:00.500600+0000","last_clean":"2024-09-10T10:44:00.500600+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500600+0000","last_undegraded":"2024-09-10T10:44:00.500600+0000","last_fullsized":"2024-09-10T10:44:00.500600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506810+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:00.506810+0000","last_peered":"2024-09-10T10:44:00.506810+0000","last_clean":"2024-09-10T10:44:00.506810+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:00.506810+0000","last_undegraded":"2024-09-10T10:44:00.506810+0000","last_fullsized":"2024-09-10T10:44:00.506810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502965+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.502965+0000","last_peered":"2024-09-10T10:44:00.502965+0000","last_clean":"2024-09-10T10:44:00.502965+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.502965+0000","last_undegraded":"2024-09-10T10:44:00.502965+0000","last_fullsized":"2024-09-10T10:44:00.502965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515028+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515028+0000","last_peered":"2024-09-10T10:44:00.515028+0000","last_clean":"2024-09-10T10:44:00.515028+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515028+0000","last_undegraded":"2024-09-10T10:44:00.515028+0000","last_fullsized":"2024-09-10T10:44:00.515028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503368+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:00.503368+0000","last_peered":"2024-09-10T10:44:00.503368+0000","last_clean":"2024-09-10T10:44:00.503368+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:00.503368+0000","last_undegraded":"2024-09-10T10:44:00.503368+0000","last_fullsized":"2024-09-10T10:44:00.503368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506407+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:44:00.506407+0000","last_peered":"2024-09-10T10:44:00.506407+0000","last_clean":"2024-09-10T10:44:00.506407+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:44:00.506407+0000","last_undegraded":"2024-09-10T10:44:00.506407+0000","last_fullsized":"2024-09-10T10:44:00.506407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515245+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515245+0000","last_peered":"2024-09-10T10:44:00.515245+0000","last_clean":"2024-09-10T10:44:00.515245+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515245+0000","last_undegraded":"2024-09-10T10:44:00.515245+0000","last_fullsized":"2024-09-10T10:44:00.515245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515403+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:00.515403+0000","last_peered":"2024-09-10T10:44:00.515403+0000","last_clean":"2024-09-10T10:44:00.515403+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:00.515403+0000","last_undegraded":"2024-09-10T10:44:00.515403+0000","last_fullsized":"2024-09-10T10:44:00.515403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505315+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505315+0000","last_peered":"2024-09-10T10:44:00.505315+0000","last_clean":"2024-09-10T10:44:00.505315+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505315+0000","last_undegraded":"2024-09-10T10:44:00.505315+0000","last_fullsized":"2024-09-10T10:44:00.505315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506639+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.506639+0000","last_peered":"2024-09-10T10:44:00.506639+0000","last_clean":"2024-09-10T10:44:00.506639+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.506639+0000","last_undegraded":"2024-09-10T10:44:00.506639+0000","last_fullsized":"2024-09-10T10:44:00.506639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505838+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:00.505838+0000","last_peered":"2024-09-10T10:44:00.505838+0000","last_clean":"2024-09-10T10:44:00.505838+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:00.505838+0000","last_undegraded":"2024-09-10T10:44:00.505838+0000","last_fullsized":"2024-09-10T10:44:00.505838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:21.628209+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:44:21.628209+0000","last_peered":"2024-09-10T10:44:21.628209+0000","last_clean":"2024-09-10T10:44:21.628209+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:44:21.628209+0000","last_undegraded":"2024-09-10T10:44:21.628209+0000","last_fullsized":"2024-09-10T10:44:21.628209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500337+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500337+0000","last_peered":"2024-09-10T10:44:00.500337+0000","last_clean":"2024-09-10T10:44:00.500337+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500337+0000","last_undegraded":"2024-09-10T10:44:00.500337+0000","last_fullsized":"2024-09-10T10:44:00.500337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509410+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:00.509410+0000","last_peered":"2024-09-10T10:44:00.509410+0000","last_clean":"2024-09-10T10:44:00.509410+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:00.509410+0000","last_undegraded":"2024-09-10T10:44:00.509410+0000","last_fullsized":"2024-09-10T10:44:00.509410+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515450+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515450+0000","last_peered":"2024-09-10T10:44:00.515450+0000","last_clean":"2024-09-10T10:44:00.515450+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515450+0000","last_undegraded":"2024-09-10T10:44:00.515450+0000","last_fullsized":"2024-09-10T10:44:00.515450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502859+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502859+0000","last_peered":"2024-09-10T10:44:00.502859+0000","last_clean":"2024-09-10T10:44:00.502859+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502859+0000","last_undegraded":"2024-09-10T10:44:00.502859+0000","last_fullsized":"2024-09-10T10:44:00.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502657+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:00.502657+0000","last_peered":"2024-09-10T10:44:00.502657+0000","last_clean":"2024-09-10T10:44:00.502657+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:00.502657+0000","last_undegraded":"2024-09-10T10:44:00.502657+0000","last_fullsized":"2024-09-10T10:44:00.502657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516076+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516076+0000","last_peered":"2024-09-10T10:44:00.516076+0000","last_clean":"2024-09-10T10:44:00.516076+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516076+0000","last_undegraded":"2024-09-10T10:44:00.516076+0000","last_fullsized":"2024-09-10T10:44:00.516076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506151+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:00.506151+0000","last_peered":"2024-09-10T10:44:00.506151+0000","last_clean":"2024-09-10T10:44:00.506151+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:00.506151+0000","last_undegraded":"2024-09-10T10:44:00.506151+0000","last_fullsized":"2024-09-10T10:44:00.506151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'34","reported_seq":195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:21.544799+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:44:21.544799+0000","last_peered":"2024-09-10T10:44:21.544799+0000","last_clean":"2024-09-10T10:44:21.544799+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:44:21.544799+0000","last_undegraded":"2024-09-10T10:44:21.544799+0000","last_fullsized":"2024-09-10T10:44:21.544799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":111,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500085+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500085+0000","last_peered":"2024-09-10T10:44:00.500085+0000","last_clean":"2024-09-10T10:44:00.500085+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500085+0000","last_undegraded":"2024-09-10T10:44:00.500085+0000","last_fullsized":"2024-09-10T10:44:00.500085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509983+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:00.509983+0000","last_peered":"2024-09-10T10:44:00.509983+0000","last_clean":"2024-09-10T10:44:00.509983+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:00.509983+0000","last_undegraded":"2024-09-10T10:44:00.509983+0000","last_fullsized":"2024-09-10T10:44:00.509983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516260+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516260+0000","last_peered":"2024-09-10T10:44:00.516260+0000","last_clean":"2024-09-10T10:44:00.516260+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516260+0000","last_undegraded":"2024-09-10T10:44:00.516260+0000","last_fullsized":"2024-09-10T10:44:00.516260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503778+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503778+0000","last_peered":"2024-09-10T10:44:00.503778+0000","last_clean":"2024-09-10T10:44:00.503778+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503778+0000","last_undegraded":"2024-09-10T10:44:00.503778+0000","last_fullsized":"2024-09-10T10:44:00.503778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501331+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:00.501331+0000","last_peered":"2024-09-10T10:44:00.501331+0000","last_clean":"2024-09-10T10:44:00.501331+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:00.501331+0000","last_undegraded":"2024-09-10T10:44:00.501331+0000","last_fullsized":"2024-09-10T10:44:00.501331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506602+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506602+0000","last_peered":"2024-09-10T10:44:00.506602+0000","last_clean":"2024-09-10T10:44:00.506602+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506602+0000","last_undegraded":"2024-09-10T10:44:00.506602+0000","last_fullsized":"2024-09-10T10:44:00.506602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499856+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499856+0000","last_peered":"2024-09-10T10:44:00.499856+0000","last_clean":"2024-09-10T10:44:00.499856+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499856+0000","last_undegraded":"2024-09-10T10:44:00.499856+0000","last_fullsized":"2024-09-10T10:44:00.499856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510665+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:00.510665+0000","last_peered":"2024-09-10T10:44:00.510665+0000","last_clean":"2024-09-10T10:44:00.510665+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:00.510665+0000","last_undegraded":"2024-09-10T10:44:00.510665+0000","last_fullsized":"2024-09-10T10:44:00.510665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:20.631722+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:44:20.631722+0000","last_peered":"2024-09-10T10:44:20.631722+0000","last_clean":"2024-09-10T10:44:20.631722+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:44:20.631722+0000","last_undegraded":"2024-09-10T10:44:20.631722+0000","last_fullsized":"2024-09-10T10:44:20.631722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504723+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504723+0000","last_peered":"2024-09-10T10:44:00.504723+0000","last_clean":"2024-09-10T10:44:00.504723+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504723+0000","last_undegraded":"2024-09-10T10:44:00.504723+0000","last_fullsized":"2024-09-10T10:44:00.504723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500883+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500883+0000","last_peered":"2024-09-10T10:44:00.500883+0000","last_clean":"2024-09-10T10:44:00.500883+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500883+0000","last_undegraded":"2024-09-10T10:44:00.500883+0000","last_fullsized":"2024-09-10T10:44:00.500883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500063+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:00.500063+0000","last_peered":"2024-09-10T10:44:00.500063+0000","last_clean":"2024-09-10T10:44:00.500063+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:00.500063+0000","last_undegraded":"2024-09-10T10:44:00.500063+0000","last_fullsized":"2024-09-10T10:44:00.500063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515537+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515537+0000","last_peered":"2024-09-10T10:44:00.515537+0000","last_clean":"2024-09-10T10:44:00.515537+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515537+0000","last_undegraded":"2024-09-10T10:44:00.515537+0000","last_fullsized":"2024-09-10T10:44:00.515537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509118+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509118+0000","last_peered":"2024-09-10T10:44:00.509118+0000","last_clean":"2024-09-10T10:44:00.509118+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509118+0000","last_undegraded":"2024-09-10T10:44:00.509118+0000","last_fullsized":"2024-09-10T10:44:00.509118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504540+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:00.504540+0000","last_peered":"2024-09-10T10:44:00.504540+0000","last_clean":"2024-09-10T10:44:00.504540+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:00.504540+0000","last_undegraded":"2024-09-10T10:44:00.504540+0000","last_fullsized":"2024-09-10T10:44:00.504540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502463+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502463+0000","last_peered":"2024-09-10T10:44:00.502463+0000","last_clean":"2024-09-10T10:44:00.502463+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502463+0000","last_undegraded":"2024-09-10T10:44:00.502463+0000","last_fullsized":"2024-09-10T10:44:00.502463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505325+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505325+0000","last_peered":"2024-09-10T10:44:00.505325+0000","last_clean":"2024-09-10T10:44:00.505325+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505325+0000","last_undegraded":"2024-09-10T10:44:00.505325+0000","last_fullsized":"2024-09-10T10:44:00.505325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515781+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:00.515781+0000","last_peered":"2024-09-10T10:44:00.515781+0000","last_clean":"2024-09-10T10:44:00.515781+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:00.515781+0000","last_undegraded":"2024-09-10T10:44:00.515781+0000","last_fullsized":"2024-09-10T10:44:00.515781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:20.632303+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:44:20.632303+0000","last_peered":"2024-09-10T10:44:20.632303+0000","last_clean":"2024-09-10T10:44:20.632303+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:44:20.632303+0000","last_undegraded":"2024-09-10T10:44:20.632303+0000","last_fullsized":"2024-09-10T10:44:20.632303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505450+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:00.505450+0000","last_peered":"2024-09-10T10:44:00.505450+0000","last_clean":"2024-09-10T10:44:00.505450+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:00.505450+0000","last_undegraded":"2024-09-10T10:44:00.505450+0000","last_fullsized":"2024-09-10T10:44:00.505450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502492+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:00.502492+0000","last_peered":"2024-09-10T10:44:00.502492+0000","last_clean":"2024-09-10T10:44:00.502492+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:00.502492+0000","last_undegraded":"2024-09-10T10:44:00.502492+0000","last_fullsized":"2024-09-10T10:44:00.502492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501098+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:00.501098+0000","last_peered":"2024-09-10T10:44:00.501098+0000","last_clean":"2024-09-10T10:44:00.501098+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:00.501098+0000","last_undegraded":"2024-09-10T10:44:00.501098+0000","last_fullsized":"2024-09-10T10:44:00.501098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500890+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:00.500890+0000","last_peered":"2024-09-10T10:44:00.500890+0000","last_clean":"2024-09-10T10:44:00.500890+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:00.500890+0000","last_undegraded":"2024-09-10T10:44:00.500890+0000","last_fullsized":"2024-09-10T10:44:00.500890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500656+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:00.500656+0000","last_peered":"2024-09-10T10:44:00.500656+0000","last_clean":"2024-09-10T10:44:00.500656+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:00.500656+0000","last_undegraded":"2024-09-10T10:44:00.500656+0000","last_fullsized":"2024-09-10T10:44:00.500656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502676+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:00.502676+0000","last_peered":"2024-09-10T10:44:00.502676+0000","last_clean":"2024-09-10T10:44:00.502676+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:00.502676+0000","last_undegraded":"2024-09-10T10:44:00.502676+0000","last_fullsized":"2024-09-10T10:44:00.502676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503284+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:00.503284+0000","last_peered":"2024-09-10T10:44:00.503284+0000","last_clean":"2024-09-10T10:44:00.503284+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:00.503284+0000","last_undegraded":"2024-09-10T10:44:00.503284+0000","last_fullsized":"2024-09-10T10:44:00.503284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501548+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:00.501548+0000","last_peered":"2024-09-10T10:44:00.501548+0000","last_clean":"2024-09-10T10:44:00.501548+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:00.501548+0000","last_undegraded":"2024-09-10T10:44:00.501548+0000","last_fullsized":"2024-09-10T10:44:00.501548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504578+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:00.504578+0000","last_peered":"2024-09-10T10:44:00.504578+0000","last_clean":"2024-09-10T10:44:00.504578+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:00.504578+0000","last_undegraded":"2024-09-10T10:44:00.504578+0000","last_fullsized":"2024-09-10T10:44:00.504578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515247+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:00.515247+0000","last_peered":"2024-09-10T10:44:00.515247+0000","last_clean":"2024-09-10T10:44:00.515247+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:00.515247+0000","last_undegraded":"2024-09-10T10:44:00.515247+0000","last_fullsized":"2024-09-10T10:44:00.515247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504827+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:00.504827+0000","last_peered":"2024-09-10T10:44:00.504827+0000","last_clean":"2024-09-10T10:44:00.504827+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:00.504827+0000","last_undegraded":"2024-09-10T10:44:00.504827+0000","last_fullsized":"2024-09-10T10:44:00.504827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509843+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:00.509843+0000","last_peered":"2024-09-10T10:44:00.509843+0000","last_clean":"2024-09-10T10:44:00.509843+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:00.509843+0000","last_undegraded":"2024-09-10T10:44:00.509843+0000","last_fullsized":"2024-09-10T10:44:00.509843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503053+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:00.503053+0000","last_peered":"2024-09-10T10:44:00.503053+0000","last_clean":"2024-09-10T10:44:00.503053+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:00.503053+0000","last_undegraded":"2024-09-10T10:44:00.503053+0000","last_fullsized":"2024-09-10T10:44:00.503053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502974+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:00.502974+0000","last_peered":"2024-09-10T10:44:00.502974+0000","last_clean":"2024-09-10T10:44:00.502974+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:00.502974+0000","last_undegraded":"2024-09-10T10:44:00.502974+0000","last_fullsized":"2024-09-10T10:44:00.502974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501471+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:00.501471+0000","last_peered":"2024-09-10T10:44:00.501471+0000","last_clean":"2024-09-10T10:44:00.501471+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:00.501471+0000","last_undegraded":"2024-09-10T10:44:00.501471+0000","last_fullsized":"2024-09-10T10:44:00.501471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500678+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:00.500678+0000","last_peered":"2024-09-10T10:44:00.500678+0000","last_clean":"2024-09-10T10:44:00.500678+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:00.500678+0000","last_undegraded":"2024-09-10T10:44:00.500678+0000","last_fullsized":"2024-09-10T10:44:00.500678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504016+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:00.504016+0000","last_peered":"2024-09-10T10:44:00.504016+0000","last_clean":"2024-09-10T10:44:00.504016+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:00.504016+0000","last_undegraded":"2024-09-10T10:44:00.504016+0000","last_fullsized":"2024-09-10T10:44:00.504016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509857+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:00.509857+0000","last_peered":"2024-09-10T10:44:00.509857+0000","last_clean":"2024-09-10T10:44:00.509857+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:00.509857+0000","last_undegraded":"2024-09-10T10:44:00.509857+0000","last_fullsized":"2024-09-10T10:44:00.509857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500445+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:00.500445+0000","last_peered":"2024-09-10T10:44:00.500445+0000","last_clean":"2024-09-10T10:44:00.500445+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:00.500445+0000","last_undegraded":"2024-09-10T10:44:00.500445+0000","last_fullsized":"2024-09-10T10:44:00.500445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500374+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:00.500374+0000","last_peered":"2024-09-10T10:44:00.500374+0000","last_clean":"2024-09-10T10:44:00.500374+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:00.500374+0000","last_undegraded":"2024-09-10T10:44:00.500374+0000","last_fullsized":"2024-09-10T10:44:00.500374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502968+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:00.502968+0000","last_peered":"2024-09-10T10:44:00.502968+0000","last_clean":"2024-09-10T10:44:00.502968+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:00.502968+0000","last_undegraded":"2024-09-10T10:44:00.502968+0000","last_fullsized":"2024-09-10T10:44:00.502968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.508081+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:00.508081+0000","last_peered":"2024-09-10T10:44:00.508081+0000","last_clean":"2024-09-10T10:44:00.508081+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:00.508081+0000","last_undegraded":"2024-09-10T10:44:00.508081+0000","last_fullsized":"2024-09-10T10:44:00.508081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507227+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:00.507227+0000","last_peered":"2024-09-10T10:44:00.507227+0000","last_clean":"2024-09-10T10:44:00.507227+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:00.507227+0000","last_undegraded":"2024-09-10T10:44:00.507227+0000","last_fullsized":"2024-09-10T10:44:00.507227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506065+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:00.506065+0000","last_peered":"2024-09-10T10:44:00.506065+0000","last_clean":"2024-09-10T10:44:00.506065+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:00.506065+0000","last_undegraded":"2024-09-10T10:44:00.506065+0000","last_fullsized":"2024-09-10T10:44:00.506065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510905+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:00.510905+0000","last_peered":"2024-09-10T10:44:00.510905+0000","last_clean":"2024-09-10T10:44:00.510905+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:00.510905+0000","last_undegraded":"2024-09-10T10:44:00.510905+0000","last_fullsized":"2024-09-10T10:44:00.510905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504451+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:00.504451+0000","last_peered":"2024-09-10T10:44:00.504451+0000","last_clean":"2024-09-10T10:44:00.504451+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:00.504451+0000","last_undegraded":"2024-09-10T10:44:00.504451+0000","last_fullsized":"2024-09-10T10:44:00.504451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506004+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:00.506004+0000","last_peered":"2024-09-10T10:44:00.506004+0000","last_clean":"2024-09-10T10:44:00.506004+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:00.506004+0000","last_undegraded":"2024-09-10T10:44:00.506004+0000","last_fullsized":"2024-09-10T10:44:00.506004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505316+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:00.505316+0000","last_peered":"2024-09-10T10:44:00.505316+0000","last_clean":"2024-09-10T10:44:00.505316+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:00.505316+0000","last_undegraded":"2024-09-10T10:44:00.505316+0000","last_fullsized":"2024-09-10T10:44:00.505316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505598+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:00.505598+0000","last_peered":"2024-09-10T10:44:00.505598+0000","last_clean":"2024-09-10T10:44:00.505598+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:00.505598+0000","last_undegraded":"2024-09-10T10:44:00.505598+0000","last_fullsized":"2024-09-10T10:44:00.505598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504402+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:00.504402+0000","last_peered":"2024-09-10T10:44:00.504402+0000","last_clean":"2024-09-10T10:44:00.504402+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:00.504402+0000","last_undegraded":"2024-09-10T10:44:00.504402+0000","last_fullsized":"2024-09-10T10:44:00.504402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500788+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:00.500788+0000","last_peered":"2024-09-10T10:44:00.500788+0000","last_clean":"2024-09-10T10:44:00.500788+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:00.500788+0000","last_undegraded":"2024-09-10T10:44:00.500788+0000","last_fullsized":"2024-09-10T10:44:00.500788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500884+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:00.500884+0000","last_peered":"2024-09-10T10:44:00.500884+0000","last_clean":"2024-09-10T10:44:00.500884+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:00.500884+0000","last_undegraded":"2024-09-10T10:44:00.500884+0000","last_fullsized":"2024-09-10T10:44:00.500884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503942+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:00.503942+0000","last_peered":"2024-09-10T10:44:00.503942+0000","last_clean":"2024-09-10T10:44:00.503942+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:00.503942+0000","last_undegraded":"2024-09-10T10:44:00.503942+0000","last_fullsized":"2024-09-10T10:44:00.503942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69284,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":526,"num_read_kb":340,"num_write":248,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":500498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1712128,"data_stored":1153418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939690,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168623,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1634304,"data_stored":1096844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":495471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1642496,"data_stored":1092033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1093632,"data_stored":562014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1024000,"data_stored":497046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1081344,"data_stored":558577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":497320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:27.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:44:27.654 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:27.775 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:27 smithi143 ceph-mon[31057]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:27.775 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:27 smithi143 ceph-mon[31057]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:27 smithi178 ceph-mon[34813]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:27 smithi178 ceph-mon[34813]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:27 smithi144 ceph-mon[34230]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:27 smithi144 ceph-mon[34230]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:28.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.121+0000 7fd8307a1640 1 -- 172.21.15.143:0/1973781766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828100e20 msgr2=0x7fd828101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.121+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1973781766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828100e20 0x7fd828101200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd824009900 tx=0x7fd82402a480 comp rx=0 tx=0).stop 2024-09-10T10:44:28.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- 172.21.15.143:0/1973781766 shutdown_connections 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1973781766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828102ef0 0x7fd82810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1973781766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd828102570 0x7fd8281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1973781766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828100e20 0x7fd828101200 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- 172.21.15.143:0/1973781766 >> 172.21.15.143:0/1973781766 conn(0x7fd8280fc7d0 msgr2=0x7fd8280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- 172.21.15.143:0/1973781766 shutdown_connections 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- 172.21.15.143:0/1973781766 wait complete. 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 Processor -- start 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- start start 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828100e20 0x7fd8281972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 0x7fd8281977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd828102ef0 0x7fd82819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd828110e30 con 0x7fd828102570 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd828110cb0 con 0x7fd828102ef0 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.122+0000 7fd8307a1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd828110fb0 con 0x7fd828100e20 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 0x7fd8281977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 0x7fd8281977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45090/0 (socket says 172.21.15.143:45090) 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 -- 172.21.15.143:0/1947836072 learned_addr learned my addr 172.21.15.143:0/1947836072 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82ed3e640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd828102ef0 0x7fd82819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82e53d640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828100e20 0x7fd8281972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 -- 172.21.15.143:0/1947836072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828100e20 msgr2=0x7fd8281972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828100e20 0x7fd8281972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 -- 172.21.15.143:0/1947836072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd828102ef0 msgr2=0x7fd82819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd828102ef0 0x7fd82819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 -- 172.21.15.143:0/1947836072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82819c260 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82e53d640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828100e20 0x7fd8281972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd82dd3c640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 0x7fd8281977f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd81800e9f0 tx=0x7fd81800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81800cd80 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82819c550 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8180108e0 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818018920 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.123+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8281a3e30 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.124+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8281021b0 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.125+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd818010a90 con 0x7fd828102570 2024-09-10T10:44:28.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.126+0000 7fd81f7fe640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd8140a2c10 0x7fd8140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.126+0000 7fd82e53d640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd8140a2c10 0x7fd8140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.126+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd8180cf8b0 con 0x7fd828102570 2024-09-10T10:44:28.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.127+0000 7fd82e53d640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd8140a2c10 0x7fd8140a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd824005ab0 tx=0x7fd824005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:28.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.131+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd81809c350 con 0x7fd828102570 2024-09-10T10:44:28.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.296+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd828062630 con 0x7fd8140a2c10 2024-09-10T10:44:28.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.304+0000 7fd81f7fe640 1 -- 172.21.15.143:0/1947836072 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340432 (secure 0 0 0) 0x7fd828062630 con 0x7fd8140a2c10 2024-09-10T10:44:28.309 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:28.310 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd8140a2c10 msgr2=0x7fd8140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd8140a2c10 0x7fd8140a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd824005ab0 tx=0x7fd824005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 msgr2=0x7fd8281977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 0x7fd8281977f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd81800e9f0 tx=0x7fd81800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 shutdown_connections 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd8140a2c10 0x7fd8140a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd828102ef0 0x7fd82819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828102570 0x7fd8281977f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 --2- 172.21.15.143:0/1947836072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd828100e20 0x7fd8281972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 >> 172.21.15.143:0/1947836072 conn(0x7fd8280fc7d0 msgr2=0x7fd8280fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 shutdown_connections 2024-09-10T10:44:28.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:28.307+0000 7fd8307a1640 1 -- 172.21.15.143:0/1947836072 wait complete. 2024-09-10T10:44:28.805 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:28 smithi143 ceph-mon[31057]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:28 smithi178 ceph-mon[34813]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:28.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:28 smithi144 ceph-mon[34230]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:29.103 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-10T10:44:27.442170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659886,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":692,"num_read_kb":469,"num_write":431,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375852,"kb_used_data":14036,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565972,"statfs":{"total":1151940427776,"available":1151555555328,"internally_reserved":0,"allocated":14372864,"data_stored":7948712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003199"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:24.642966+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:44:24.642966+0000","last_peered":"2024-09-10T10:44:24.642966+0000","last_clean":"2024-09-10T10:44:24.642966+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:44:24.642966+0000","last_undegraded":"2024-09-10T10:44:24.642966+0000","last_fullsized":"2024-09-10T10:44:24.642966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504790+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:00.504790+0000","last_peered":"2024-09-10T10:44:00.504790+0000","last_clean":"2024-09-10T10:44:00.504790+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:00.504790+0000","last_undegraded":"2024-09-10T10:44:00.504790+0000","last_fullsized":"2024-09-10T10:44:00.504790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500745+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500745+0000","last_peered":"2024-09-10T10:44:00.500745+0000","last_clean":"2024-09-10T10:44:00.500745+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500745+0000","last_undegraded":"2024-09-10T10:44:00.500745+0000","last_fullsized":"2024-09-10T10:44:00.500745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505003+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505003+0000","last_peered":"2024-09-10T10:44:00.505003+0000","last_clean":"2024-09-10T10:44:00.505003+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505003+0000","last_undegraded":"2024-09-10T10:44:00.505003+0000","last_fullsized":"2024-09-10T10:44:00.505003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501309+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:00.501309+0000","last_peered":"2024-09-10T10:44:00.501309+0000","last_clean":"2024-09-10T10:44:00.501309+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:00.501309+0000","last_undegraded":"2024-09-10T10:44:00.501309+0000","last_fullsized":"2024-09-10T10:44:00.501309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503391+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503391+0000","last_peered":"2024-09-10T10:44:00.503391+0000","last_clean":"2024-09-10T10:44:00.503391+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503391+0000","last_undegraded":"2024-09-10T10:44:00.503391+0000","last_fullsized":"2024-09-10T10:44:00.503391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506237+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:44:00.506237+0000","last_peered":"2024-09-10T10:44:00.506237+0000","last_clean":"2024-09-10T10:44:00.506237+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:44:00.506237+0000","last_undegraded":"2024-09-10T10:44:00.506237+0000","last_fullsized":"2024-09-10T10:44:00.506237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509720+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:00.509720+0000","last_peered":"2024-09-10T10:44:00.509720+0000","last_clean":"2024-09-10T10:44:00.509720+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:00.509720+0000","last_undegraded":"2024-09-10T10:44:00.509720+0000","last_fullsized":"2024-09-10T10:44:00.509720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505929+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505929+0000","last_peered":"2024-09-10T10:44:00.505929+0000","last_clean":"2024-09-10T10:44:00.505929+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505929+0000","last_undegraded":"2024-09-10T10:44:00.505929+0000","last_fullsized":"2024-09-10T10:44:00.505929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516100+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:00.516100+0000","last_peered":"2024-09-10T10:44:00.516100+0000","last_clean":"2024-09-10T10:44:00.516100+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:00.516100+0000","last_undegraded":"2024-09-10T10:44:00.516100+0000","last_fullsized":"2024-09-10T10:44:00.516100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506910+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:00.506910+0000","last_peered":"2024-09-10T10:44:00.506910+0000","last_clean":"2024-09-10T10:44:00.506910+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:00.506910+0000","last_undegraded":"2024-09-10T10:44:00.506910+0000","last_fullsized":"2024-09-10T10:44:00.506910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503914+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503914+0000","last_peered":"2024-09-10T10:44:00.503914+0000","last_clean":"2024-09-10T10:44:00.503914+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503914+0000","last_undegraded":"2024-09-10T10:44:00.503914+0000","last_fullsized":"2024-09-10T10:44:00.503914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505145+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:00.505145+0000","last_peered":"2024-09-10T10:44:00.505145+0000","last_clean":"2024-09-10T10:44:00.505145+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:00.505145+0000","last_undegraded":"2024-09-10T10:44:00.505145+0000","last_fullsized":"2024-09-10T10:44:00.505145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500867+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:44:00.500867+0000","last_peered":"2024-09-10T10:44:00.500867+0000","last_clean":"2024-09-10T10:44:00.500867+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:44:00.500867+0000","last_undegraded":"2024-09-10T10:44:00.500867+0000","last_fullsized":"2024-09-10T10:44:00.500867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.523722+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.523722+0000","last_peered":"2024-09-10T10:44:00.523722+0000","last_clean":"2024-09-10T10:44:00.523722+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.523722+0000","last_undegraded":"2024-09-10T10:44:00.523722+0000","last_fullsized":"2024-09-10T10:44:00.523722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502524+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:00.502524+0000","last_peered":"2024-09-10T10:44:00.502524+0000","last_clean":"2024-09-10T10:44:00.502524+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:00.502524+0000","last_undegraded":"2024-09-10T10:44:00.502524+0000","last_fullsized":"2024-09-10T10:44:00.502524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504899+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504899+0000","last_peered":"2024-09-10T10:44:00.504899+0000","last_clean":"2024-09-10T10:44:00.504899+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504899+0000","last_undegraded":"2024-09-10T10:44:00.504899+0000","last_fullsized":"2024-09-10T10:44:00.504899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500299+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500299+0000","last_peered":"2024-09-10T10:44:00.500299+0000","last_clean":"2024-09-10T10:44:00.500299+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500299+0000","last_undegraded":"2024-09-10T10:44:00.500299+0000","last_fullsized":"2024-09-10T10:44:00.500299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505392+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:00.505392+0000","last_peered":"2024-09-10T10:44:00.505392+0000","last_clean":"2024-09-10T10:44:00.505392+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:00.505392+0000","last_undegraded":"2024-09-10T10:44:00.505392+0000","last_fullsized":"2024-09-10T10:44:00.505392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510007+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510007+0000","last_peered":"2024-09-10T10:44:00.510007+0000","last_clean":"2024-09-10T10:44:00.510007+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510007+0000","last_undegraded":"2024-09-10T10:44:00.510007+0000","last_fullsized":"2024-09-10T10:44:00.510007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504053+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504053+0000","last_peered":"2024-09-10T10:44:00.504053+0000","last_clean":"2024-09-10T10:44:00.504053+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504053+0000","last_undegraded":"2024-09-10T10:44:00.504053+0000","last_fullsized":"2024-09-10T10:44:00.504053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500989+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:00.500989+0000","last_peered":"2024-09-10T10:44:00.500989+0000","last_clean":"2024-09-10T10:44:00.500989+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:00.500989+0000","last_undegraded":"2024-09-10T10:44:00.500989+0000","last_fullsized":"2024-09-10T10:44:00.500989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504293+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504293+0000","last_peered":"2024-09-10T10:44:00.504293+0000","last_clean":"2024-09-10T10:44:00.504293+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504293+0000","last_undegraded":"2024-09-10T10:44:00.504293+0000","last_fullsized":"2024-09-10T10:44:00.504293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500883+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500883+0000","last_peered":"2024-09-10T10:44:00.500883+0000","last_clean":"2024-09-10T10:44:00.500883+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500883+0000","last_undegraded":"2024-09-10T10:44:00.500883+0000","last_fullsized":"2024-09-10T10:44:00.500883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504671+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:00.504671+0000","last_peered":"2024-09-10T10:44:00.504671+0000","last_clean":"2024-09-10T10:44:00.504671+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:00.504671+0000","last_undegraded":"2024-09-10T10:44:00.504671+0000","last_fullsized":"2024-09-10T10:44:00.504671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500843+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:44:00.500843+0000","last_peered":"2024-09-10T10:44:00.500843+0000","last_clean":"2024-09-10T10:44:00.500843+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:44:00.500843+0000","last_undegraded":"2024-09-10T10:44:00.500843+0000","last_fullsized":"2024-09-10T10:44:00.500843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504281+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504281+0000","last_peered":"2024-09-10T10:44:00.504281+0000","last_clean":"2024-09-10T10:44:00.504281+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504281+0000","last_undegraded":"2024-09-10T10:44:00.504281+0000","last_fullsized":"2024-09-10T10:44:00.504281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502773+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:00.502773+0000","last_peered":"2024-09-10T10:44:00.502773+0000","last_clean":"2024-09-10T10:44:00.502773+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:00.502773+0000","last_undegraded":"2024-09-10T10:44:00.502773+0000","last_fullsized":"2024-09-10T10:44:00.502773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506283+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506283+0000","last_peered":"2024-09-10T10:44:00.506283+0000","last_clean":"2024-09-10T10:44:00.506283+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506283+0000","last_undegraded":"2024-09-10T10:44:00.506283+0000","last_fullsized":"2024-09-10T10:44:00.506283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503163+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503163+0000","last_peered":"2024-09-10T10:44:00.503163+0000","last_clean":"2024-09-10T10:44:00.503163+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503163+0000","last_undegraded":"2024-09-10T10:44:00.503163+0000","last_fullsized":"2024-09-10T10:44:00.503163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505442+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:00.505442+0000","last_peered":"2024-09-10T10:44:00.505442+0000","last_clean":"2024-09-10T10:44:00.505442+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:00.505442+0000","last_undegraded":"2024-09-10T10:44:00.505442+0000","last_fullsized":"2024-09-10T10:44:00.505442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501327+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501327+0000","last_peered":"2024-09-10T10:44:00.501327+0000","last_clean":"2024-09-10T10:44:00.501327+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501327+0000","last_undegraded":"2024-09-10T10:44:00.501327+0000","last_fullsized":"2024-09-10T10:44:00.501327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505290+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505290+0000","last_peered":"2024-09-10T10:44:00.505290+0000","last_clean":"2024-09-10T10:44:00.505290+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505290+0000","last_undegraded":"2024-09-10T10:44:00.505290+0000","last_fullsized":"2024-09-10T10:44:00.505290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505242+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:00.505242+0000","last_peered":"2024-09-10T10:44:00.505242+0000","last_clean":"2024-09-10T10:44:00.505242+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:00.505242+0000","last_undegraded":"2024-09-10T10:44:00.505242+0000","last_fullsized":"2024-09-10T10:44:00.505242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:23.642236+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:44:23.642236+0000","last_peered":"2024-09-10T10:44:23.642236+0000","last_clean":"2024-09-10T10:44:23.642236+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:44:23.642236+0000","last_undegraded":"2024-09-10T10:44:23.642236+0000","last_fullsized":"2024-09-10T10:44:23.642236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502584+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502584+0000","last_peered":"2024-09-10T10:44:00.502584+0000","last_clean":"2024-09-10T10:44:00.502584+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502584+0000","last_undegraded":"2024-09-10T10:44:00.502584+0000","last_fullsized":"2024-09-10T10:44:00.502584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515343+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:00.515343+0000","last_peered":"2024-09-10T10:44:00.515343+0000","last_clean":"2024-09-10T10:44:00.515343+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:00.515343+0000","last_undegraded":"2024-09-10T10:44:00.515343+0000","last_fullsized":"2024-09-10T10:44:00.515343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.629112+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:44:22.629112+0000","last_peered":"2024-09-10T10:44:22.629112+0000","last_clean":"2024-09-10T10:44:22.629112+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:44:22.629112+0000","last_undegraded":"2024-09-10T10:44:22.629112+0000","last_fullsized":"2024-09-10T10:44:22.629112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502695+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502695+0000","last_peered":"2024-09-10T10:44:00.502695+0000","last_clean":"2024-09-10T10:44:00.502695+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502695+0000","last_undegraded":"2024-09-10T10:44:00.502695+0000","last_fullsized":"2024-09-10T10:44:00.502695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502417+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:00.502417+0000","last_peered":"2024-09-10T10:44:00.502417+0000","last_clean":"2024-09-10T10:44:00.502417+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:00.502417+0000","last_undegraded":"2024-09-10T10:44:00.502417+0000","last_fullsized":"2024-09-10T10:44:00.502417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506043+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506043+0000","last_peered":"2024-09-10T10:44:00.506043+0000","last_clean":"2024-09-10T10:44:00.506043+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506043+0000","last_undegraded":"2024-09-10T10:44:00.506043+0000","last_fullsized":"2024-09-10T10:44:00.506043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515952+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.515952+0000","last_peered":"2024-09-10T10:44:00.515952+0000","last_clean":"2024-09-10T10:44:00.515952+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.515952+0000","last_undegraded":"2024-09-10T10:44:00.515952+0000","last_fullsized":"2024-09-10T10:44:00.515952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516146+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:00.516146+0000","last_peered":"2024-09-10T10:44:00.516146+0000","last_clean":"2024-09-10T10:44:00.516146+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:00.516146+0000","last_undegraded":"2024-09-10T10:44:00.516146+0000","last_fullsized":"2024-09-10T10:44:00.516146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503897+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503897+0000","last_peered":"2024-09-10T10:44:00.503897+0000","last_clean":"2024-09-10T10:44:00.503897+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503897+0000","last_undegraded":"2024-09-10T10:44:00.503897+0000","last_fullsized":"2024-09-10T10:44:00.503897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507045+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507045+0000","last_peered":"2024-09-10T10:44:00.507045+0000","last_clean":"2024-09-10T10:44:00.507045+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507045+0000","last_undegraded":"2024-09-10T10:44:00.507045+0000","last_fullsized":"2024-09-10T10:44:00.507045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507455+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:00.507455+0000","last_peered":"2024-09-10T10:44:00.507455+0000","last_clean":"2024-09-10T10:44:00.507455+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:00.507455+0000","last_undegraded":"2024-09-10T10:44:00.507455+0000","last_fullsized":"2024-09-10T10:44:00.507455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:24.657433+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:44:24.657433+0000","last_peered":"2024-09-10T10:44:24.657433+0000","last_clean":"2024-09-10T10:44:24.657433+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:44:24.657433+0000","last_undegraded":"2024-09-10T10:44:24.657433+0000","last_fullsized":"2024-09-10T10:44:24.657433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509606+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509606+0000","last_peered":"2024-09-10T10:44:00.509606+0000","last_clean":"2024-09-10T10:44:00.509606+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509606+0000","last_undegraded":"2024-09-10T10:44:00.509606+0000","last_fullsized":"2024-09-10T10:44:00.509606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506829+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:00.506829+0000","last_peered":"2024-09-10T10:44:00.506829+0000","last_clean":"2024-09-10T10:44:00.506829+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:00.506829+0000","last_undegraded":"2024-09-10T10:44:00.506829+0000","last_fullsized":"2024-09-10T10:44:00.506829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516333+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516333+0000","last_peered":"2024-09-10T10:44:00.516333+0000","last_clean":"2024-09-10T10:44:00.516333+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516333+0000","last_undegraded":"2024-09-10T10:44:00.516333+0000","last_fullsized":"2024-09-10T10:44:00.516333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500600+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500600+0000","last_peered":"2024-09-10T10:44:00.500600+0000","last_clean":"2024-09-10T10:44:00.500600+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500600+0000","last_undegraded":"2024-09-10T10:44:00.500600+0000","last_fullsized":"2024-09-10T10:44:00.500600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506810+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:00.506810+0000","last_peered":"2024-09-10T10:44:00.506810+0000","last_clean":"2024-09-10T10:44:00.506810+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:00.506810+0000","last_undegraded":"2024-09-10T10:44:00.506810+0000","last_fullsized":"2024-09-10T10:44:00.506810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502965+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.502965+0000","last_peered":"2024-09-10T10:44:00.502965+0000","last_clean":"2024-09-10T10:44:00.502965+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.502965+0000","last_undegraded":"2024-09-10T10:44:00.502965+0000","last_fullsized":"2024-09-10T10:44:00.502965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515028+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515028+0000","last_peered":"2024-09-10T10:44:00.515028+0000","last_clean":"2024-09-10T10:44:00.515028+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515028+0000","last_undegraded":"2024-09-10T10:44:00.515028+0000","last_fullsized":"2024-09-10T10:44:00.515028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503368+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:00.503368+0000","last_peered":"2024-09-10T10:44:00.503368+0000","last_clean":"2024-09-10T10:44:00.503368+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:00.503368+0000","last_undegraded":"2024-09-10T10:44:00.503368+0000","last_fullsized":"2024-09-10T10:44:00.503368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506407+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:44:00.506407+0000","last_peered":"2024-09-10T10:44:00.506407+0000","last_clean":"2024-09-10T10:44:00.506407+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:44:00.506407+0000","last_undegraded":"2024-09-10T10:44:00.506407+0000","last_fullsized":"2024-09-10T10:44:00.506407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515245+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515245+0000","last_peered":"2024-09-10T10:44:00.515245+0000","last_clean":"2024-09-10T10:44:00.515245+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515245+0000","last_undegraded":"2024-09-10T10:44:00.515245+0000","last_fullsized":"2024-09-10T10:44:00.515245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515403+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:00.515403+0000","last_peered":"2024-09-10T10:44:00.515403+0000","last_clean":"2024-09-10T10:44:00.515403+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:00.515403+0000","last_undegraded":"2024-09-10T10:44:00.515403+0000","last_fullsized":"2024-09-10T10:44:00.515403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505315+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505315+0000","last_peered":"2024-09-10T10:44:00.505315+0000","last_clean":"2024-09-10T10:44:00.505315+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505315+0000","last_undegraded":"2024-09-10T10:44:00.505315+0000","last_fullsized":"2024-09-10T10:44:00.505315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506639+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.506639+0000","last_peered":"2024-09-10T10:44:00.506639+0000","last_clean":"2024-09-10T10:44:00.506639+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.506639+0000","last_undegraded":"2024-09-10T10:44:00.506639+0000","last_fullsized":"2024-09-10T10:44:00.506639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505838+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:00.505838+0000","last_peered":"2024-09-10T10:44:00.505838+0000","last_clean":"2024-09-10T10:44:00.505838+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:00.505838+0000","last_undegraded":"2024-09-10T10:44:00.505838+0000","last_fullsized":"2024-09-10T10:44:00.505838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":94,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:21.628209+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:44:21.628209+0000","last_peered":"2024-09-10T10:44:21.628209+0000","last_clean":"2024-09-10T10:44:21.628209+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:44:21.628209+0000","last_undegraded":"2024-09-10T10:44:21.628209+0000","last_fullsized":"2024-09-10T10:44:21.628209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500337+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500337+0000","last_peered":"2024-09-10T10:44:00.500337+0000","last_clean":"2024-09-10T10:44:00.500337+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500337+0000","last_undegraded":"2024-09-10T10:44:00.500337+0000","last_fullsized":"2024-09-10T10:44:00.500337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509410+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:00.509410+0000","last_peered":"2024-09-10T10:44:00.509410+0000","last_clean":"2024-09-10T10:44:00.509410+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:00.509410+0000","last_undegraded":"2024-09-10T10:44:00.509410+0000","last_fullsized":"2024-09-10T10:44:00.509410+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515450+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515450+0000","last_peered":"2024-09-10T10:44:00.515450+0000","last_clean":"2024-09-10T10:44:00.515450+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515450+0000","last_undegraded":"2024-09-10T10:44:00.515450+0000","last_fullsized":"2024-09-10T10:44:00.515450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502859+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502859+0000","last_peered":"2024-09-10T10:44:00.502859+0000","last_clean":"2024-09-10T10:44:00.502859+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502859+0000","last_undegraded":"2024-09-10T10:44:00.502859+0000","last_fullsized":"2024-09-10T10:44:00.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502657+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:00.502657+0000","last_peered":"2024-09-10T10:44:00.502657+0000","last_clean":"2024-09-10T10:44:00.502657+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:00.502657+0000","last_undegraded":"2024-09-10T10:44:00.502657+0000","last_fullsized":"2024-09-10T10:44:00.502657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516076+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516076+0000","last_peered":"2024-09-10T10:44:00.516076+0000","last_clean":"2024-09-10T10:44:00.516076+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516076+0000","last_undegraded":"2024-09-10T10:44:00.516076+0000","last_fullsized":"2024-09-10T10:44:00.516076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506151+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:00.506151+0000","last_peered":"2024-09-10T10:44:00.506151+0000","last_clean":"2024-09-10T10:44:00.506151+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:00.506151+0000","last_undegraded":"2024-09-10T10:44:00.506151+0000","last_fullsized":"2024-09-10T10:44:00.506151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'34","reported_seq":205,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:26.552033+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:44:26.552033+0000","last_peered":"2024-09-10T10:44:26.552033+0000","last_clean":"2024-09-10T10:44:26.552033+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:44:26.552033+0000","last_undegraded":"2024-09-10T10:44:26.552033+0000","last_fullsized":"2024-09-10T10:44:26.552033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":116,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500085+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500085+0000","last_peered":"2024-09-10T10:44:00.500085+0000","last_clean":"2024-09-10T10:44:00.500085+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500085+0000","last_undegraded":"2024-09-10T10:44:00.500085+0000","last_fullsized":"2024-09-10T10:44:00.500085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509983+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:00.509983+0000","last_peered":"2024-09-10T10:44:00.509983+0000","last_clean":"2024-09-10T10:44:00.509983+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:00.509983+0000","last_undegraded":"2024-09-10T10:44:00.509983+0000","last_fullsized":"2024-09-10T10:44:00.509983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516260+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516260+0000","last_peered":"2024-09-10T10:44:00.516260+0000","last_clean":"2024-09-10T10:44:00.516260+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516260+0000","last_undegraded":"2024-09-10T10:44:00.516260+0000","last_fullsized":"2024-09-10T10:44:00.516260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503778+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503778+0000","last_peered":"2024-09-10T10:44:00.503778+0000","last_clean":"2024-09-10T10:44:00.503778+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503778+0000","last_undegraded":"2024-09-10T10:44:00.503778+0000","last_fullsized":"2024-09-10T10:44:00.503778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501331+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:00.501331+0000","last_peered":"2024-09-10T10:44:00.501331+0000","last_clean":"2024-09-10T10:44:00.501331+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:00.501331+0000","last_undegraded":"2024-09-10T10:44:00.501331+0000","last_fullsized":"2024-09-10T10:44:00.501331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506602+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506602+0000","last_peered":"2024-09-10T10:44:00.506602+0000","last_clean":"2024-09-10T10:44:00.506602+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506602+0000","last_undegraded":"2024-09-10T10:44:00.506602+0000","last_fullsized":"2024-09-10T10:44:00.506602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499856+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499856+0000","last_peered":"2024-09-10T10:44:00.499856+0000","last_clean":"2024-09-10T10:44:00.499856+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499856+0000","last_undegraded":"2024-09-10T10:44:00.499856+0000","last_fullsized":"2024-09-10T10:44:00.499856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510665+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:00.510665+0000","last_peered":"2024-09-10T10:44:00.510665+0000","last_clean":"2024-09-10T10:44:00.510665+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:00.510665+0000","last_undegraded":"2024-09-10T10:44:00.510665+0000","last_fullsized":"2024-09-10T10:44:00.510665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:25.638908+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:44:25.638908+0000","last_peered":"2024-09-10T10:44:25.638908+0000","last_clean":"2024-09-10T10:44:25.638908+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:44:25.638908+0000","last_undegraded":"2024-09-10T10:44:25.638908+0000","last_fullsized":"2024-09-10T10:44:25.638908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504723+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504723+0000","last_peered":"2024-09-10T10:44:00.504723+0000","last_clean":"2024-09-10T10:44:00.504723+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504723+0000","last_undegraded":"2024-09-10T10:44:00.504723+0000","last_fullsized":"2024-09-10T10:44:00.504723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500883+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500883+0000","last_peered":"2024-09-10T10:44:00.500883+0000","last_clean":"2024-09-10T10:44:00.500883+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500883+0000","last_undegraded":"2024-09-10T10:44:00.500883+0000","last_fullsized":"2024-09-10T10:44:00.500883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500063+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:00.500063+0000","last_peered":"2024-09-10T10:44:00.500063+0000","last_clean":"2024-09-10T10:44:00.500063+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:00.500063+0000","last_undegraded":"2024-09-10T10:44:00.500063+0000","last_fullsized":"2024-09-10T10:44:00.500063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515537+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515537+0000","last_peered":"2024-09-10T10:44:00.515537+0000","last_clean":"2024-09-10T10:44:00.515537+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515537+0000","last_undegraded":"2024-09-10T10:44:00.515537+0000","last_fullsized":"2024-09-10T10:44:00.515537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509118+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509118+0000","last_peered":"2024-09-10T10:44:00.509118+0000","last_clean":"2024-09-10T10:44:00.509118+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509118+0000","last_undegraded":"2024-09-10T10:44:00.509118+0000","last_fullsized":"2024-09-10T10:44:00.509118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504540+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:00.504540+0000","last_peered":"2024-09-10T10:44:00.504540+0000","last_clean":"2024-09-10T10:44:00.504540+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:00.504540+0000","last_undegraded":"2024-09-10T10:44:00.504540+0000","last_fullsized":"2024-09-10T10:44:00.504540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502463+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502463+0000","last_peered":"2024-09-10T10:44:00.502463+0000","last_clean":"2024-09-10T10:44:00.502463+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502463+0000","last_undegraded":"2024-09-10T10:44:00.502463+0000","last_fullsized":"2024-09-10T10:44:00.502463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505325+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505325+0000","last_peered":"2024-09-10T10:44:00.505325+0000","last_clean":"2024-09-10T10:44:00.505325+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505325+0000","last_undegraded":"2024-09-10T10:44:00.505325+0000","last_fullsized":"2024-09-10T10:44:00.505325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515781+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:00.515781+0000","last_peered":"2024-09-10T10:44:00.515781+0000","last_clean":"2024-09-10T10:44:00.515781+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:00.515781+0000","last_undegraded":"2024-09-10T10:44:00.515781+0000","last_fullsized":"2024-09-10T10:44:00.515781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:25.639493+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:44:25.639493+0000","last_peered":"2024-09-10T10:44:25.639493+0000","last_clean":"2024-09-10T10:44:25.639493+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:44:25.639493+0000","last_undegraded":"2024-09-10T10:44:25.639493+0000","last_fullsized":"2024-09-10T10:44:25.639493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505450+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:00.505450+0000","last_peered":"2024-09-10T10:44:00.505450+0000","last_clean":"2024-09-10T10:44:00.505450+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:00.505450+0000","last_undegraded":"2024-09-10T10:44:00.505450+0000","last_fullsized":"2024-09-10T10:44:00.505450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502492+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:00.502492+0000","last_peered":"2024-09-10T10:44:00.502492+0000","last_clean":"2024-09-10T10:44:00.502492+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:00.502492+0000","last_undegraded":"2024-09-10T10:44:00.502492+0000","last_fullsized":"2024-09-10T10:44:00.502492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501098+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:00.501098+0000","last_peered":"2024-09-10T10:44:00.501098+0000","last_clean":"2024-09-10T10:44:00.501098+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:00.501098+0000","last_undegraded":"2024-09-10T10:44:00.501098+0000","last_fullsized":"2024-09-10T10:44:00.501098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500890+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:00.500890+0000","last_peered":"2024-09-10T10:44:00.500890+0000","last_clean":"2024-09-10T10:44:00.500890+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:00.500890+0000","last_undegraded":"2024-09-10T10:44:00.500890+0000","last_fullsized":"2024-09-10T10:44:00.500890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500656+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:00.500656+0000","last_peered":"2024-09-10T10:44:00.500656+0000","last_clean":"2024-09-10T10:44:00.500656+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:00.500656+0000","last_undegraded":"2024-09-10T10:44:00.500656+0000","last_fullsized":"2024-09-10T10:44:00.500656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502676+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:00.502676+0000","last_peered":"2024-09-10T10:44:00.502676+0000","last_clean":"2024-09-10T10:44:00.502676+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:00.502676+0000","last_undegraded":"2024-09-10T10:44:00.502676+0000","last_fullsized":"2024-09-10T10:44:00.502676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503284+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:00.503284+0000","last_peered":"2024-09-10T10:44:00.503284+0000","last_clean":"2024-09-10T10:44:00.503284+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:00.503284+0000","last_undegraded":"2024-09-10T10:44:00.503284+0000","last_fullsized":"2024-09-10T10:44:00.503284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501548+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:00.501548+0000","last_peered":"2024-09-10T10:44:00.501548+0000","last_clean":"2024-09-10T10:44:00.501548+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:00.501548+0000","last_undegraded":"2024-09-10T10:44:00.501548+0000","last_fullsized":"2024-09-10T10:44:00.501548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504578+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:00.504578+0000","last_peered":"2024-09-10T10:44:00.504578+0000","last_clean":"2024-09-10T10:44:00.504578+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:00.504578+0000","last_undegraded":"2024-09-10T10:44:00.504578+0000","last_fullsized":"2024-09-10T10:44:00.504578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515247+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:00.515247+0000","last_peered":"2024-09-10T10:44:00.515247+0000","last_clean":"2024-09-10T10:44:00.515247+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:00.515247+0000","last_undegraded":"2024-09-10T10:44:00.515247+0000","last_fullsized":"2024-09-10T10:44:00.515247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504827+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:00.504827+0000","last_peered":"2024-09-10T10:44:00.504827+0000","last_clean":"2024-09-10T10:44:00.504827+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:00.504827+0000","last_undegraded":"2024-09-10T10:44:00.504827+0000","last_fullsized":"2024-09-10T10:44:00.504827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509843+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:00.509843+0000","last_peered":"2024-09-10T10:44:00.509843+0000","last_clean":"2024-09-10T10:44:00.509843+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:00.509843+0000","last_undegraded":"2024-09-10T10:44:00.509843+0000","last_fullsized":"2024-09-10T10:44:00.509843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503053+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:00.503053+0000","last_peered":"2024-09-10T10:44:00.503053+0000","last_clean":"2024-09-10T10:44:00.503053+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:00.503053+0000","last_undegraded":"2024-09-10T10:44:00.503053+0000","last_fullsized":"2024-09-10T10:44:00.503053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502974+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:00.502974+0000","last_peered":"2024-09-10T10:44:00.502974+0000","last_clean":"2024-09-10T10:44:00.502974+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:00.502974+0000","last_undegraded":"2024-09-10T10:44:00.502974+0000","last_fullsized":"2024-09-10T10:44:00.502974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501471+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:00.501471+0000","last_peered":"2024-09-10T10:44:00.501471+0000","last_clean":"2024-09-10T10:44:00.501471+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:00.501471+0000","last_undegraded":"2024-09-10T10:44:00.501471+0000","last_fullsized":"2024-09-10T10:44:00.501471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500678+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:00.500678+0000","last_peered":"2024-09-10T10:44:00.500678+0000","last_clean":"2024-09-10T10:44:00.500678+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:00.500678+0000","last_undegraded":"2024-09-10T10:44:00.500678+0000","last_fullsized":"2024-09-10T10:44:00.500678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504016+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:00.504016+0000","last_peered":"2024-09-10T10:44:00.504016+0000","last_clean":"2024-09-10T10:44:00.504016+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:00.504016+0000","last_undegraded":"2024-09-10T10:44:00.504016+0000","last_fullsized":"2024-09-10T10:44:00.504016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509857+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:00.509857+0000","last_peered":"2024-09-10T10:44:00.509857+0000","last_clean":"2024-09-10T10:44:00.509857+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:00.509857+0000","last_undegraded":"2024-09-10T10:44:00.509857+0000","last_fullsized":"2024-09-10T10:44:00.509857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500445+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:00.500445+0000","last_peered":"2024-09-10T10:44:00.500445+0000","last_clean":"2024-09-10T10:44:00.500445+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:00.500445+0000","last_undegraded":"2024-09-10T10:44:00.500445+0000","last_fullsized":"2024-09-10T10:44:00.500445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500374+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:00.500374+0000","last_peered":"2024-09-10T10:44:00.500374+0000","last_clean":"2024-09-10T10:44:00.500374+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:00.500374+0000","last_undegraded":"2024-09-10T10:44:00.500374+0000","last_fullsized":"2024-09-10T10:44:00.500374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502968+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:00.502968+0000","last_peered":"2024-09-10T10:44:00.502968+0000","last_clean":"2024-09-10T10:44:00.502968+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:00.502968+0000","last_undegraded":"2024-09-10T10:44:00.502968+0000","last_fullsized":"2024-09-10T10:44:00.502968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.508081+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:00.508081+0000","last_peered":"2024-09-10T10:44:00.508081+0000","last_clean":"2024-09-10T10:44:00.508081+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:00.508081+0000","last_undegraded":"2024-09-10T10:44:00.508081+0000","last_fullsized":"2024-09-10T10:44:00.508081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507227+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:00.507227+0000","last_peered":"2024-09-10T10:44:00.507227+0000","last_clean":"2024-09-10T10:44:00.507227+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:00.507227+0000","last_undegraded":"2024-09-10T10:44:00.507227+0000","last_fullsized":"2024-09-10T10:44:00.507227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506065+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:00.506065+0000","last_peered":"2024-09-10T10:44:00.506065+0000","last_clean":"2024-09-10T10:44:00.506065+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:00.506065+0000","last_undegraded":"2024-09-10T10:44:00.506065+0000","last_fullsized":"2024-09-10T10:44:00.506065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510905+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:00.510905+0000","last_peered":"2024-09-10T10:44:00.510905+0000","last_clean":"2024-09-10T10:44:00.510905+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:00.510905+0000","last_undegraded":"2024-09-10T10:44:00.510905+0000","last_fullsized":"2024-09-10T10:44:00.510905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504451+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:00.504451+0000","last_peered":"2024-09-10T10:44:00.504451+0000","last_clean":"2024-09-10T10:44:00.504451+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:00.504451+0000","last_undegraded":"2024-09-10T10:44:00.504451+0000","last_fullsized":"2024-09-10T10:44:00.504451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506004+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:00.506004+0000","last_peered":"2024-09-10T10:44:00.506004+0000","last_clean":"2024-09-10T10:44:00.506004+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:00.506004+0000","last_undegraded":"2024-09-10T10:44:00.506004+0000","last_fullsized":"2024-09-10T10:44:00.506004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505316+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:00.505316+0000","last_peered":"2024-09-10T10:44:00.505316+0000","last_clean":"2024-09-10T10:44:00.505316+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:00.505316+0000","last_undegraded":"2024-09-10T10:44:00.505316+0000","last_fullsized":"2024-09-10T10:44:00.505316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505598+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:00.505598+0000","last_peered":"2024-09-10T10:44:00.505598+0000","last_clean":"2024-09-10T10:44:00.505598+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:00.505598+0000","last_undegraded":"2024-09-10T10:44:00.505598+0000","last_fullsized":"2024-09-10T10:44:00.505598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504402+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:00.504402+0000","last_peered":"2024-09-10T10:44:00.504402+0000","last_clean":"2024-09-10T10:44:00.504402+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:00.504402+0000","last_undegraded":"2024-09-10T10:44:00.504402+0000","last_fullsized":"2024-09-10T10:44:00.504402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500788+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:00.500788+0000","last_peered":"2024-09-10T10:44:00.500788+0000","last_clean":"2024-09-10T10:44:00.500788+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:00.500788+0000","last_undegraded":"2024-09-10T10:44:00.500788+0000","last_fullsized":"2024-09-10T10:44:00.500788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500884+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:00.500884+0000","last_peered":"2024-09-10T10:44:00.500884+0000","last_clean":"2024-09-10T10:44:00.500884+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:00.500884+0000","last_undegraded":"2024-09-10T10:44:00.500884+0000","last_fullsized":"2024-09-10T10:44:00.500884+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503942+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:00.503942+0000","last_peered":"2024-09-10T10:44:00.503942+0000","last_clean":"2024-09-10T10:44:00.503942+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:00.503942+0000","last_undegraded":"2024-09-10T10:44:00.503942+0000","last_fullsized":"2024-09-10T10:44:00.503942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69284,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":566,"num_read_kb":360,"num_write":248,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547230,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":500498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1712128,"data_stored":1153418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939691,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1634304,"data_stored":1096844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":495471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1642496,"data_stored":1092033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1093632,"data_stored":562014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1024000,"data_stored":497046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1081344,"data_stored":558577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":497320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:29.109 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:44:29.109 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T10:44:29.109 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T10:44:29.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T10:44:29.365 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:29.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:29 smithi143 ceph-mon[31057]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 -- 172.21.15.143:0/4276377601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 msgr2=0x7f95281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 --2- 172.21.15.143:0/4276377601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f95281021c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9518009900 tx=0x7f951802a410 comp rx=0 tx=0).stop 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 -- 172.21.15.143:0/4276377601 shutdown_connections 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 --2- 172.21.15.143:0/4276377601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528106f10 0x7f952810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 --2- 172.21.15.143:0/4276377601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9528102790 0x7f95281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 --2- 172.21.15.143:0/4276377601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f95281021c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 -- 172.21.15.143:0/4276377601 >> 172.21.15.143:0/4276377601 conn(0x7f95280fd8c0 msgr2=0x7f95280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 -- 172.21.15.143:0/4276377601 shutdown_connections 2024-09-10T10:44:29.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.882+0000 7f952d9cd640 1 -- 172.21.15.143:0/4276377601 wait complete. 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 Processor -- start 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 -- start start 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f9528198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528102790 0x7f95281988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9528106f10 0x7f952819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111e20 con 0x7f9528101de0 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111ca0 con 0x7f9528102790 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f952d9cd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111fa0 con 0x7f9528106f10 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f9528198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f9528198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45112/0 (socket says 172.21.15.143:45112) 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 -- 172.21.15.143:0/2118247104 learned_addr learned my addr 172.21.15.143:0/2118247104 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f951ffff640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528102790 0x7f95281988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 -- 172.21.15.143:0/2118247104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9528106f10 msgr2=0x7f952819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9528106f10 0x7f952819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 -- 172.21.15.143:0/2118247104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528102790 msgr2=0x7f95281988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528102790 0x7f95281988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 -- 172.21.15.143:0/2118247104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952819d330 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f951ffff640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528102790 0x7f95281988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.883+0000 7f9526ffd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f9528198380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9518006010 tx=0x7f9518004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.884+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951803f030 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.884+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95180035f0 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.884+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952819d5c0 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.884+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518035070 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.884+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a4f00 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.885+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95281021c0 con 0x7f9528101de0 2024-09-10T10:44:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.886+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9518003b10 con 0x7f9528101de0 2024-09-10T10:44:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.887+0000 7f952c9cb640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95140a2c90 0x7f95140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.887+0000 7f951ffff640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95140a2c90 0x7f95140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.887+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95180f0340 con 0x7f9528101de0 2024-09-10T10:44:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.887+0000 7f951ffff640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95140a2c90 0x7f95140a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95280fdd50 tx=0x7f9510005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:29.890+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95180bcde0 con 0x7f9528101de0 2024-09-10T10:44:29.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:29 smithi178 ceph-mon[34813]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:29.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:29 smithi144 ceph-mon[34230]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:30.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.092+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f952810a0f0 con 0x7f9528101de0 2024-09-10T10:44:30.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.093+0000 7f952c9cb640 1 -- 172.21.15.143:0/2118247104 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f95180bc6e0 con 0x7f9528101de0 2024-09-10T10:44:30.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:30.094 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95140a2c90 msgr2=0x7f95140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95140a2c90 0x7f95140a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95280fdd50 tx=0x7f9510005d30 comp rx=0 tx=0).stop 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 msgr2=0x7f9528198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f9528198380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9518006010 tx=0x7f9518004270 comp rx=0 tx=0).stop 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 shutdown_connections 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95140a2c90 0x7f95140a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9528106f10 0x7f952819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528102790 0x7f95281988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 --2- 172.21.15.143:0/2118247104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9528101de0 0x7f9528198380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 >> 172.21.15.143:0/2118247104 conn(0x7f95280fd8c0 msgr2=0x7f95280fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 shutdown_connections 2024-09-10T10:44:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:30.095+0000 7f952d9cd640 1 -- 172.21.15.143:0/2118247104 wait complete. 2024-09-10T10:44:30.856 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:30 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2118247104' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:44:30.857 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T10:44:30.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:44:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:30 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2118247104' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:44:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:30 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2118247104' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:44:31.110 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2097210505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac10c170 msgr2=0x7fd7ac10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2097210505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac10c170 0x7fd7ac10e610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009900 tx=0x7fd7a402a480 comp rx=0 tx=0).stop 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2097210505 shutdown_connections 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2097210505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac10c170 0x7fd7ac10e610 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2097210505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac103660 0x7fd7ac10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2097210505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ac102d40 0x7fd7ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2097210505 >> 172.21.15.143:0/2097210505 conn(0x7fd7ac0fc7a0 msgr2=0x7fd7ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2097210505 shutdown_connections 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.629+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2097210505 wait complete. 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 Processor -- start 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 -- start start 2024-09-10T10:44:31.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ac102d40 0x7fd7ac198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 0x7fd7ac198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac10c170 0x7fd7ac19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac111de0 con 0x7fd7ac103660 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac111c60 con 0x7fd7ac102d40 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b2d0a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac111f60 con 0x7fd7ac10c170 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 0x7fd7ac198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7abfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 0x7fd7ac198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45126/0 (socket says 172.21.15.143:45126) 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7abfff640 1 -- 172.21.15.143:0/2051052681 learned_addr learned my addr 172.21.15.143:0/2051052681 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.630+0000 7fd7b0aa6640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ac102d40 0x7fd7ac198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7abfff640 1 -- 172.21.15.143:0/2051052681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac10c170 msgr2=0x7fd7ac19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7b12a7640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac10c170 0x7fd7ac19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7abfff640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac10c170 0x7fd7ac19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7abfff640 1 -- 172.21.15.143:0/2051052681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ac102d40 msgr2=0x7fd7ac198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7abfff640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ac102d40 0x7fd7ac198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7abfff640 1 -- 172.21.15.143:0/2051052681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac19d2b0 con 0x7fd7ac103660 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7b12a7640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac10c170 0x7fd7ac19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7abfff640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 0x7fd7ac198840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00ea00 tx=0x7fd79c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00cd80 con 0x7fd7ac103660 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd79c011920 con 0x7fd7ac103660 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac19d5a0 con 0x7fd7ac103660 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c01a9d0 con 0x7fd7ac103660 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.631+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac1a4e80 con 0x7fd7ac103660 2024-09-10T10:44:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.632+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac103120 con 0x7fd7ac103660 2024-09-10T10:44:31.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.633+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd79c011ad0 con 0x7fd7ac103660 2024-09-10T10:44:31.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.634+0000 7fd7a9ffb640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7980a2c10 0x7fd7980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.634+0000 7fd7b0aa6640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7980a2c10 0x7fd7980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.634+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd79c0cf960 con 0x7fd7ac103660 2024-09-10T10:44:31.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.635+0000 7fd7b0aa6640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7980a2c10 0x7fd7980a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7940058c0 tx=0x7fd79400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:31.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.638+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd79c016030 con 0x7fd7ac103660 2024-09-10T10:44:31.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:31 smithi143 ceph-mon[31057]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:31.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.843+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7ac02d7c0 con 0x7fd7ac103660 2024-09-10T10:44:31.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.843+0000 7fd7a9ffb640 1 -- 172.21.15.143:0/2051052681 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7fd79c09c400 con 0x7fd7ac103660 2024-09-10T10:44:31.845 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:31.845 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"btime":"2024-09-10T10:43:32:550770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:32.550735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:44:31.845 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:44:31.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7980a2c10 msgr2=0x7fd7980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7980a2c10 0x7fd7980a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7940058c0 tx=0x7fd79400a2d0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 msgr2=0x7fd7ac198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 0x7fd7ac198840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00ea00 tx=0x7fd79c00eec0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 shutdown_connections 2024-09-10T10:44:31.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7980a2c10 0x7fd7980a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7ac10c170 0x7fd7ac19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7ac103660 0x7fd7ac198840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 --2- 172.21.15.143:0/2051052681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7ac102d40 0x7fd7ac198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 >> 172.21.15.143:0/2051052681 conn(0x7fd7ac0fc7a0 msgr2=0x7fd7ac0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 shutdown_connections 2024-09-10T10:44:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:31.846+0000 7fd7b2d0a640 1 -- 172.21.15.143:0/2051052681 wait complete. 2024-09-10T10:44:31.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:31 smithi178 ceph-mon[34813]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:31 smithi144 ceph-mon[34230]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:32.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:44:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:32 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2051052681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:32 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2051052681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:32.961 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:32 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2051052681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 -- 172.21.15.143:0/4014263333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8106b80 msgr2=0x7f93c8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 --2- 172.21.15.143:0/4014263333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8106b80 0x7f93c8106f60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009ca0 tx=0x7f93b802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 -- 172.21.15.143:0/4014263333 shutdown_connections 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 --2- 172.21.15.143:0/4014263333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c8100aa0 0x7f93c810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 --2- 172.21.15.143:0/4014263333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c8100100 0x7f93c8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 --2- 172.21.15.143:0/4014263333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8106b80 0x7f93c8106f60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 -- 172.21.15.143:0/4014263333 >> 172.21.15.143:0/4014263333 conn(0x7f93c8077230 msgr2=0x7f93c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 -- 172.21.15.143:0/4014263333 shutdown_connections 2024-09-10T10:44:33.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 -- 172.21.15.143:0/4014263333 wait complete. 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 Processor -- start 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.322+0000 7f93d0129640 1 -- start start 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93d0129640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 0x7f93c81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93d0129640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c8100aa0 0x7f93c8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93d0129640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c8106b80 0x7f93c8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93d0129640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8111fc0 con 0x7f93c8100100 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93d0129640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8111e40 con 0x7f93c8100aa0 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93d0129640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8112140 con 0x7f93c8106b80 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 0x7f93c81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 0x7f93c81012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45144/0 (socket says 172.21.15.143:45144) 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 -- 172.21.15.143:0/3855837850 learned_addr learned my addr 172.21.15.143:0/3855837850 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93ce6c6640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c8106b80 0x7f93c8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cd6c4640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c8100aa0 0x7f93c8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 -- 172.21.15.143:0/3855837850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c8106b80 msgr2=0x7f93c8101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c8106b80 0x7f93c8101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 -- 172.21.15.143:0/3855837850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c8100aa0 msgr2=0x7f93c8101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c8100aa0 0x7f93c8101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 -- 172.21.15.143:0/3855837850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c81a6980 con 0x7f93c8100100 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.323+0000 7f93cdec5640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 0x7f93c81012e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006010 tx=0x7f93b8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.324+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8004040 con 0x7f93c8100100 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.324+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b8004690 con 0x7f93c8100100 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.324+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c81a6c10 con 0x7f93c8100100 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.324+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b802f4c0 con 0x7f93c8100100 2024-09-10T10:44:33.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.324+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c81a7150 con 0x7f93c8100100 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.325+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c8100560 con 0x7f93c8100100 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.326+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f93b80305c0 con 0x7f93c8100100 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.326+0000 7f93c6ffd640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93b00a2c10 0x7f93b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.327+0000 7f93cd6c4640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93b00a2c10 0x7f93b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.327+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f93b80f0c10 con 0x7f93c8100100 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.327+0000 7f93cd6c4640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93b00a2c10 0x7f93b00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93c8102d70 tx=0x7f93bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.331+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93b80bd7c0 con 0x7f93c8100100 2024-09-10T10:44:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.530+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93c810a0e0 con 0x7f93c8100100 2024-09-10T10:44:33.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.530+0000 7f93c6ffd640 1 -- 172.21.15.143:0/3855837850 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f93b80bd0c0 con 0x7f93c8100100 2024-09-10T10:44:33.532 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:33.532 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"btime":"2024-09-10T10:43:32:550770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:32.550735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:44:33.532 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:44:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93b00a2c10 msgr2=0x7f93b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93b00a2c10 0x7f93b00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93c8102d70 tx=0x7f93bc008040 comp rx=0 tx=0).stop 2024-09-10T10:44:33.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 msgr2=0x7f93c81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 0x7f93c81012e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006010 tx=0x7f93b8003ba0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 shutdown_connections 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f93b00a2c10 0x7f93b00a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93c8106b80 0x7f93c8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93c8100aa0 0x7f93c8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 --2- 172.21.15.143:0/3855837850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93c8100100 0x7f93c81012e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 >> 172.21.15.143:0/3855837850 conn(0x7f93c8077230 msgr2=0x7f93c8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 shutdown_connections 2024-09-10T10:44:33.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:33.533+0000 7f93d0129640 1 -- 172.21.15.143:0/3855837850 wait complete. 2024-09-10T10:44:33.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:33 smithi143 ceph-mon[31057]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T10:44:33.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:33 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3855837850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:33.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:33 smithi178 ceph-mon[34813]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T10:44:33.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:33 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3855837850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:33 smithi144 ceph-mon[34230]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T10:44:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:33 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3855837850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:34.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:44:34.505 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 -- 172.21.15.143:0/3380264016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c101de0 msgr2=0x7f539c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3380264016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c101de0 0x7f539c1021c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f538c009900 tx=0x7f538c02a410 comp rx=0 tx=0).stop 2024-09-10T10:44:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 -- 172.21.15.143:0/3380264016 shutdown_connections 2024-09-10T10:44:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3380264016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c106f10 0x7f539c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3380264016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f539c102790 0x7f539c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3380264016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c101de0 0x7f539c1021c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 -- 172.21.15.143:0/3380264016 >> 172.21.15.143:0/3380264016 conn(0x7f539c0fd8c0 msgr2=0x7f539c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 -- 172.21.15.143:0/3380264016 shutdown_connections 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 -- 172.21.15.143:0/3380264016 wait complete. 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 Processor -- start 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.988+0000 7f53a2f58640 1 -- start start 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101de0 0x7f539c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 0x7f539c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f539c106f10 0x7f539c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f539c111e20 con 0x7f539c102790 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f539c111ca0 con 0x7f539c101de0 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f539c111fa0 con 0x7f539c106f10 2024-09-10T10:44:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 0x7f539c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 0x7f539c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45152/0 (socket says 172.21.15.143:45152) 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 -- 172.21.15.143:0/3965743210 learned_addr learned my addr 172.21.15.143:0/3965743210 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a14f5640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f539c106f10 0x7f539c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a0cf4640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101de0 0x7f539c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 -- 172.21.15.143:0/3965743210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f539c106f10 msgr2=0x7f539c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f539c106f10 0x7f539c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 -- 172.21.15.143:0/3965743210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101de0 msgr2=0x7f539c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101de0 0x7f539c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 -- 172.21.15.143:0/3965743210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c19d310 con 0x7f539c102790 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a0cf4640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101de0 0x7f539c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f539bfff640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 0x7f539c1988a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9f0 tx=0x7f539000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539000cd80 con 0x7f539c102790 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53900054d0 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.989+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c19d600 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.990+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390010490 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.990+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1a4ee0 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.991+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c1021c0 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.992+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5390005020 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.992+0000 7f5399ffb640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f53840a2c90 0x7f53840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.993+0000 7f53a0cf4640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f53840a2c90 0x7f53840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.993+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f53900cf940 con 0x7f539c102790 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.993+0000 7f53a0cf4640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f53840a2c90 0x7f53840a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f538c031440 tx=0x7f538c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:34.997+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f539009c380 con 0x7f539c102790 2024-09-10T10:44:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.166+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f539c02d7c0 con 0x7f539c102790 2024-09-10T10:44:35.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.168+0000 7f5399ffb640 1 -- 172.21.15.143:0/3965743210 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7f539009bc80 con 0x7f539c102790 2024-09-10T10:44:35.169 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:35.169 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:35.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f53840a2c90 msgr2=0x7f53840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f53840a2c90 0x7f53840a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f538c031440 tx=0x7f538c005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:35.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 msgr2=0x7f539c1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 0x7f539c1988a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9f0 tx=0x7f539000edf0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 shutdown_connections 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f53840a2c90 0x7f53840a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f539c106f10 0x7f539c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f539c102790 0x7f539c1988a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 --2- 172.21.15.143:0/3965743210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101de0 0x7f539c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 >> 172.21.15.143:0/3965743210 conn(0x7f539c0fd8c0 msgr2=0x7f539c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 shutdown_connections 2024-09-10T10:44:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:35.171+0000 7f53a2f58640 1 -- 172.21.15.143:0/3965743210 wait complete. 2024-09-10T10:44:35.540 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:35 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3965743210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:35.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:35 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3965743210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:35.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:35 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3965743210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:36.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:44:36.355 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:36.476 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:36 smithi143 ceph-mon[31057]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-10T10:44:36.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:36 smithi178 ceph-mon[34813]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-10T10:44:36.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:36 smithi144 ceph-mon[34230]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-10T10:44:36.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 -- 172.21.15.143:0/3427879436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc101420 msgr2=0x7f02cc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 --2- 172.21.15.143:0/3427879436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc101420 0x7f02cc10d950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009cd0 tx=0x7f02c402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 -- 172.21.15.143:0/3427879436 shutdown_connections 2024-09-10T10:44:36.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 --2- 172.21.15.143:0/3427879436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc101420 0x7f02cc10d950 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 --2- 172.21.15.143:0/3427879436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc100a80 0x7f02cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 --2- 172.21.15.143:0/3427879436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc106a80 0x7f02cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 -- 172.21.15.143:0/3427879436 >> 172.21.15.143:0/3427879436 conn(0x7f02cc0fc7a0 msgr2=0x7f02cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 -- 172.21.15.143:0/3427879436 shutdown_connections 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.874+0000 7f02d0972640 1 -- 172.21.15.143:0/3427879436 wait complete. 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 Processor -- start 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 -- start start 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 0x7f02cc102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc101420 0x7f02cc102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc106a80 0x7f02cc1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc110d30 con 0x7f02cc100a80 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc110bb0 con 0x7f02cc101420 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02d0972640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc110eb0 con 0x7f02cc106a80 2024-09-10T10:44:36.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02ca59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 0x7f02cc102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02ca59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 0x7f02cc102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49746/0 (socket says 172.21.15.143:49746) 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02ca59c640 1 -- 172.21.15.143:0/227706862 learned_addr learned my addr 172.21.15.143:0/227706862 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02cad9d640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc106a80 0x7f02cc1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.875+0000 7f02c3fff640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc101420 0x7f02cc102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02ca59c640 1 -- 172.21.15.143:0/227706862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc106a80 msgr2=0x7f02cc1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02ca59c640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc106a80 0x7f02cc1030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02ca59c640 1 -- 172.21.15.143:0/227706862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc101420 msgr2=0x7f02cc102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02ca59c640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc101420 0x7f02cc102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02ca59c640 1 -- 172.21.15.143:0/227706862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc1a6810 con 0x7f02cc100a80 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02cad9d640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc106a80 0x7f02cc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02c3fff640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc101420 0x7f02cc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:36.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02ca59c640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 0x7f02cc102650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00d950 tx=0x7f02bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc017070 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02bc009c30 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00f620 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc1a6b00 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.876+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc1a7090 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.877+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02cc100ee0 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.878+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f02bc0109a0 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.879+0000 7f02c37fe640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f02b80a2c90 0x7f02b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.879+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f02bc0cf110 con 0x7f02cc100a80 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.879+0000 7f02c3fff640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f02b80a2c90 0x7f02b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.879+0000 7f02c3fff640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f02b80a2c90 0x7f02b80a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f02b40058c0 tx=0x7f02b4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:36.883+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02bc009de0 con 0x7f02cc100a80 2024-09-10T10:44:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.087+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02cc103b80 con 0x7f02cc100a80 2024-09-10T10:44:37.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.088+0000 7f02c37fe640 1 -- 172.21.15.143:0/227706862 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f02bc09bbb0 con 0x7f02cc100a80 2024-09-10T10:44:37.092 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:37.093 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"btime":"2024-09-10T10:43:32:550770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:32.550735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:44:37.093 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.090+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f02b80a2c90 msgr2=0x7f02b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.090+0000 7f02d0972640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f02b80a2c90 0x7f02b80a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f02b40058c0 tx=0x7f02b4005850 comp rx=0 tx=0).stop 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 msgr2=0x7f02cc102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 0x7f02cc102650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00d950 tx=0x7f02bc00de10 comp rx=0 tx=0).stop 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 shutdown_connections 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f02b80a2c90 0x7f02b80a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02cc106a80 0x7f02cc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc101420 0x7f02cc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 --2- 172.21.15.143:0/227706862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02cc100a80 0x7f02cc102650 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 >> 172.21.15.143:0/227706862 conn(0x7f02cc0fc7a0 msgr2=0x7f02cc1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 shutdown_connections 2024-09-10T10:44:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:37.091+0000 7f02d0972640 1 -- 172.21.15.143:0/227706862 wait complete. 2024-09-10T10:44:37.529 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:37 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/227706862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:37.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:37 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/227706862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:37.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:37 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/227706862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:37.867 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:42:53.956292+0000', 'modified': '2024-09-10T10:43:32.550735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24502, 'mds_1': 24485, 'mds_2': 24481, 'mds_3': 24475, 'mds_4': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6839/2518540467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2518540467}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2518540467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6841/1083221625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6840', 'nonce': 1083221625}, {'type': 'v1', 'addr': '172.21.15.143:6841', 'nonce': 1083221625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3121359191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3121359191}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3121359191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1016684638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1016684638}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1016684638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3190468681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3190468681}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3190468681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.144:6837/3174266316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 3174266316}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 3174266316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.178:6835/261851614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 261851614}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 261851614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.178:6837/2330516637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2330516637}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2330516637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.144:6839/1994626034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 1994626034}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 1994626034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.178:6839/774166191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 774166191}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 774166191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24502, 'qdb_cluster': [24502, 24485, 24481, 24475, 24458]} 2024-09-10T10:44:37.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T10:44:37.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T10:44:38.116 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:38.549 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:38 smithi143 ceph-mon[31057]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T10:44:38.576 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:38 smithi178 ceph-mon[34813]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T10:44:38.587 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:38 smithi144 ceph-mon[34230]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T10:44:38.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 -- 172.21.15.143:0/1885201446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780101de0 msgr2=0x7f37801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 --2- 172.21.15.143:0/1885201446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780101de0 0x7f37801021c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f377c009900 tx=0x7f377c02a410 comp rx=0 tx=0).stop 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 -- 172.21.15.143:0/1885201446 shutdown_connections 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 --2- 172.21.15.143:0/1885201446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3780106f10 0x7f378010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 --2- 172.21.15.143:0/1885201446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780102790 0x7f37801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 --2- 172.21.15.143:0/1885201446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780101de0 0x7f37801021c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 -- 172.21.15.143:0/1885201446 >> 172.21.15.143:0/1885201446 conn(0x7f37800fd8c0 msgr2=0x7f37800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 -- 172.21.15.143:0/1885201446 shutdown_connections 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 -- 172.21.15.143:0/1885201446 wait complete. 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 Processor -- start 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 -- start start 2024-09-10T10:44:38.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.629+0000 7f37878de640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3780101de0 0x7f37800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 0x7f3780074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780106f10 0x7f3780074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3780111e90 con 0x7f3780102790 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3780111d10 con 0x7f3780106f10 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3780112010 con 0x7f3780101de0 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 0x7f3780074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 0x7f3780074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49776/0 (socket says 172.21.15.143:49776) 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 -- 172.21.15.143:0/3689667418 learned_addr learned my addr 172.21.15.143:0/3689667418 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780106f10 0x7f3780074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f378567a640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3780101de0 0x7f37800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 -- 172.21.15.143:0/3689667418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3780101de0 msgr2=0x7f37800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3780101de0 0x7f37800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 -- 172.21.15.143:0/3689667418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780106f10 msgr2=0x7f3780074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780106f10 0x7f3780074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 -- 172.21.15.143:0/3689667418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37801a68f0 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780106f10 0x7f3780074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f3784e79640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 0x7f3780074a30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f377400c890 tx=0x7f377400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37727fc640 1 -- 172.21.15.143:0/3689667418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774004220 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37727fc640 1 -- 172.21.15.143:0/3689667418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37740043c0 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37801a6be0 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37727fc640 1 -- 172.21.15.143:0/3689667418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774004a00 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.630+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801a7120 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f37727fc640 1 -- 172.21.15.143:0/3689667418 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3774004ba0 con 0x7f3780102790 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 Processor -- start 2024-09-10T10:44:38.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 -- start start 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e57db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e62670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e64fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561685dbe090 con 0x561685e579d0 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561685d65a70 con 0x561685e62bb0 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3789cb7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561685dba050 con 0x561685e582f0 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3785e7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3785e7b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e57db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49782/0 (socket says 172.21.15.143:49782) 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.633+0000 7f3785e7b640 1 -- 172.21.15.143:0/3024814724 learned_addr learned my addr 172.21.15.143:0/3024814724 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f378567a640 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e62670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f37727fc640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f376c0a2c10 0x7f376c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3784e79640 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e64fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 -- 172.21.15.143:0/3024814724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 msgr2=0x561685e62670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e62670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 -- 172.21.15.143:0/3024814724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 msgr2=0x561685e64fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e64fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 -- 172.21.15.143:0/3024814724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561685d72130 con 0x561685e579d0 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3784e79640 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e64fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f376c0a2c10 0x7f376c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f37727fc640 1 -- 172.21.15.143:0/3689667418 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f37740cf250 con 0x7f3780102790 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e57db0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f377800c550 tx=0x7f377800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ced3f33c51ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f376b7fe640 1 -- 172.21.15.143:0/3024814724 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778018640 con 0x561685e579d0 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f376c0a2c10 0x7f376c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f377801f000 tx=0x7f3778018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f376b7fe640 1 -- 172.21.15.143:0/3024814724 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3778007e70 con 0x561685e579d0 2024-09-10T10:44:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.634+0000 7f376b7fe640 1 -- 172.21.15.143:0/3024814724 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800aa10 con 0x561685e579d0 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 -- 172.21.15.143:0/3024814724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 msgr2=0x561685e57db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e57db0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f377800c550 tx=0x7f377800ca10 comp rx=0 tx=0).stop 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 -- 172.21.15.143:0/3024814724 shutdown_connections 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e64fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e62670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3024814724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e57db0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 -- 172.21.15.143:0/3024814724 >> 172.21.15.143:0/3024814724 conn(0x561685e53f60 msgr2=0x561685e54340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 -- 172.21.15.143:0/3024814724 shutdown_connections 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 -- 172.21.15.143:0/3024814724 wait complete. 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.635+0000 7f3789cb7740 1 -- start start 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 Processor -- start 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e579d0 0x561685e624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e5f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561685e65780 con 0x561685e582f0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561685d78f20 con 0x561685e62bb0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561685e65900 con 0x561685e579d0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3789cb7740 10 client.0 _ll_register_callbacks cb 0x561685d42030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e5f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e5f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49788/0 (socket says 172.21.15.143:49788) 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 -- 172.21.15.143:0/3906408589 learned_addr learned my addr 172.21.15.143:0/3906408589 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f378567a640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e579d0 0x561685e624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 -- 172.21.15.143:0/3906408589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e579d0 msgr2=0x561685e624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e579d0 0x561685e624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 -- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 msgr2=0x561685e5fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e5fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3784e79640 1 -- 172.21.15.143:0/3906408589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561685b2f2b0 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f378567a640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e579d0 0x561685e624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.636+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e5fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3784e79640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e5f7f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37740979f0 tx=0x7f377409b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8fa2113f743fff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3768ff9640 1 -- 172.21.15.143:0/3906408589 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377401d070 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3778007e70 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3768ff9640 1 -- 172.21.15.143:0/3906408589 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3774004e90 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3768ff9640 1 -- 172.21.15.143:0/3906408589 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f377409bc90 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 10 client.15051 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561685e710f0 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3768ff9640 1 -- 172.21.15.143:0/3906408589 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37740b9080 con 0x561685e582f0 2024-09-10T10:44:38.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3768ff9640 1 -- 172.21.15.143:0/3906408589 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f37740cca90 con 0x561685e582f0 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 10 client.15051 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 10 client.15051 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 20 client.15051 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685e736f0 0x561685e74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 4 client.15051 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.637+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561685dcb600 con 0x561685e736f0 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.638+0000 7f378567a640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685e736f0 0x561685e74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.638+0000 7f378567a640 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685e736f0 0x561685e74b60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac775f11702a2da8 server_cookie=e8228b8ec76a02f0 in_seq=0 out_seq=0 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.638+0000 7f3768ff9640 10 client.15051 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3768ff9640 1 -- 172.21.15.143:0/3906408589 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561685dcb600 con 0x561685e736f0 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3768ff9640 10 client.15051 handle_command_reply: tid=0 2024-09-10T10:44:38.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 client.15051 shutdown 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 msgr2=0x561685e5f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e5f7f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37740979f0 tx=0x7f377409b0d0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 shutdown_connections 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685e736f0 0x561685e74b60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e5fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e5f7f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3906408589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e579d0 0x561685e624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3768ff9640 0 client.15051 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 >> 172.21.15.143:0/3906408589 conn(0x561685e53f60 msgr2=0x561685e59c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 shutdown_connections 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 1 -- 172.21.15.143:0/3906408589 wait complete. 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.639+0000 7f3789cb7740 20 client.15051 trim_cache size 0 max 16384 2024-09-10T10:44:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 Processor -- start 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 -- start start 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e73dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e4ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561685e72ea0 con 0x561685e579d0 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561685e73570 con 0x561685e62bb0 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3789cb7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561685e73c40 con 0x561685e582f0 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e73dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e73dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49796/0 (socket says 172.21.15.143:49796) 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 -- 172.21.15.143:0/488666087 learned_addr learned my addr 172.21.15.143:0/488666087 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3784e79640 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e4ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f3785e7b640 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 -- 172.21.15.143:0/488666087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 msgr2=0x561685e4ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e4ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 -- 172.21.15.143:0/488666087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 msgr2=0x561685e61650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e61650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 -- 172.21.15.143:0/488666087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561685e5e7c0 con 0x561685e579d0 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.657+0000 7f378567a640 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e73dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f377c009230 tx=0x7f377c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7ef0f34601b59a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f376affd640 1 -- 172.21.15.143:0/488666087 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c035040 con 0x561685e579d0 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f376affd640 1 -- 172.21.15.143:0/488666087 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f377c04a070 con 0x561685e579d0 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f376affd640 1 -- 172.21.15.143:0/488666087 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c003850 con 0x561685e579d0 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 -- 172.21.15.143:0/488666087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 msgr2=0x561685e73dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e73dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f377c009230 tx=0x7f377c005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 -- 172.21.15.143:0/488666087 shutdown_connections 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e62bb0 0x561685e61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e582f0 0x561685e4ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 --2- 172.21.15.143:0/488666087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e579d0 0x561685e73dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 -- 172.21.15.143:0/488666087 >> 172.21.15.143:0/488666087 conn(0x561685e53f60 msgr2=0x561685e60f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 -- 172.21.15.143:0/488666087 shutdown_connections 2024-09-10T10:44:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.658+0000 7f3789cb7740 1 -- 172.21.15.143:0/488666087 wait complete. 2024-09-10T10:44:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 -- start start 2024-09-10T10:44:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 Processor -- start 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e579d0 0x561685e76ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e77010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e62bb0 0x561685e77550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561685e749e0 con 0x561685e582f0 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561685e74310 con 0x561685e579d0 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561685e750b0 con 0x561685e62bb0 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3789cb7740 10 client.0 _ll_register_callbacks cb 0x7f3774004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3784e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e77010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3784e79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e77010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49800/0 (socket says 172.21.15.143:49800) 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3784e79640 1 -- 172.21.15.143:0/3392571494 learned_addr learned my addr 172.21.15.143:0/3392571494 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f378567a640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e579d0 0x561685e76ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.659+0000 7f3785e7b640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e62bb0 0x561685e77550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3784e79640 1 -- 172.21.15.143:0/3392571494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e62bb0 msgr2=0x561685e77550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3784e79640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e62bb0 0x561685e77550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3784e79640 1 -- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e579d0 msgr2=0x561685e76ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3784e79640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e579d0 0x561685e76ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3784e79640 1 -- 172.21.15.143:0/3392571494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561685e7b2c0 con 0x561685e582f0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f378567a640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e579d0 0x561685e76ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3784e79640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e77010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3774097f10 tx=0x7f377409afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74c4c16870151ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3768ff9640 1 -- 172.21.15.143:0/3392571494 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377401d070 con 0x561685e582f0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3768ff9640 1 -- 172.21.15.143:0/3392571494 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3774004e90 con 0x561685e582f0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377c04a070 con 0x561685e582f0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3768ff9640 1 -- 172.21.15.143:0/3392571494 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f377409b590 con 0x561685e582f0 2024-09-10T10:44:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3789cb7740 10 client.15057 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561685ede020 con 0x561685e582f0 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.660+0000 7f3768ff9640 1 -- 172.21.15.143:0/3392571494 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f377c04a070 con 0x561685e582f0 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3768ff9640 1 -- 172.21.15.143:0/3392571494 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f37740972e0 con 0x561685e582f0 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 10 client.15057 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 10 client.15057 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 20 client.15057 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685ee2c80 0x561685ee1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 4 client.15057 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "status"}) -- 0x561685dc5b10 con 0x561685ee2c80 2024-09-10T10:44:38.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f378567a640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685ee2c80 0x561685ee1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f378567a640 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685ee2c80 0x561685ee1990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70ea8e16aa845212 server_cookie=f142c3f8679898f1 in_seq=0 out_seq=0 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3768ff9640 10 client.15057 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3768ff9640 1 -- 172.21.15.143:0/3392571494 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561685dc5b10 con 0x561685ee2c80 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3768ff9640 10 client.15057 handle_command_reply: tid=0 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.661+0000 7f3789cb7740 1 client.15057 shutdown 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 msgr2=0x561685e77010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e77010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3774097f10 tx=0x7f377409afe0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 shutdown_connections 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561685ee2c80 0x561685ee1990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561685e62bb0 0x561685e77550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561685e582f0 0x561685e77010 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 --2- 172.21.15.143:0/3392571494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561685e579d0 0x561685e76ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3768ff9640 0 client.15057 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 >> 172.21.15.143:0/3392571494 conn(0x561685e53f60 msgr2=0x561685e53000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 shutdown_connections 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 1 -- 172.21.15.143:0/3392571494 wait complete. 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f3789cb7740 20 client.15057 trim_cache size 0 max 16384 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 4, 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24458, 2024-09-10T10:44:38.679 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 67.120112840000004, 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 127.15629617899999 2024-09-10T10:44:38.680 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f376c0a2c10 msgr2=0x7f376c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f37878de640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f376c0a2c10 0x7f376c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f377801f000 tx=0x7f3778018000 comp rx=0 tx=0).stop 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 msgr2=0x7f3780074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.662+0000 7f37878de640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 0x7f3780074a30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f377400c890 tx=0x7f377400cd50 comp rx=0 tx=0).stop 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 shutdown_connections 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f376c0a2c10 0x7f376c0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3780106f10 0x7f3780074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3780102790 0x7f3780074a30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 --2- 172.21.15.143:0/3689667418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3780101de0 0x7f37800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 >> 172.21.15.143:0/3689667418 conn(0x7f37800fd8c0 msgr2=0x7f37800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 shutdown_connections 2024-09-10T10:44:38.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:38.663+0000 7f37878de640 1 -- 172.21.15.143:0/3689667418 wait complete. 2024-09-10T10:44:39.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T10:44:39.839 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:40.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 -- 172.21.15.143:0/3425515615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011e0 msgr2=0x7f82c8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/3425515615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011e0 0x7f82c8105980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402a410 comp rx=0 tx=0).stop 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 -- 172.21.15.143:0/3425515615 shutdown_connections 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/3425515615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c8105ec0 0x7f82c8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/3425515615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81011e0 0x7f82c8105980 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/3425515615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81008c0 0x7f82c8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.323+0000 7f82cf7d1640 1 -- 172.21.15.143:0/3425515615 >> 172.21.15.143:0/3425515615 conn(0x7f82c80fc7d0 msgr2=0x7f82c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.324+0000 7f82cf7d1640 1 -- 172.21.15.143:0/3425515615 shutdown_connections 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.324+0000 7f82cf7d1640 1 -- 172.21.15.143:0/3425515615 wait complete. 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.324+0000 7f82cf7d1640 1 Processor -- start 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.324+0000 7f82cf7d1640 1 -- start start 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 0x7f82c81a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81011e0 0x7f82c81a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c8105ec0 0x7f82c819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f82c8118540 con 0x7f82c81008c0 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82c81183c0 con 0x7f82c8105ec0 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82c81186c0 con 0x7f82c81011e0 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 0x7f82c81a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 0x7f82c81a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49818/0 (socket says 172.21.15.143:49818) 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 -- 172.21.15.143:0/4104592480 learned_addr learned my addr 172.21.15.143:0/4104592480 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81011e0 0x7f82c81a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c8105ec0 0x7f82c819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 -- 172.21.15.143:0/4104592480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81011e0 msgr2=0x7f82c81a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81011e0 0x7f82c81a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 -- 172.21.15.143:0/4104592480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c8105ec0 msgr2=0x7f82c819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c8105ec0 0x7f82c819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 -- 172.21.15.143:0/4104592480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c819f060 con 0x7f82c81008c0 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81011e0 0x7f82c81a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cd56d640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 0x7f82c81a4770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b4a0 tx=0x7f82b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82be7fc640 1 -- 172.21.15.143:0/4104592480 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8014070 con 0x7f82c81008c0 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82be7fc640 1 -- 172.21.15.143:0/4104592480 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b8010ac0 con 0x7f82c81008c0 2024-09-10T10:44:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c819f350 con 0x7f82c81008c0 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82be7fc640 1 -- 172.21.15.143:0/4104592480 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8017c30 con 0x7f82c81008c0 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.325+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c81ab640 con 0x7f82c81008c0 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82be7fc640 1 -- 172.21.15.143:0/4104592480 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f82b8017070 con 0x7f82c81008c0 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 Processor -- start 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 -- start start 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad620db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad62b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad62dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5606ad587090 con 0x5606ad6212f0 2024-09-10T10:44:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5606ad52ea70 con 0x5606ad62bbb0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.328+0000 7f82d1baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606ad583050 con 0x5606ad6209d0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82be7fc640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f82b40a2c10 0x7f82b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad62b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad62b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49830/0 (socket says 172.21.15.143:49830) 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 -- 172.21.15.143:0/3649695039 learned_addr learned my addr 172.21.15.143:0/3649695039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f82b40a2c10 0x7f82b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad62dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad620db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82be7fc640 1 -- 172.21.15.143:0/4104592480 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f82b80ceba0 con 0x7f82c81008c0 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 -- 172.21.15.143:0/3649695039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 msgr2=0x5606ad620db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad620db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 -- 172.21.15.143:0/3649695039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 msgr2=0x5606ad62dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad62dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 -- 172.21.15.143:0/3649695039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ad53b130 con 0x5606ad6212f0 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cd56d640 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad62b670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cc240 tx=0x7f82b800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f499653b35570a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f82b40a2c10 0x7f82b40a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82c0005c40 tx=0x7f82c000ac90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82b37fe640 1 -- 172.21.15.143:0/3649695039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b801e070 con 0x5606ad6212f0 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82b37fe640 1 -- 172.21.15.143:0/3649695039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b809b050 con 0x5606ad6212f0 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.329+0000 7f82b37fe640 1 -- 172.21.15.143:0/3649695039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80cc440 con 0x5606ad6212f0 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 -- 172.21.15.143:0/3649695039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 msgr2=0x5606ad62b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad62b670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f82b80cc240 tx=0x7f82b800bf90 comp rx=0 tx=0).stop 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 -- 172.21.15.143:0/3649695039 shutdown_connections 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad62dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad62b670 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 --2- 172.21.15.143:0/3649695039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad620db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 -- 172.21.15.143:0/3649695039 >> 172.21.15.143:0/3649695039 conn(0x5606ad61cf60 msgr2=0x5606ad61d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 -- 172.21.15.143:0/3649695039 shutdown_connections 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.330+0000 7f82d1baa740 1 -- 172.21.15.143:0/3649695039 wait complete. 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 -- start start 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 Processor -- start 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad62b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad628c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5606ad62e780 con 0x5606ad6212f0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5606ad541f20 con 0x5606ad62bbb0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606ad62e900 con 0x5606ad6209d0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82d1baa740 10 client.0 _ll_register_callbacks cb 0x5606ad50b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6286f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49842/0 (socket says 172.21.15.143:49842) 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 -- 172.21.15.143:0/2421075809 learned_addr learned my addr 172.21.15.143:0/2421075809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad628c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82cd56d640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad62b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 -- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 msgr2=0x5606ad62b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad62b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 -- 172.21.15.143:0/2421075809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 msgr2=0x5606ad628c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.331+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad628c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82ccd6c640 1 -- 172.21.15.143:0/2421075809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ad2f82b0 con 0x5606ad6212f0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6286f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f82c402a930 tx=0x7f82c40317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db668510e1d5332 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2421075809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4035040 con 0x5606ad6212f0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2421075809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f82c4004240 con 0x5606ad6212f0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b809b050 con 0x5606ad6212f0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2421075809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82c4004440 con 0x5606ad6212f0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82d1baa740 10 client.15069 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606ad639ff0 con 0x5606ad6212f0 2024-09-10T10:44:40.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2421075809 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b809b050 con 0x5606ad6212f0 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2421075809 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f82c4045710 con 0x5606ad6212f0 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.332+0000 7f82d1baa740 10 client.15069 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82d1baa740 10 client.15069 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82d1baa740 20 client.15069 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad63c630 0x5606ad63daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82d1baa740 4 client.15069 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606ad594600 con 0x5606ad63c630 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82cd56d640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad63c630 0x5606ad63daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82b0ff9640 10 client.15069 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.333+0000 7f82cd56d640 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad63c630 0x5606ad63daa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b4dadd06467b67e server_cookie=bc8ca4b178f13e67 in_seq=0 out_seq=0 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2421075809 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606ad594600 con 0x5606ad63c630 2024-09-10T10:44:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82b0ff9640 10 client.15069 handle_command_reply: tid=0 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 client.15069 shutdown 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 msgr2=0x5606ad6286f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6286f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f82c402a930 tx=0x7f82c40317a0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 shutdown_connections 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad63c630 0x5606ad63daa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad628c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6286f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82b0ff9640 0 client.15069 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2421075809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad62b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 >> 172.21.15.143:0/2421075809 conn(0x5606ad61cf60 msgr2=0x5606ad622c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 shutdown_connections 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 1 -- 172.21.15.143:0/2421075809 wait complete. 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.334+0000 7f82d1baa740 20 client.15069 trim_cache size 0 max 16384 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 Processor -- start 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 -- start start 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad63daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad623b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5606ad63bde0 con 0x5606ad6212f0 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5606ad63c4b0 con 0x5606ad62bbb0 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82d1baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606ad63cb80 con 0x5606ad6209d0 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6235c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49848/0 (socket says 172.21.15.143:49848) 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 -- 172.21.15.143:0/1493842114 learned_addr learned my addr 172.21.15.143:0/1493842114 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82cd56d640 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad63daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad623b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 -- 172.21.15.143:0/1493842114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 msgr2=0x5606ad63daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad63daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 -- 172.21.15.143:0/1493842114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 msgr2=0x5606ad623b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad623b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82ccd6c640 1 -- 172.21.15.143:0/1493842114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ad61cd90 con 0x5606ad6212f0 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.352+0000 7f82cd56d640 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad63daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6235c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82c40059d0 tx=0x7f82c4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed2d1576a10fd1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1493842114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4058030 con 0x5606ad6212f0 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1493842114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82c40318d0 con 0x5606ad6212f0 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82b2ffd640 1 -- 172.21.15.143:0/1493842114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4035050 con 0x5606ad6212f0 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 -- 172.21.15.143:0/1493842114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 msgr2=0x5606ad6235c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6235c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82c40059d0 tx=0x7f82c4005e00 comp rx=0 tx=0).stop 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 -- 172.21.15.143:0/1493842114 shutdown_connections 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad62bbb0 0x5606ad623b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad6212f0 0x5606ad6235c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 --2- 172.21.15.143:0/1493842114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6209d0 0x5606ad63daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 -- 172.21.15.143:0/1493842114 >> 172.21.15.143:0/1493842114 conn(0x5606ad61cf60 msgr2=0x5606ad626fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 -- 172.21.15.143:0/1493842114 shutdown_connections 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.353+0000 7f82d1baa740 1 -- 172.21.15.143:0/1493842114 wait complete. 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 -- start start 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 Processor -- start 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad6209d0 0x5606ad646320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6212f0 0x5606ad642fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 0x5606ad6434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5606ad63d920 con 0x5606ad62bbb0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5606ad63d250 con 0x5606ad6209d0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606ad63dff0 con 0x5606ad6212f0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82d1baa740 10 client.0 _ll_register_callbacks cb 0x7f82c4031e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82cdd6e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 0x5606ad6434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82cdd6e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 0x5606ad6434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49864/0 (socket says 172.21.15.143:49864) 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.354+0000 7f82cdd6e640 1 -- 172.21.15.143:0/2794791786 learned_addr learned my addr 172.21.15.143:0/2794791786 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6212f0 0x5606ad642fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cd56d640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad6209d0 0x5606ad646320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cdd6e640 1 -- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6212f0 msgr2=0x5606ad642fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6212f0 0x5606ad642fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cdd6e640 1 -- 172.21.15.143:0/2794791786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad6209d0 msgr2=0x5606ad646320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad6209d0 0x5606ad646320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cdd6e640 1 -- 172.21.15.143:0/2794791786 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ad643ae0 con 0x5606ad62bbb0 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82ccd6c640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6212f0 0x5606ad642fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82cdd6e640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 0x5606ad6434f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82c0010ff0 tx=0x7f82c0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b7261ee776b43e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2794791786 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0003b70 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2794791786 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f82c0003d10 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c40318d0 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2794791786 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82c0004020 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82d1baa740 10 client.15075 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606ad62d900 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.355+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2794791786 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82c40318d0 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2794791786 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f82c00119c0 con 0x5606ad62bbb0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 10 client.15075 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 10 client.15075 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 20 client.15075 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad6ab760 0x5606ad6aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 4 client.15075 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "status"}) -- 0x5606ad58eb10 con 0x5606ad6ab760 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82cd56d640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad6ab760 0x5606ad6aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82cd56d640 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad6ab760 0x5606ad6aa470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37c843001a2d1a84 server_cookie=ea5ad64cb9eeda23 in_seq=0 out_seq=0 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82b0ff9640 10 client.15075 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82b0ff9640 1 -- 172.21.15.143:0/2794791786 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5606ad58eb10 con 0x5606ad6ab760 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82b0ff9640 10 client.15075 handle_command_reply: tid=0 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 3, 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24475, 2024-09-10T10:44:40.380 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 69.825279257000005, 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 134.11978497800001 2024-09-10T10:44:40.381 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.356+0000 7f82d1baa740 1 client.15075 shutdown 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 msgr2=0x5606ad6434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 0x5606ad6434f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82c0010ff0 tx=0x7f82c0005bd0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 shutdown_connections 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5606ad6ab760 0x5606ad6aa470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606ad62bbb0 0x5606ad6434f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606ad6212f0 0x5606ad642fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82b0ff9640 0 client.15075 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 --2- 172.21.15.143:0/2794791786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606ad6209d0 0x5606ad646320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 >> 172.21.15.143:0/2794791786 conn(0x5606ad61cf60 msgr2=0x5606ad623550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 shutdown_connections 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 1 -- 172.21.15.143:0/2794791786 wait complete. 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82d1baa740 20 client.15075 trim_cache size 0 max 16384 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f82b40a2c10 msgr2=0x7f82b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f82b40a2c10 0x7f82b40a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82c0005c40 tx=0x7f82c000ac90 comp rx=0 tx=0).stop 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 msgr2=0x7f82c81a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 0x7f82c81a4770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b4a0 tx=0x7f82b800b960 comp rx=0 tx=0).stop 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 shutdown_connections 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f82b40a2c10 0x7f82b40a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82c8105ec0 0x7f82c819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c81011e0 0x7f82c81a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 --2- 172.21.15.143:0/4104592480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c81008c0 0x7f82c81a4770 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.357+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 >> 172.21.15.143:0/4104592480 conn(0x7f82c80fc7d0 msgr2=0x7f82c8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.358+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 shutdown_connections 2024-09-10T10:44:40.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:40.358+0000 7f82cf7d1640 1 -- 172.21.15.143:0/4104592480 wait complete. 2024-09-10T10:44:40.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:40 smithi143 ceph-mon[31057]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T10:44:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:40 smithi178 ceph-mon[34813]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T10:44:40.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:40 smithi144 ceph-mon[34230]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T10:44:41.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T10:44:41.449 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 -- 172.21.15.143:0/2568044806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714106f10 msgr2=0x7fb71410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 --2- 172.21.15.143:0/2568044806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714106f10 0x7fb71410ea90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009ca0 tx=0x7fb70c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 -- 172.21.15.143:0/2568044806 shutdown_connections 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 --2- 172.21.15.143:0/2568044806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714106f10 0x7fb71410ea90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 --2- 172.21.15.143:0/2568044806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714102790 0x7fb7141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 --2- 172.21.15.143:0/2568044806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 0x7fb7141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 -- 172.21.15.143:0/2568044806 >> 172.21.15.143:0/2568044806 conn(0x7fb7140fd8c0 msgr2=0x7fb7140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 -- 172.21.15.143:0/2568044806 shutdown_connections 2024-09-10T10:44:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.942+0000 7fb71a49f640 1 -- 172.21.15.143:0/2568044806 wait complete. 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 Processor -- start 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 -- start start 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 0x7fb7140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 0x7fb714074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714106f10 0x7fb714074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb714111ec0 con 0x7fb714102790 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb714111d40 con 0x7fb714106f10 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb71a49f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb714112040 con 0x7fb714101de0 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 0x7fb714074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 0x7fb714074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49872/0 (socket says 172.21.15.143:49872) 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb7137fe640 1 -- 172.21.15.143:0/3302187658 learned_addr learned my addr 172.21.15.143:0/3302187658 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb718a3c640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714106f10 0x7fb714074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.943+0000 7fb713fff640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 0x7fb7140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7137fe640 1 -- 172.21.15.143:0/3302187658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 msgr2=0x7fb7140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7137fe640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 0x7fb7140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7137fe640 1 -- 172.21.15.143:0/3302187658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714106f10 msgr2=0x7fb714074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7137fe640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714106f10 0x7fb714074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7137fe640 1 -- 172.21.15.143:0/3302187658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7141a68c0 con 0x7fb714102790 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb718a3c640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714106f10 0x7fb714074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb713fff640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 0x7fb7140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7137fe640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 0x7fb714074a30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb70800b4a0 tx=0x7fb70800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7117fa640 1 -- 172.21.15.143:0/3302187658 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708014070 con 0x7fb714102790 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7117fa640 1 -- 172.21.15.143:0/3302187658 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb708010ac0 con 0x7fb714102790 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7141a6bb0 con 0x7fb714102790 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb7117fa640 1 -- 172.21.15.143:0/3302187658 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708017c30 con 0x7fb714102790 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.944+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7141a7140 con 0x7fb714102790 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.946+0000 7fb7117fa640 1 -- 172.21.15.143:0/3302187658 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb708017070 con 0x7fb714102790 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 Processor -- start 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 -- start start 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 0x562bc732bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7336670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb7117fa640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb6fc0a2c90 0x7fb6fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7338fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562bc7292090 con 0x562bc732b9d0 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562bc7239a70 con 0x562bc732c2f0 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb71c878740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562bc728e050 con 0x562bc7336bb0 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 0x562bc732bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 0x562bc732bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49874/0 (socket says 172.21.15.143:49874) 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 -- 172.21.15.143:0/4101295239 learned_addr learned my addr 172.21.15.143:0/4101295239 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb7137fe640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb6fc0a2c90 0x7fb6fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb713fff640 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7336670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb7117fa640 1 -- 172.21.15.143:0/3302187658 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb7080ceb00 con 0x7fb714102790 2024-09-10T10:44:41.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 -- 172.21.15.143:0/4101295239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 msgr2=0x562bc7338fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7338fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 -- 172.21.15.143:0/4101295239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 msgr2=0x562bc7336670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7336670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb718a3c640 1 -- 172.21.15.143:0/4101295239 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc7246130 con 0x562bc732b9d0 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb713fff640 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7336670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.947+0000 7fb7137fe640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb6fc0a2c90 0x7fb6fc0a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb70809af40 tx=0x7fb7080bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb718a3c640 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 0x562bc732bdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009430 tx=0x7fb70c031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e6f4e38c16524 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb6fa7fc640 1 -- 172.21.15.143:0/4101295239 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c041070 con 0x562bc732b9d0 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb6fa7fc640 1 -- 172.21.15.143:0/4101295239 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb70c036050 con 0x562bc732b9d0 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb6fa7fc640 1 -- 172.21.15.143:0/4101295239 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0309c0 con 0x562bc732b9d0 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 -- 172.21.15.143:0/4101295239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 msgr2=0x562bc732bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 0x562bc732bdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009430 tx=0x7fb70c031b80 comp rx=0 tx=0).stop 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 -- 172.21.15.143:0/4101295239 shutdown_connections 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7338fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7336670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 --2- 172.21.15.143:0/4101295239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732b9d0 0x562bc732bdb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 -- 172.21.15.143:0/4101295239 >> 172.21.15.143:0/4101295239 conn(0x562bc7327f60 msgr2=0x562bc7328340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 -- 172.21.15.143:0/4101295239 shutdown_connections 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.948+0000 7fb71c878740 1 -- 172.21.15.143:0/4101295239 wait complete. 2024-09-10T10:44:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 -- start start 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 Processor -- start 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc7336450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 0x562bc73337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7333ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562bc7339780 con 0x562bc732c2f0 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562bc724cf20 con 0x562bc732b9d0 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:44:41.998 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 2, 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24481, 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 72.449554804000002, 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 133.134205772 2024-09-10T10:44:41.999 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562bc7339900 con 0x562bc7336bb0 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb71c878740 10 client.0 _ll_register_callbacks cb 0x562bc7216030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb713fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 0x562bc73337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb713fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 0x562bc73337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49890/0 (socket says 172.21.15.143:49890) 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.949+0000 7fb713fff640 1 -- 172.21.15.143:0/2512324511 learned_addr learned my addr 172.21.15.143:0/2512324511 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb718a3c640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7333ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb718a3c640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc7336450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb713fff640 1 -- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 msgr2=0x562bc7333ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb713fff640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7333ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb713fff640 1 -- 172.21.15.143:0/2512324511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 msgr2=0x562bc7336450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb713fff640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc7336450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb713fff640 1 -- 172.21.15.143:0/2512324511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc70032b0 con 0x562bc732c2f0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb718a3c640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc7336450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb713fff640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 0x562bc73337a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb70400e2d0 tx=0x7fb70400e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796360c2f13f8620 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb6f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb6f37fe640 1 -- 172.21.15.143:0/2512324511 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704018020 con 0x562bc732c2f0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb6f37fe640 1 -- 172.21.15.143:0/2512324511 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb704013660 con 0x562bc732c2f0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb70c036050 con 0x562bc732c2f0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb6f37fe640 1 -- 172.21.15.143:0/2512324511 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb704023050 con 0x562bc732c2f0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb71c878740 10 client.15087 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.950+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562bc7345130 con 0x562bc732c2f0 2024-09-10T10:44:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb6f37fe640 1 -- 172.21.15.143:0/2512324511 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb70c036050 con 0x562bc732c2f0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb6f37fe640 1 -- 172.21.15.143:0/2512324511 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb704014a80 con 0x562bc732c2f0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb71c878740 10 client.15087 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb71c878740 10 client.15087 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb71c878740 20 client.15087 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb71c878740 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc7348c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb71c878740 4 client.15087 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bc729f600 con 0x562bc73477a0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb718a3c640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc7348c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb6f37fe640 10 client.15087 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.951+0000 7fb718a3c640 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc7348c10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18c5353c133f7267 server_cookie=5e7968a9efa8a6a5 in_seq=0 out_seq=0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.952+0000 7fb6f37fe640 1 -- 172.21.15.143:0/2512324511 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bc729f600 con 0x562bc73477a0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.952+0000 7fb6f37fe640 10 client.15087 handle_command_reply: tid=0 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.952+0000 7fb71c878740 1 client.15087 shutdown 2024-09-10T10:44:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.952+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 msgr2=0x562bc73337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.952+0000 7fb71c878740 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 0x562bc73337a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb70400e2d0 tx=0x7fb70400e790 comp rx=0 tx=0).stop 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 shutdown_connections 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc7348c10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb6f37fe640 0 client.15087 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc7336bb0 0x562bc7333ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc732c2f0 0x562bc73337a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 --2- 172.21.15.143:0/2512324511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc7336450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 >> 172.21.15.143:0/2512324511 conn(0x562bc7327f60 msgr2=0x562bc732dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 shutdown_connections 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 1 -- 172.21.15.143:0/2512324511 wait complete. 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.953+0000 7fb71c878740 20 client.15087 trim_cache size 0 max 16384 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 Processor -- start 2024-09-10T10:44:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 -- start start 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732b9d0 0x562bc7348c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7328340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc7350ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562bc7346f50 con 0x562bc7336bb0 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562bc7347620 con 0x562bc732c2f0 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb71c878740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562bc7347cf0 con 0x562bc732b9d0 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.970+0000 7fb718a3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc7350ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc7350ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49902/0 (socket says 172.21.15.143:49902) 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 -- 172.21.15.143:0/30407919 learned_addr learned my addr 172.21.15.143:0/30407919 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732b9d0 0x562bc7348c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 -- 172.21.15.143:0/30407919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732b9d0 msgr2=0x562bc7348c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732b9d0 0x562bc7348c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 -- 172.21.15.143:0/30407919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 msgr2=0x562bc7328340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb713fff640 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7328340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7328340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 -- 172.21.15.143:0/30407919 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc732c120 con 0x562bc7336bb0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb718a3c640 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc7350ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0319b0 tx=0x7fb70c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3e9378cf9049d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb6f9ffb640 1 -- 172.21.15.143:0/30407919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c041040 con 0x562bc7336bb0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb6f9ffb640 1 -- 172.21.15.143:0/30407919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb70c036050 con 0x562bc7336bb0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.971+0000 7fb6f9ffb640 1 -- 172.21.15.143:0/30407919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c030cd0 con 0x562bc7336bb0 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 -- 172.21.15.143:0/30407919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 msgr2=0x562bc7350ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc7350ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0319b0 tx=0x7fb70c000f40 comp rx=0 tx=0).stop 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 -- 172.21.15.143:0/30407919 shutdown_connections 2024-09-10T10:44:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc7350ec0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732c2f0 0x562bc7328340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 --2- 172.21.15.143:0/30407919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732b9d0 0x562bc7348c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 -- 172.21.15.143:0/30407919 >> 172.21.15.143:0/30407919 conn(0x562bc7327f60 msgr2=0x562bc7335710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 -- 172.21.15.143:0/30407919 shutdown_connections 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 -- 172.21.15.143:0/30407919 wait complete. 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 -- start start 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 Processor -- start 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.972+0000 7fb71c878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc734bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732c2f0 0x562bc734c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc73afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562bc7348a90 con 0x562bc7336bb0 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562bc73483c0 con 0x562bc732b9d0 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562bc7349160 con 0x562bc732c2f0 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 10 client.0 _ll_register_callbacks cb 0x7fb70400c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc73afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc73afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49904/0 (socket says 172.21.15.143:49904) 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 -- 172.21.15.143:0/3554619864 learned_addr learned my addr 172.21.15.143:0/3554619864 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb713fff640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732c2f0 0x562bc734c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 -- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732c2f0 msgr2=0x562bc734c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732c2f0 0x562bc734c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 -- 172.21.15.143:0/3554619864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 msgr2=0x562bc734bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc734bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 -- 172.21.15.143:0/3554619864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc734cd80 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb713fff640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732c2f0 0x562bc734c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb718a3c640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc73afea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0305f0 tx=0x7fb70c030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb04fb7f1c25be62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb6f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb6f37fe640 1 -- 172.21.15.143:0/3554619864 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c041040 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb70c036050 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb6f37fe640 1 -- 172.21.15.143:0/3554619864 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb70c0317a0 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb6f37fe640 1 -- 172.21.15.143:0/3554619864 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb70c030950 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 10 client.15093 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.973+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562bc73b1510 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb6f37fe640 1 -- 172.21.15.143:0/3554619864 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb70c036050 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb6f37fe640 1 -- 172.21.15.143:0/3554619864 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb70c004380 con 0x562bc7336bb0 2024-09-10T10:44:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb71c878740 10 client.15093 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb71c878740 10 client.15093 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb71c878740 20 client.15093 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb71c878740 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc73b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb71c878740 4 client.15093 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "status"}) -- 0x562bc7299b10 con 0x562bc73477a0 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb713fff640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc73b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb713fff640 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc73b47e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4328f1fbddaaf0af server_cookie=6ebb4cdbda7a856c in_seq=0 out_seq=0 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.974+0000 7fb6f37fe640 10 client.15093 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb6f37fe640 1 -- 172.21.15.143:0/3554619864 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562bc7299b10 con 0x562bc73477a0 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb6f37fe640 10 client.15093 handle_command_reply: tid=0 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 client.15093 shutdown 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 msgr2=0x562bc73afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc73afea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0305f0 tx=0x7fb70c030620 comp rx=0 tx=0).stop 2024-09-10T10:44:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 shutdown_connections 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562bc73477a0 0x562bc73b47e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562bc7336bb0 0x562bc73afea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562bc732c2f0 0x562bc734c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb6f37fe640 0 client.15093 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 --2- 172.21.15.143:0/3554619864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bc732b9d0 0x562bc734bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 >> 172.21.15.143:0/3554619864 conn(0x562bc7327f60 msgr2=0x562bc7331f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 shutdown_connections 2024-09-10T10:44:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 1 -- 172.21.15.143:0/3554619864 wait complete. 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.975+0000 7fb71c878740 20 client.15093 trim_cache size 0 max 16384 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb6fc0a2c90 msgr2=0x7fb6fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb6fc0a2c90 0x7fb6fc0a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb70809af40 tx=0x7fb7080bb040 comp rx=0 tx=0).stop 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 msgr2=0x7fb714074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 0x7fb714074a30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb70800b4a0 tx=0x7fb70800b960 comp rx=0 tx=0).stop 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 shutdown_connections 2024-09-10T10:44:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb6fc0a2c90 0x7fb6fc0a5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb714106f10 0x7fb714074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb714102790 0x7fb714074a30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 --2- 172.21.15.143:0/3302187658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb714101de0 0x7fb7140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 >> 172.21.15.143:0/3302187658 conn(0x7fb7140fd8c0 msgr2=0x7fb7140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 shutdown_connections 2024-09-10T10:44:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:41.976+0000 7fb71a49f640 1 -- 172.21.15.143:0/3302187658 wait complete. 2024-09-10T10:44:42.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:42 smithi143 ceph-mon[31057]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T10:44:42.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:44:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:42 smithi178 ceph-mon[34813]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T10:44:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:42 smithi144 ceph-mon[34230]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T10:44:43.127 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 -- 172.21.15.143:0/577432029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c105a70 msgr2=0x7f089c105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 --2- 172.21.15.143:0/577432029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c105a70 0x7f089c105e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f088c009900 tx=0x7f088c02a410 comp rx=0 tx=0).stop 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 -- 172.21.15.143:0/577432029 shutdown_connections 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 --2- 172.21.15.143:0/577432029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c101ba0 0x7f089c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 --2- 172.21.15.143:0/577432029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f089c101220 0x7f089c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 --2- 172.21.15.143:0/577432029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c105a70 0x7f089c105e50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 -- 172.21.15.143:0/577432029 >> 172.21.15.143:0/577432029 conn(0x7f089c0fbee0 msgr2=0x7f089c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 -- 172.21.15.143:0/577432029 shutdown_connections 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.621+0000 7f08a27e3640 1 -- 172.21.15.143:0/577432029 wait complete. 2024-09-10T10:44:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 Processor -- start 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 -- start start 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 0x7f089c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f089c101ba0 0x7f089c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c105a70 0x7f089c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f089c110e40 con 0x7f089c101220 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f089c110cc0 con 0x7f089c101ba0 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a27e3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f089c110fc0 con 0x7f089c105a70 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 0x7f089c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 0x7f089c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49922/0 (socket says 172.21.15.143:49922) 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 -- 172.21.15.143:0/1636862615 learned_addr learned my addr 172.21.15.143:0/1636862615 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089b7fe640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f089c101ba0 0x7f089c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c105a70 0x7f089c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 -- 172.21.15.143:0/1636862615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c105a70 msgr2=0x7f089c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c105a70 0x7f089c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 -- 172.21.15.143:0/1636862615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f089c101ba0 msgr2=0x7f089c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.622+0000 7f089bfff640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f089c101ba0 0x7f089c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f089bfff640 1 -- 172.21.15.143:0/1636862615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19d340 con 0x7f089c101220 2024-09-10T10:44:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c105a70 0x7f089c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f089bfff640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 0x7f089c198390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f088c006010 tx=0x7f088c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f08997fa640 1 -- 172.21.15.143:0/1636862615 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c03f070 con 0x7f089c101220 2024-09-10T10:44:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f08997fa640 1 -- 172.21.15.143:0/1636862615 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f088c003aa0 con 0x7f089c101220 2024-09-10T10:44:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19d5d0 con 0x7f089c101220 2024-09-10T10:44:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f08997fa640 1 -- 172.21.15.143:0/1636862615 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c035030 con 0x7f089c101220 2024-09-10T10:44:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.623+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a4f10 con 0x7f089c101220 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.625+0000 7f08997fa640 1 -- 172.21.15.143:0/1636862615 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f088c042070 con 0x7f089c101220 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.625+0000 7f08a4bbc740 1 Processor -- start 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 -- start start 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08997fa640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08840a2c10 0x7f08840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459be4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be51d0 0x564459bf3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564459b4b090 con 0x564459bf3fc0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564459af2a70 con 0x564459be48b0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564459b47050 con 0x564459be51d0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08840a2c10 0x7f08840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08997fa640 1 -- 172.21.15.143:0/1636862615 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f088c0efee0 con 0x7f089c101220 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49928/0 (socket says 172.21.15.143:49928) 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 -- 172.21.15.143:0/3255503964 learned_addr learned my addr 172.21.15.143:0/3255503964 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f089bfff640 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459be4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f089b7fe640 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be51d0 0x564459bf3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 -- 172.21.15.143:0/3255503964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be51d0 msgr2=0x564459bf3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be51d0 0x564459bf3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 -- 172.21.15.143:0/3255503964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 msgr2=0x564459be4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459be4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 -- 172.21.15.143:0/3255503964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564459aff130 con 0x564459bf3fc0 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f089bfff640 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459be4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.626+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08840a2c10 0x7f08840a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f089400ec60 tx=0x7f0894009e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a0d80640 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf63b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f089400cfd0 tx=0x7f0894018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c341790cf53fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08827fc640 1 -- 172.21.15.143:0/3255503964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894020020 con 0x564459bf3fc0 2024-09-10T10:44:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08827fc640 1 -- 172.21.15.143:0/3255503964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f089400e510 con 0x564459bf3fc0 2024-09-10T10:44:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08827fc640 1 -- 172.21.15.143:0/3255503964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089401a3f0 con 0x564459bf3fc0 2024-09-10T10:44:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 -- 172.21.15.143:0/3255503964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 msgr2=0x564459bf63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf63b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f089400cfd0 tx=0x7f0894018000 comp rx=0 tx=0).stop 2024-09-10T10:44:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 -- 172.21.15.143:0/3255503964 shutdown_connections 2024-09-10T10:44:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf63b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be51d0 0x564459bf3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/3255503964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459be4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 -- 172.21.15.143:0/3255503964 >> 172.21.15.143:0/3255503964 conn(0x564459be0f60 msgr2=0x564459be1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 -- 172.21.15.143:0/3255503964 shutdown_connections 2024-09-10T10:44:43.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.627+0000 7f08a4bbc740 1 -- 172.21.15.143:0/3255503964 wait complete. 2024-09-10T10:44:43.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 -- start start 2024-09-10T10:44:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 Processor -- start 2024-09-10T10:44:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459bf3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459bf0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564459bf6b90 con 0x564459bf3fc0 2024-09-10T10:44:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564459b05f20 con 0x564459be51d0 2024-09-10T10:44:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564459bf6d10 con 0x564459be48b0 2024-09-10T10:44:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a4bbc740 10 client.0 _ll_register_callbacks cb 0x564459acf030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a0d80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a0d80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49936/0 (socket says 172.21.15.143:49936) 2024-09-10T10:44:43.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.628+0000 7f08a0d80640 1 -- 172.21.15.143:0/1150498683 learned_addr learned my addr 172.21.15.143:0/1150498683 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f089b7fe640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459bf0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f089bfff640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459bf3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a0d80640 1 -- 172.21.15.143:0/1150498683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 msgr2=0x564459bf3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459bf3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a0d80640 1 -- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 msgr2=0x564459bf0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459bf0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a0d80640 1 -- 172.21.15.143:0/1150498683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644598bc2b0 con 0x564459bf3fc0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f089bfff640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459bf3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf1100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f089401e040 tx=0x7f089401ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253268720c053068 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f087b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f087b7fe640 1 -- 172.21.15.143:0/1150498683 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089402b020 con 0x564459bf3fc0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f087b7fe640 1 -- 172.21.15.143:0/1150498683 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f089401c430 con 0x564459bf3fc0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f089400e510 con 0x564459bf3fc0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f087b7fe640 1 -- 172.21.15.143:0/1150498683 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0894036050 con 0x564459bf3fc0 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a4bbc740 10 client.15105 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:43.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564459c02570 con 0x564459bf3fc0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f087b7fe640 1 -- 172.21.15.143:0/1150498683 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f089400e510 con 0x564459bf3fc0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f087b7fe640 1 -- 172.21.15.143:0/1150498683 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f089401b070 con 0x564459bf3fc0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.629+0000 7f08a4bbc740 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f08a4bbc740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f08a4bbc740 20 client.15105 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c05e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f08a4bbc740 4 client.15105 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564459b58600 con 0x564459c049c0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f089bfff640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c05e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f087b7fe640 10 client.15105 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.630+0000 7f089bfff640 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c05e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e55277837d127a2 server_cookie=10ccc6d2a4ef833b in_seq=0 out_seq=0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f087b7fe640 1 -- 172.21.15.143:0/1150498683 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564459b58600 con 0x564459c049c0 2024-09-10T10:44:43.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f087b7fe640 10 client.15105 handle_command_reply: tid=0 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 client.15105 shutdown 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 msgr2=0x564459bf1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf1100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f089401e040 tx=0x7f089401ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 shutdown_connections 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bf1100 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c05e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f087b7fe640 0 client.15105 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459bf0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1150498683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459bf3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 >> 172.21.15.143:0/1150498683 conn(0x564459be0f60 msgr2=0x564459be6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 shutdown_connections 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1150498683 wait complete. 2024-09-10T10:44:43.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.631+0000 7f08a4bbc740 20 client.15105 trim_cache size 0 max 16384 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 Processor -- start 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 -- start start 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459c05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459be05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564459c04170 con 0x564459bf3fc0 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564459c04840 con 0x564459be51d0 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564459c04f10 con 0x564459be48b0 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bede20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49944/0 (socket says 172.21.15.143:49944) 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 -- 172.21.15.143:0/1163862592 learned_addr learned my addr 172.21.15.143:0/1163862592 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f089bfff640 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459c05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f089b7fe640 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459be05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 -- 172.21.15.143:0/1163862592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 msgr2=0x564459c05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459c05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 -- 172.21.15.143:0/1163862592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 msgr2=0x564459be05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459be05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 -- 172.21.15.143:0/1163862592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564459be20d0 con 0x564459bf3fc0 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.649+0000 7f08a0d80640 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bede20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0894004b90 tx=0x7f0894009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cf1a5adc5b8ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f0881ffb640 1 -- 172.21.15.143:0/1163862592 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089402b070 con 0x564459bf3fc0 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f0881ffb640 1 -- 172.21.15.143:0/1163862592 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0894027dd0 con 0x564459bf3fc0 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f0881ffb640 1 -- 172.21.15.143:0/1163862592 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894020040 con 0x564459bf3fc0 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1163862592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 msgr2=0x564459bede20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bede20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0894004b90 tx=0x7f0894009620 comp rx=0 tx=0).stop 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1163862592 shutdown_connections 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459bf3fc0 0x564459bede20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be51d0 0x564459be05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/1163862592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459be48b0 0x564459c05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1163862592 >> 172.21.15.143:0/1163862592 conn(0x564459be0f60 msgr2=0x564459bec240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1163862592 shutdown_connections 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 -- 172.21.15.143:0/1163862592 wait complete. 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.650+0000 7f08a4bbc740 1 -- start start 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 Processor -- start 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459c0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 0x564459c0e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459bf3fc0 0x564459c075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564459c05cb0 con 0x564459be51d0 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564459c055e0 con 0x564459be48b0 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564459c06380 con 0x564459bf3fc0 2024-09-10T10:44:43.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a4bbc740 10 client.0 _ll_register_callbacks cb 0x7f088c005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 0x564459c0e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 0x564459c0e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49958/0 (socket says 172.21.15.143:49958) 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 -- 172.21.15.143:0/135938253 learned_addr learned my addr 172.21.15.143:0/135938253 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24485, 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:44:43.678 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 75.136272797000004, 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 120.61627239400001 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089bfff640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459c0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a0d80640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459bf3fc0 0x564459c075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 -- 172.21.15.143:0/135938253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459bf3fc0 msgr2=0x564459c075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459bf3fc0 0x564459c075f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 -- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 msgr2=0x564459c0e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459c0e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089b7fe640 1 -- 172.21.15.143:0/135938253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564459c07e00 con 0x564459be51d0 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f089bfff640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459c0e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.651+0000 7f08a0d80640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459bf3fc0 0x564459c075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f089b7fe640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 0x564459c0e800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x564459be75c0 tx=0x7f0890012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de56636e7a00dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f087b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f087b7fe640 1 -- 172.21.15.143:0/135938253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089000f950 con 0x564459be51d0 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f087b7fe640 1 -- 172.21.15.143:0/135938253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0890019040 con 0x564459be51d0 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0894027dd0 con 0x564459be51d0 2024-09-10T10:44:43.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f087b7fe640 1 -- 172.21.15.143:0/135938253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0890029050 con 0x564459be51d0 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 10 client.15111 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564459c0c890 con 0x564459be51d0 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f087b7fe640 1 -- 172.21.15.143:0/135938253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0894027dd0 con 0x564459be51d0 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f087b7fe640 1 -- 172.21.15.143:0/135938253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0890010a00 con 0x564459be51d0 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 10 client.15111 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 20 client.15111 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c72530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 4 client.15111 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.652+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "status"}) -- 0x564459b52b10 con 0x564459c049c0 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f089bfff640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c72530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f089bfff640 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c72530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51e1b5582b0bce06 server_cookie=a0b0fd6804d82a15 in_seq=0 out_seq=0 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f087b7fe640 10 client.15111 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f087b7fe640 1 -- 172.21.15.143:0/135938253 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564459b52b10 con 0x564459c049c0 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f087b7fe640 10 client.15111 handle_command_reply: tid=0 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 client.15111 shutdown 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 msgr2=0x564459c0e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 0x564459c0e800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x564459be75c0 tx=0x7f0890012e10 comp rx=0 tx=0).stop 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 shutdown_connections 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564459bf3fc0 0x564459c075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564459be51d0 0x564459c0e800 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x564459c049c0 0x564459c72530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.653+0000 7f08a4bbc740 1 --2- 172.21.15.143:0/135938253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564459be48b0 0x564459c0e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 >> 172.21.15.143:0/135938253 conn(0x564459be0f60 msgr2=0x564459bef490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f087b7fe640 0 client.15111 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:44:43.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 shutdown_connections 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a4bbc740 1 -- 172.21.15.143:0/135938253 wait complete. 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a4bbc740 20 client.15111 trim_cache size 0 max 16384 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08840a2c10 msgr2=0x7f08840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08840a2c10 0x7f08840a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f089400ec60 tx=0x7f0894009e30 comp rx=0 tx=0).stop 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 msgr2=0x7f089c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 0x7f089c198390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f088c006010 tx=0x7f088c0042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 shutdown_connections 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08840a2c10 0x7f08840a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f089c105a70 0x7f089c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f089c101ba0 0x7f089c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 --2- 172.21.15.143:0/1636862615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f089c101220 0x7f089c198390 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.654+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 >> 172.21.15.143:0/1636862615 conn(0x7f089c0fbee0 msgr2=0x7f089c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.655+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 shutdown_connections 2024-09-10T10:44:43.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:43.655+0000 7f08a27e3640 1 -- 172.21.15.143:0/1636862615 wait complete. 2024-09-10T10:44:44.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T10:44:44.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:44 smithi143 ceph-mon[31057]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T10:44:44.748 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:44.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:44 smithi178 ceph-mon[34813]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T10:44:44.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:44 smithi144 ceph-mon[34230]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 -- 172.21.15.143:0/20802272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 msgr2=0x7f1c1c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 --2- 172.21.15.143:0/20802272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c1021d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009900 tx=0x7f1c1802a410 comp rx=0 tx=0).stop 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 -- 172.21.15.143:0/20802272 shutdown_connections 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 --2- 172.21.15.143:0/20802272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c106f20 0x7f1c1c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 --2- 172.21.15.143:0/20802272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 0x7f1c1c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 --2- 172.21.15.143:0/20802272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c1021d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 -- 172.21.15.143:0/20802272 >> 172.21.15.143:0/20802272 conn(0x7f1c1c0fd8f0 msgr2=0x7f1c1c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 -- 172.21.15.143:0/20802272 shutdown_connections 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.251+0000 7f1c247a6640 1 -- 172.21.15.143:0/20802272 wait complete. 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 Processor -- start 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 -- start start 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 0x7f1c1c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c106f20 0x7f1c1c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c111e30 con 0x7f1c1c101df0 2024-09-10T10:44:45.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c111cb0 con 0x7f1c1c106f20 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c247a6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c111fb0 con 0x7f1c1c1027a0 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c22542640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c22542640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49978/0 (socket says 172.21.15.143:49978) 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c22542640 1 -- 172.21.15.143:0/4135491026 learned_addr learned my addr 172.21.15.143:0/4135491026 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c21d41640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 0x7f1c1c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.252+0000 7f1c22d43640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c106f20 0x7f1c1c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c22542640 1 -- 172.21.15.143:0/4135491026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 msgr2=0x7f1c1c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c22542640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 0x7f1c1c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c22542640 1 -- 172.21.15.143:0/4135491026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c106f20 msgr2=0x7f1c1c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c22542640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c106f20 0x7f1c1c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c22542640 1 -- 172.21.15.143:0/4135491026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c19d340 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c21d41640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 0x7f1c1c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c22542640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c198390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005940 tx=0x7f1c180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c137fe640 1 -- 172.21.15.143:0/4135491026 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1803f030 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c19d5d0 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c137fe640 1 -- 172.21.15.143:0/4135491026 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c180044c0 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c137fe640 1 -- 172.21.15.143:0/4135491026 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18035070 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.253+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c19da30 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.256+0000 7f1c137fe640 1 -- 172.21.15.143:0/4135491026 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1c18042030 con 0x7f1c1c101df0 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.256+0000 7f1c26b7f740 1 Processor -- start 2024-09-10T10:44:45.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.256+0000 7f1c26b7f740 1 -- start start 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.256+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a289d0 0x558ca3a28db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.256+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a33670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a35fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c137fe640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c080a2c90 0x7f1c080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558ca398f090 con 0x558ca3a289d0 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558ca3936a70 con 0x558ca3a33bb0 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ca398b050 con 0x558ca3a292f0 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22d43640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a289d0 0x558ca3a28db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a33670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a33670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:57178/0 (socket says 172.21.15.143:57178) 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c21d41640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a35fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 -- 172.21.15.143:0/2211690723 learned_addr learned my addr 172.21.15.143:0/2211690723 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:45.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22d43640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c080a2c90 0x7f1c080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c137fe640 1 -- 172.21.15.143:0/4135491026 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1c180ef7b0 con 0x7f1c1c101df0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 -- 172.21.15.143:0/2211690723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 msgr2=0x558ca3a35fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a35fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 -- 172.21.15.143:0/2211690723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a289d0 msgr2=0x558ca3a28db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a289d0 0x558ca3a28db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 -- 172.21.15.143:0/2211690723 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca3943130 con 0x558ca3a292f0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c21d41640 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22d43640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c080a2c90 0x7f1c080a5150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009fd0 tx=0x7f1c14004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c22542640 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a33670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c180b85d0 tx=0x7f1c18032a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd306f68f9285d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.257+0000 7f1c10ff9640 1 -- 172.21.15.143:0/2211690723 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18049070 con 0x558ca3a292f0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c10ff9640 1 -- 172.21.15.143:0/2211690723 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c180bbc00 con 0x558ca3a292f0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c10ff9640 1 -- 172.21.15.143:0/2211690723 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1808d0d0 con 0x558ca3a292f0 2024-09-10T10:44:45.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 -- 172.21.15.143:0/2211690723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 msgr2=0x558ca3a33670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a33670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c180b85d0 tx=0x7f1c18032a90 comp rx=0 tx=0).stop 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 -- 172.21.15.143:0/2211690723 shutdown_connections 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a33670 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/2211690723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a289d0 0x558ca3a28db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 -- 172.21.15.143:0/2211690723 >> 172.21.15.143:0/2211690723 conn(0x558ca3a24f60 msgr2=0x558ca3a25340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 -- 172.21.15.143:0/2211690723 shutdown_connections 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.258+0000 7f1c26b7f740 1 -- 172.21.15.143:0/2211690723 wait complete. 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 -- start start 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 Processor -- start 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a289d0 0x558ca3a333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a30700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 0x558ca3a30c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a36540 con 0x558ca3a33bb0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558ca3949f20 con 0x558ca3a289d0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a366c0 con 0x558ca3a292f0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c26b7f740 10 client.0 _ll_register_callbacks cb 0x558ca3913030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c22d43640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 0x558ca3a30c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c22d43640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 0x558ca3a30c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50010/0 (socket says 172.21.15.143:50010) 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c22d43640 1 -- 172.21.15.143:0/78431351 learned_addr learned my addr 172.21.15.143:0/78431351 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c22542640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a289d0 0x558ca3a333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.259+0000 7f1c21d41640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a30700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c22d43640 1 -- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 msgr2=0x558ca3a30700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c22d43640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a30700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c22d43640 1 -- 172.21.15.143:0/78431351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a289d0 msgr2=0x558ca3a333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c22d43640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a289d0 0x558ca3a333b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c22d43640 1 -- 172.21.15.143:0/78431351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca37002b0 con 0x558ca3a33bb0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c22d43640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 0x558ca3a30c40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1c140106e0 tx=0x7f1c14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d27d689d04d0173 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c05ffb640 1 -- 172.21.15.143:0/78431351 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1401f600 con 0x558ca3a33bb0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c05ffb640 1 -- 172.21.15.143:0/78431351 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1c1401f7a0 con 0x558ca3a33bb0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c180bbc00 con 0x558ca3a33bb0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c05ffb640 1 -- 172.21.15.143:0/78431351 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c14035050 con 0x558ca3a33bb0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c26b7f740 10 client.15123 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ca3a41d40 con 0x558ca3a33bb0 2024-09-10T10:44:45.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c05ffb640 1 -- 172.21.15.143:0/78431351 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c180bbc00 con 0x558ca3a33bb0 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c05ffb640 1 -- 172.21.15.143:0/78431351 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1c1402a980 con 0x558ca3a33bb0 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.260+0000 7f1c26b7f740 10 client.15123 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c26b7f740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c26b7f740 20 client.15123 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3a44450 0x558ca3a458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c26b7f740 4 client.15123 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca399c600 con 0x558ca3a44450 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c22542640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3a44450 0x558ca3a458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c22542640 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3a44450 0x558ca3a458c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e4d979d7741dbb server_cookie=6ee801a947ebf181 in_seq=0 out_seq=0 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.261+0000 7f1c05ffb640 10 client.15123 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c05ffb640 1 -- 172.21.15.143:0/78431351 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ca399c600 con 0x558ca3a44450 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c05ffb640 10 client.15123 handle_command_reply: tid=0 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 client.15123 shutdown 2024-09-10T10:44:45.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 msgr2=0x558ca3a30c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 0x558ca3a30c40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1c140106e0 tx=0x7f1c14004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 shutdown_connections 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a33bb0 0x558ca3a30c40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3a44450 0x558ca3a458c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a292f0 0x558ca3a30700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/78431351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a289d0 0x558ca3a333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c05ffb640 0 client.15123 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 >> 172.21.15.143:0/78431351 conn(0x558ca3a24f60 msgr2=0x558ca3a2ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 shutdown_connections 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 1 -- 172.21.15.143:0/78431351 wait complete. 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.262+0000 7f1c26b7f740 20 client.15123 trim_cache size 0 max 16384 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 Processor -- start 2024-09-10T10:44:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 -- start start 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a43c00 con 0x558ca3a292f0 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a442d0 con 0x558ca3a33bb0 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a449a0 con 0x558ca3a289d0 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a3f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50026/0 (socket says 172.21.15.143:50026) 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 -- 172.21.15.143:0/1522382969 learned_addr learned my addr 172.21.15.143:0/1522382969 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c22542640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c22d43640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 -- 172.21.15.143:0/1522382969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 msgr2=0x558ca3a458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 -- 172.21.15.143:0/1522382969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 msgr2=0x558ca3a306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c21d41640 1 -- 172.21.15.143:0/1522382969 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca3a24d90 con 0x558ca3a292f0 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c22542640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.280+0000 7f1c22d43640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c21d41640 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a3f620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0105c0 tx=0x7f1c0c010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7105ca4ea413ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c07fff640 1 -- 172.21.15.143:0/1522382969 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c021070 con 0x558ca3a292f0 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c07fff640 1 -- 172.21.15.143:0/1522382969 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c0c004310 con 0x558ca3a292f0 2024-09-10T10:44:45.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c07fff640 1 -- 172.21.15.143:0/1522382969 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00f9a0 con 0x558ca3a292f0 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 -- 172.21.15.143:0/1522382969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 msgr2=0x558ca3a3f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a3f620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0105c0 tx=0x7f1c0c010a80 comp rx=0 tx=0).stop 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 -- 172.21.15.143:0/1522382969 shutdown_connections 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3a306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a3f620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/1522382969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 -- 172.21.15.143:0/1522382969 >> 172.21.15.143:0/1522382969 conn(0x558ca3a24f60 msgr2=0x558ca3a1fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 -- 172.21.15.143:0/1522382969 shutdown_connections 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.281+0000 7f1c26b7f740 1 -- 172.21.15.143:0/1522382969 wait complete. 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 -- start start 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 Processor -- start 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a49070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3aacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a45740 con 0x558ca3a292f0 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a45070 con 0x558ca3a33bb0 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ca3a45e10 con 0x558ca3a289d0 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c26b7f740 10 client.0 _ll_register_callbacks cb 0x7f1c1400ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50042/0 (socket says 172.21.15.143:50042) 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 -- 172.21.15.143:0/3186474336 learned_addr learned my addr 172.21.15.143:0/3186474336 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c22d43640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3aacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 -- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 msgr2=0x558ca3a49070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c22542640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a49070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a49070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 -- 172.21.15.143:0/3186474336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 msgr2=0x558ca3aacb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3aacb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.282+0000 7f1c21d41640 1 -- 172.21.15.143:0/3186474336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca3aad110 con 0x558ca3a292f0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c21d41640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a495b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c01f040 tx=0x7f1c0c003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ba7b0b32dedf41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c05ffb640 1 -- 172.21.15.143:0/3186474336 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c02b060 con 0x558ca3a292f0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c05ffb640 1 -- 172.21.15.143:0/3186474336 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1c0c00f8f0 con 0x558ca3a292f0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c004310 con 0x558ca3a292f0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c05ffb640 1 -- 172.21.15.143:0/3186474336 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c032050 con 0x558ca3a292f0 2024-09-10T10:44:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c26b7f740 10 client.15129 fetch_fsmap learned FSMap version 28 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ca3aae360 con 0x558ca3a292f0 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c05ffb640 1 -- 172.21.15.143:0/3186474336 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c0c004310 con 0x558ca3a292f0 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c05ffb640 1 -- 172.21.15.143:0/3186474336 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1c0c01b030 con 0x558ca3a292f0 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c26b7f740 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.283+0000 7f1c26b7f740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 20 client.15129 populate_metadata read hostname 'smithi143' 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3ab2b70 0x558ca3ab1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 4 client.15129 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "status"}) -- 0x558ca3996b10 con 0x558ca3ab2b70 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c22542640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3ab2b70 0x558ca3ab1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c22542640 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3ab2b70 0x558ca3ab1880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c14c0d5bcad9e51 server_cookie=e98a217cb114252d in_seq=0 out_seq=0 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c05ffb640 10 client.15129 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c05ffb640 1 -- 172.21.15.143:0/3186474336 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558ca3996b10 con 0x558ca3ab2b70 2024-09-10T10:44:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c05ffb640 10 client.15129 handle_command_reply: tid=0 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 client.15129 shutdown 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 msgr2=0x558ca3a495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a495b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c01f040 tx=0x7f1c0c003d20 comp rx=0 tx=0).stop 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 shutdown_connections 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ca3a33bb0 0x558ca3aacb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558ca3a292f0 0x558ca3a495b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558ca3ab2b70 0x558ca3ab1880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 --2- 172.21.15.143:0/3186474336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ca3a289d0 0x558ca3a49070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.284+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 >> 172.21.15.143:0/3186474336 conn(0x558ca3a24f60 msgr2=0x558ca3a306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c05ffb640 0 client.15129 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:44:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 shutdown_connections 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c26b7f740 1 -- 172.21.15.143:0/3186474336 wait complete. 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c26b7f740 20 client.15129 trim_cache size 0 max 16384 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24502, 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:44:45.303 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 111.040150122, 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 131.18810355299999 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:44:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c080a2c90 msgr2=0x7f1c080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c080a2c90 0x7f1c080a5150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009fd0 tx=0x7f1c14004ab0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 msgr2=0x7f1c1c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c198390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005940 tx=0x7f1c180040c0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 shutdown_connections 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c080a2c90 0x7f1c080a5150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c1c106f20 0x7f1c1c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c1c1027a0 0x7f1c1c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 --2- 172.21.15.143:0/4135491026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c1c101df0 0x7f1c1c198390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 >> 172.21.15.143:0/4135491026 conn(0x7f1c1c0fd8f0 msgr2=0x7f1c1c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 shutdown_connections 2024-09-10T10:44:45.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:45.285+0000 7f1c247a6640 1 -- 172.21.15.143:0/4135491026 wait complete. 2024-09-10T10:44:46.140 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T10:44:46.140 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T10:44:46.150 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T10:44:46.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T10:44:46.397 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:46.798 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:46 smithi143 ceph-mon[31057]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:46 smithi178 ceph-mon[34813]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:46.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 -- 172.21.15.143:0/1779298387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0101d10 msgr2=0x7f22b0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 --2- 172.21.15.143:0/1779298387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0101d10 0x7f22b0102190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802a410 comp rx=0 tx=0).stop 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 -- 172.21.15.143:0/1779298387 shutdown_connections 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 --2- 172.21.15.143:0/1779298387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b01026d0 0x7f22b0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 --2- 172.21.15.143:0/1779298387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0101d10 0x7f22b0102190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 --2- 172.21.15.143:0/1779298387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0100b10 0x7f22b0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 -- 172.21.15.143:0/1779298387 >> 172.21.15.143:0/1779298387 conn(0x7f22b00fc2a0 msgr2=0x7f22b00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 -- 172.21.15.143:0/1779298387 shutdown_connections 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.876+0000 7f22b4b62640 1 -- 172.21.15.143:0/1779298387 wait complete. 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 Processor -- start 2024-09-10T10:44:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 -- start start 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 0x7f22b006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b0101d10 0x7f22b019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b01026d0 0x7f22b019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010c740 con 0x7f22b0100b10 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010c5c0 con 0x7f22b01026d0 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22b4b62640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010c8c0 con 0x7f22b0101d10 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 0x7f22b006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 0x7f22b006b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43794/0 (socket says 172.21.15.143:43794) 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 -- 172.21.15.143:0/884255203 learned_addr learned my addr 172.21.15.143:0/884255203 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22aed9d640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b01026d0 0x7f22b019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22add9b640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b0101d10 0x7f22b019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 -- 172.21.15.143:0/884255203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b0101d10 msgr2=0x7f22b019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b0101d10 0x7f22b019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 -- 172.21.15.143:0/884255203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b01026d0 msgr2=0x7f22b019f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b01026d0 0x7f22b019f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 -- 172.21.15.143:0/884255203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b019fc50 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22aed9d640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b01026d0 0x7f22b019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22ae59c640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 0x7f22b006b2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b450 tx=0x7f22a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.877+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0015020 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.878+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b019fee0 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.878+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0011d40 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.878+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a000f340 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.878+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b01a0460 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.879+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b0108050 con 0x7f22b0100b10 2024-09-10T10:44:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.880+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f22a0018020 con 0x7f22b0100b10 2024-09-10T10:44:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.880+0000 7f22a77fe640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f229c0a2c10 0x7f229c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.881+0000 7f22add9b640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f229c0a2c10 0x7f229c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.881+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f22a00cef50 con 0x7f22b0100b10 2024-09-10T10:44:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.881+0000 7f22add9b640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f229c0a2c10 0x7f229c0a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f229802a930 tx=0x7f2298000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:46.885+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22a001f020 con 0x7f22b0100b10 2024-09-10T10:44:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:46 smithi144 ceph-mon[34230]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:47.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.067+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f22b0062630 con 0x7f229c0a2c10 2024-09-10T10:44:47.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.069+0000 7f22a77fe640 1 -- 172.21.15.143:0/884255203 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f22b0062630 con 0x7f229c0a2c10 2024-09-10T10:44:47.070 INFO:teuthology.orchestra.run.smithi143.stdout:Backend: cephadm 2024-09-10T10:44:47.070 INFO:teuthology.orchestra.run.smithi143.stdout:Available: Yes 2024-09-10T10:44:47.070 INFO:teuthology.orchestra.run.smithi143.stdout:Paused: No 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f229c0a2c10 msgr2=0x7f229c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f229c0a2c10 0x7f229c0a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f229802a930 tx=0x7f2298000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 msgr2=0x7f22b006b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 0x7f22b006b2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b450 tx=0x7f22a000b910 comp rx=0 tx=0).stop 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 shutdown_connections 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f229c0a2c10 0x7f229c0a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b01026d0 0x7f22b019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b0101d10 0x7f22b019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 --2- 172.21.15.143:0/884255203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22b0100b10 0x7f22b006b2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 >> 172.21.15.143:0/884255203 conn(0x7f22b00fc2a0 msgr2=0x7f22b0105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:47.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 shutdown_connections 2024-09-10T10:44:47.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:47.071+0000 7f22b4b62640 1 -- 172.21.15.143:0/884255203 wait complete. 2024-09-10T10:44:47.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 -- 172.21.15.143:0/2835775557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 msgr2=0x7f3c1c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 --2- 172.21.15.143:0/2835775557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c103330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009930 tx=0x7f3c0c02a420 comp rx=0 tx=0).stop 2024-09-10T10:44:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 -- 172.21.15.143:0/2835775557 shutdown_connections 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 --2- 172.21.15.143:0/2835775557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c103870 0x7f3c1c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 --2- 172.21.15.143:0/2835775557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c103330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 --2- 172.21.15.143:0/2835775557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c0ffed0 0x7f3c1c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 -- 172.21.15.143:0/2835775557 >> 172.21.15.143:0/2835775557 conn(0x7f3c1c0fba40 msgr2=0x7f3c1c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 -- 172.21.15.143:0/2835775557 shutdown_connections 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.547+0000 7f3c2369c640 1 -- 172.21.15.143:0/2835775557 wait complete. 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 Processor -- start 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 -- start start 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c0ffed0 0x7f3c1c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c103870 0x7f3c1c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10d740 con 0x7f3c1c102eb0 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10d5c0 con 0x7f3c1c103870 2024-09-10T10:44:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c2369c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10d8c0 con 0x7f3c1c0ffed0 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43820/0 (socket says 172.21.15.143:43820) 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 -- 172.21.15.143:0/3954183777 learned_addr learned my addr 172.21.15.143:0/3954183777 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c21c39640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c103870 0x7f3c1c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c21438640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c0ffed0 0x7f3c1c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 -- 172.21.15.143:0/3954183777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c0ffed0 msgr2=0x7f3c1c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c0ffed0 0x7f3c1c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 -- 172.21.15.143:0/3954183777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c103870 msgr2=0x7f3c1c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c103870 0x7f3c1c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c20c37640 1 -- 172.21.15.143:0/3954183777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1a1d10 con 0x7f3c1c102eb0 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c21c39640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c103870 0x7f3c1c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:48.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.548+0000 7f3c21438640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c0ffed0 0x7f3c1c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.549+0000 7f3c20c37640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c19a670 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c004770 tx=0x7f3c0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.549+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c03f070 con 0x7f3c1c102eb0 2024-09-10T10:44:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.549+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c0c03b460 con 0x7f3c1c102eb0 2024-09-10T10:44:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.549+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c1a1fa0 con 0x7f3c1c102eb0 2024-09-10T10:44:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.549+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c035070 con 0x7f3c1c102eb0 2024-09-10T10:44:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.549+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1a2400 con 0x7f3c1c102eb0 2024-09-10T10:44:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.550+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c194530 con 0x7f3c1c102eb0 2024-09-10T10:44:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.551+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3c0c03b610 con 0x7f3c1c102eb0 2024-09-10T10:44:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.551+0000 7f3c127fc640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c080a2cc0 0x7f3c080a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.552+0000 7f3c21438640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c080a2cc0 0x7f3c080a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.552+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3c0c0efc00 con 0x7f3c1c102eb0 2024-09-10T10:44:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.552+0000 7f3c21438640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c080a2cc0 0x7f3c080a5180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06a590 tx=0x7f3c18005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.555+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c0c0bc6a0 con 0x7f3c1c102eb0 2024-09-10T10:44:48.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.732+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3c1c062630 con 0x7f3c080a2cc0 2024-09-10T10:44:48.733 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:48 smithi143 ceph-mon[31057]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:48.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:48 smithi143 ceph-mon[31057]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:48.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.745+0000 7f3c127fc640 1 -- 172.21.15.143:0/3954183777 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f3c1c062630 con 0x7f3c080a2cc0 2024-09-10T10:44:48.746 INFO:teuthology.orchestra.run.smithi143.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T10:44:48.746 INFO:teuthology.orchestra.run.smithi143.stdout:mds.a smithi143 running (2m) 57s ago 2m 15.6M - 19.1.1-305-g1cf819b7 41d671128357 0d481ce06357 2024-09-10T10:44:48.746 INFO:teuthology.orchestra.run.smithi143.stdout:mds.b smithi144 running (2m) 57s ago 2m 17.6M - 19.1.1-305-g1cf819b7 41d671128357 e2d6791d1098 2024-09-10T10:44:48.746 INFO:teuthology.orchestra.run.smithi143.stdout:mds.c smithi178 running (2m) 57s ago 2m 17.5M - 19.1.1-305-g1cf819b7 41d671128357 3bb6674ba0af 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.d smithi143 running (2m) 57s ago 2m 13.5M - 19.1.1-305-g1cf819b7 41d671128357 171dd23dfd04 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.e smithi144 running (2m) 57s ago 2m 19.1M - 19.1.1-305-g1cf819b7 41d671128357 a598d1eb540a 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.f smithi178 running (2m) 57s ago 2m 17.5M - 19.1.1-305-g1cf819b7 41d671128357 2380f1afd3af 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.g smithi143 running (2m) 57s ago 2m 17.3M - 19.1.1-305-g1cf819b7 41d671128357 b77e0014d299 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.h smithi144 running (2m) 57s ago 2m 17.3M - 19.1.1-305-g1cf819b7 41d671128357 556e5bc8207e 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.i smithi178 running (2m) 57s ago 2m 25.8M - 19.1.1-305-g1cf819b7 41d671128357 ac24ad84cf71 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.j smithi143 running (2m) 57s ago 2m 13.6M - 19.1.1-305-g1cf819b7 41d671128357 74278ecfee44 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.k smithi144 running (2m) 57s ago 2m 15.6M - 19.1.1-305-g1cf819b7 41d671128357 0ab487fa5156 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mds.l smithi178 running (2m) 57s ago 2m 18.1M - 19.1.1-305-g1cf819b7 41d671128357 8978b9fc4ab4 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mgr.x smithi143 *:9283,8765 running (9m) 57s ago 9m 509M - 19.1.1-305-g1cf819b7 41d671128357 9b44e769c0df 2024-09-10T10:44:48.747 INFO:teuthology.orchestra.run.smithi143.stdout:mgr.y smithi144 *:8443,8765 running (7m) 57s ago 7m 443M - 19.1.1-305-g1cf819b7 41d671128357 945d678f50e8 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:mgr.z smithi178 *:8443,8765 running (7m) 57s ago 7m 441M - 19.1.1-305-g1cf819b7 41d671128357 0a2a6923d0d5 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:mon.a smithi143 running (9m) 57s ago 9m 49.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 bd3317ce38ed 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:mon.b smithi144 running (8m) 57s ago 8m 39.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 5d8c788f62f0 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:mon.c smithi178 running (8m) 57s ago 8m 40.3M 2048M 19.1.1-305-g1cf819b7 41d671128357 4cf4bb79c3bb 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.0 smithi143 running (7m) 57s ago 7m 56.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 9870fca03ca1 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.1 smithi144 running (7m) 57s ago 7m 57.1M 4272M 19.1.1-305-g1cf819b7 41d671128357 6d21faab35c5 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.2 smithi178 running (6m) 57s ago 6m 55.9M 4271M 19.1.1-305-g1cf819b7 41d671128357 b779649f7e77 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.3 smithi143 running (6m) 57s ago 6m 47.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 6d51633059cf 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.4 smithi144 running (5m) 57s ago 5m 43.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 34e13b5574c3 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.5 smithi178 running (5m) 57s ago 5m 44.4M 4271M 19.1.1-305-g1cf819b7 41d671128357 55412e95697e 2024-09-10T10:44:48.748 INFO:teuthology.orchestra.run.smithi143.stdout:osd.6 smithi143 running (5m) 57s ago 5m 43.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 19e91a55d443 2024-09-10T10:44:48.749 INFO:teuthology.orchestra.run.smithi143.stdout:osd.7 smithi144 running (4m) 57s ago 4m 44.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 4d78244f427c 2024-09-10T10:44:48.749 INFO:teuthology.orchestra.run.smithi143.stdout:osd.8 smithi178 running (4m) 57s ago 4m 54.2M 4271M 19.1.1-305-g1cf819b7 41d671128357 4cbaa114029e 2024-09-10T10:44:48.749 INFO:teuthology.orchestra.run.smithi143.stdout:osd.9 smithi143 running (3m) 57s ago 3m 41.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 5645664645e5 2024-09-10T10:44:48.749 INFO:teuthology.orchestra.run.smithi143.stdout:osd.10 smithi144 running (3m) 57s ago 3m 46.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 b4e4235ed9b0 2024-09-10T10:44:48.749 INFO:teuthology.orchestra.run.smithi143.stdout:osd.11 smithi178 running (2m) 57s ago 2m 41.0M 4271M 19.1.1-305-g1cf819b7 41d671128357 387c29afc3c5 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c080a2cc0 msgr2=0x7f3c080a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c080a2cc0 0x7f3c080a5180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06a590 tx=0x7f3c18005e70 comp rx=0 tx=0).stop 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 msgr2=0x7f3c1c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c19a670 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c004770 tx=0x7f3c0c0042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 shutdown_connections 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c080a2cc0 0x7f3c080a5180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c1c103870 0x7f3c1c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1c102eb0 0x7f3c1c19a670 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 --2- 172.21.15.143:0/3954183777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c1c0ffed0 0x7f3c1c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 >> 172.21.15.143:0/3954183777 conn(0x7f3c1c0fba40 msgr2=0x7f3c1c1087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 shutdown_connections 2024-09-10T10:44:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:48.747+0000 7f3c2369c640 1 -- 172.21.15.143:0/3954183777 wait complete. 2024-09-10T10:44:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:48 smithi178 ceph-mon[34813]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:48.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:48 smithi178 ceph-mon[34813]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:48 smithi144 ceph-mon[34230]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:48 smithi144 ceph-mon[34230]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:49.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T10:44:49.853 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:50.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.355+0000 7f907657a640 1 -- 172.21.15.143:0/1452837608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070103870 msgr2=0x7f907010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.355+0000 7f907657a640 1 --2- 172.21.15.143:0/1452837608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070103870 0x7f907010a100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9068009cd0 tx=0x7f906802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.355+0000 7f907657a640 1 -- 172.21.15.143:0/1452837608 shutdown_connections 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.355+0000 7f907657a640 1 --2- 172.21.15.143:0/1452837608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070103870 0x7f907010a100 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.355+0000 7f907657a640 1 --2- 172.21.15.143:0/1452837608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9070102eb0 0x7f9070103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.355+0000 7f907657a640 1 --2- 172.21.15.143:0/1452837608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90700ffed0 0x7f90701002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.356+0000 7f907657a640 1 -- 172.21.15.143:0/1452837608 >> 172.21.15.143:0/1452837608 conn(0x7f90700fba40 msgr2=0x7f90700fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.356+0000 7f907657a640 1 -- 172.21.15.143:0/1452837608 shutdown_connections 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.356+0000 7f907657a640 1 -- 172.21.15.143:0/1452837608 wait complete. 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.356+0000 7f907657a640 1 Processor -- start 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.356+0000 7f907657a640 1 -- start start 2024-09-10T10:44:50.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90700ffed0 0x7f9070197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 0x7f90701983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9070103870 0x7f907019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f907010d760 con 0x7f9070102eb0 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f907010d5e0 con 0x7f90700ffed0 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f907010d8e0 con 0x7f9070103870 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 0x7f90701983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 0x7f90701983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43828/0 (socket says 172.21.15.143:43828) 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 -- 172.21.15.143:0/716933469 learned_addr learned my addr 172.21.15.143:0/716933469 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9074b17640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9070103870 0x7f907019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f906ffff640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90700ffed0 0x7f9070197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 -- 172.21.15.143:0/716933469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9070103870 msgr2=0x7f907019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9070103870 0x7f907019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 -- 172.21.15.143:0/716933469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90700ffed0 msgr2=0x7f9070197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90700ffed0 0x7f9070197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 -- 172.21.15.143:0/716933469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019fad0 con 0x7f9070102eb0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f9067fff640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 0x7f90701983e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f905800d770 tx=0x7f905800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058017070 con 0x7f9070102eb0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90580054d0 con 0x7f9070102eb0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058004cf0 con 0x7f9070102eb0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.357+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019fdc0 con 0x7f9070102eb0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.358+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a0220 con 0x7f9070102eb0 2024-09-10T10:44:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.359+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9070109140 con 0x7f9070102eb0 2024-09-10T10:44:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.360+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9058004e90 con 0x7f9070102eb0 2024-09-10T10:44:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.360+0000 7f906dffb640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f905c0a2c40 0x7f905c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.360+0000 7f906ffff640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f905c0a2c40 0x7f905c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.360+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f90580ceea0 con 0x7f9070102eb0 2024-09-10T10:44:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.361+0000 7f906ffff640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f905c0a2c40 0x7f905c0a5100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f907006a590 tx=0x7f906000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.365+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f905809b2f0 con 0x7f9070102eb0 2024-09-10T10:44:50.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.538+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9070062630 con 0x7f905c0a2c40 2024-09-10T10:44:50.540 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:50 smithi143 ceph-mon[31057]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:50.540 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:50 smithi143 ceph-mon[31057]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.544+0000 7f906dffb640 1 -- 172.21.15.143:0/716933469 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9070062630 con 0x7f905c0a2c40 2024-09-10T10:44:50.546 INFO:teuthology.orchestra.run.smithi143.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T10:44:50.546 INFO:teuthology.orchestra.run.smithi143.stdout:mds.all 12/12 59s ago 2m smithi143=a;smithi143=d;smithi143=g;smithi143=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-10T10:44:50.546 INFO:teuthology.orchestra.run.smithi143.stdout:mgr 3/3 59s ago 8m smithi143=x;smithi144=y;smithi178=z;count:3 2024-09-10T10:44:50.546 INFO:teuthology.orchestra.run.smithi143.stdout:mon 3/3 59s ago 8m smithi143:172.21.15.143=a;smithi144:172.21.15.144=b;smithi178:172.21.15.178=c;count:3 2024-09-10T10:44:50.546 INFO:teuthology.orchestra.run.smithi143.stdout:osd 12 59s ago - 2024-09-10T10:44:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.545+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f905c0a2c40 msgr2=0x7f905c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.545+0000 7f907657a640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f905c0a2c40 0x7f905c0a5100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f907006a590 tx=0x7f906000b390 comp rx=0 tx=0).stop 2024-09-10T10:44:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.545+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 msgr2=0x7f90701983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 0x7f90701983e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f905800d770 tx=0x7f905800dc30 comp rx=0 tx=0).stop 2024-09-10T10:44:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 shutdown_connections 2024-09-10T10:44:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f905c0a2c40 0x7f905c0a5100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9070103870 0x7f907019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9070102eb0 0x7f90701983e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 --2- 172.21.15.143:0/716933469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90700ffed0 0x7f9070197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 >> 172.21.15.143:0/716933469 conn(0x7f90700fba40 msgr2=0x7f9070106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 shutdown_connections 2024-09-10T10:44:50.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:50.546+0000 7f907657a640 1 -- 172.21.15.143:0/716933469 wait complete. 2024-09-10T10:44:50.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:50 smithi178 ceph-mon[34813]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:50.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:50 smithi178 ceph-mon[34813]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:50 smithi144 ceph-mon[34230]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:50 smithi144 ceph-mon[34230]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:44:51.413 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T10:44:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:51.790 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:51 smithi143 ceph-mon[31057]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:51.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:51 smithi178 ceph-mon[34813]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:51.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:51 smithi144 ceph-mon[34230]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:52.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 -- 172.21.15.143:0/81900393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940102690 msgr2=0x7f9940108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 --2- 172.21.15.143:0/81900393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940102690 0x7f9940108f20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9938009ca0 tx=0x7f993802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 -- 172.21.15.143:0/81900393 shutdown_connections 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 --2- 172.21.15.143:0/81900393 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940102690 0x7f9940108f20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 --2- 172.21.15.143:0/81900393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9940101cd0 0x7f9940102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 --2- 172.21.15.143:0/81900393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940100ad0 0x7f9940100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 -- 172.21.15.143:0/81900393 >> 172.21.15.143:0/81900393 conn(0x7f99400fc280 msgr2=0x7f99400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 -- 172.21.15.143:0/81900393 shutdown_connections 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.211+0000 7f9947526640 1 -- 172.21.15.143:0/81900393 wait complete. 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 Processor -- start 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 -- start start 2024-09-10T10:44:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 0x7f9940197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9940101cd0 0x7f9940198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940102690 0x7f994019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f994010c600 con 0x7f9940100ad0 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f994010c480 con 0x7f9940101cd0 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9947526640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f994010c780 con 0x7f9940102690 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f99452c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 0x7f9940197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f99452c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 0x7f9940197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43848/0 (socket says 172.21.15.143:43848) 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f99452c2640 1 -- 172.21.15.143:0/134514594 learned_addr learned my addr 172.21.15.143:0/134514594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9944ac1640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9940101cd0 0x7f9940198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.212+0000 7f9945ac3640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940102690 0x7f994019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99452c2640 1 -- 172.21.15.143:0/134514594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940102690 msgr2=0x7f994019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99452c2640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940102690 0x7f994019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99452c2640 1 -- 172.21.15.143:0/134514594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9940101cd0 msgr2=0x7f9940198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99452c2640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9940101cd0 0x7f9940198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99452c2640 1 -- 172.21.15.143:0/134514594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994019fa00 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f9945ac3640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940102690 0x7f994019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99452c2640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 0x7f9940197dd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d950 tx=0x7f993c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c017070 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994019fcf0 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f993c009c30 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c00f620 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.213+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a0180 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.214+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99401a0350 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.215+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f993c01a070 con 0x7f9940100ad0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.216+0000 7f99367fc640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f992c0a2c40 0x7f992c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.216+0000 7f9944ac1640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f992c0a2c40 0x7f992c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.216+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f993c0cef60 con 0x7f9940100ad0 2024-09-10T10:44:52.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.216+0000 7f9944ac1640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f992c0a2c40 0x7f992c0a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f99401992f0 tx=0x7f993000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.220+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f993c09ba00 con 0x7f9940100ad0 2024-09-10T10:44:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.407+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f994006a730 con 0x7f992c0a2c40 2024-09-10T10:44:52.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.409+0000 7f99367fc640 1 -- 172.21.15.143:0/134514594 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f994006a730 con 0x7f992c0a2c40 2024-09-10T10:44:52.411 INFO:teuthology.orchestra.run.smithi143.stdout:HOST ADDR LABELS STATUS 2024-09-10T10:44:52.411 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143 172.21.15.143 2024-09-10T10:44:52.411 INFO:teuthology.orchestra.run.smithi143.stdout:smithi144 172.21.15.144 2024-09-10T10:44:52.411 INFO:teuthology.orchestra.run.smithi143.stdout:smithi178 172.21.15.178 2024-09-10T10:44:52.412 INFO:teuthology.orchestra.run.smithi143.stdout:3 hosts in cluster 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f992c0a2c40 msgr2=0x7f992c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f992c0a2c40 0x7f992c0a5100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f99401992f0 tx=0x7f993000b410 comp rx=0 tx=0).stop 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 msgr2=0x7f9940197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 0x7f9940197dd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d950 tx=0x7f993c00de10 comp rx=0 tx=0).stop 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 shutdown_connections 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f992c0a2c40 0x7f992c0a5100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940102690 0x7f994019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9940101cd0 0x7f9940198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 --2- 172.21.15.143:0/134514594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9940100ad0 0x7f9940197dd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 >> 172.21.15.143:0/134514594 conn(0x7f99400fc280 msgr2=0x7f994006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 shutdown_connections 2024-09-10T10:44:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:52.412+0000 7f9947526640 1 -- 172.21.15.143:0/134514594 wait complete. 2024-09-10T10:44:52.791 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:52 smithi144 ceph-mon[34230]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:52.791 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:52 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:52.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:52 smithi143 ceph-mon[31057]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:52.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:52 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:52.815 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:52 smithi178 ceph-mon[34813]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:52.816 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:52 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:53.156 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T10:44:53.708 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:53 smithi143 ceph-mon[31057]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:53.708 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.709 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.709 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.709 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:53 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:53 smithi178 ceph-mon[34813]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:53.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:53 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:53 smithi144 ceph-mon[34230]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:53 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:54.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 -- 172.21.15.143:0/3463961177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314101c00 msgr2=0x7f7314102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 --2- 172.21.15.143:0/3463961177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314101c00 0x7f7314102000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7310009ca0 tx=0x7f731002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 -- 172.21.15.143:0/3463961177 shutdown_connections 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 --2- 172.21.15.143:0/3463961177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73141037c0 0x7f731410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 --2- 172.21.15.143:0/3463961177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7314102e00 0x7f7314103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 --2- 172.21.15.143:0/3463961177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314101c00 0x7f7314102000 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 -- 172.21.15.143:0/3463961177 >> 172.21.15.143:0/3463961177 conn(0x7f73140fd390 msgr2=0x7f73140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 -- 172.21.15.143:0/3463961177 shutdown_connections 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.401+0000 7f731b67f640 1 -- 172.21.15.143:0/3463961177 wait complete. 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 Processor -- start 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 -- start start 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314101c00 0x7f7314075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 0x7f73140745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73141037c0 0x7f7314074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f731410d730 con 0x7f7314102e00 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f731410d5b0 con 0x7f7314101c00 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731b67f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f731410d8b0 con 0x7f73141037c0 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7318c1a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 0x7f73140745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7318c1a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 0x7f73140745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43860/0 (socket says 172.21.15.143:43860) 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7318c1a640 1 -- 172.21.15.143:0/2963719160 learned_addr learned my addr 172.21.15.143:0/2963719160 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7319c1c640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73141037c0 0x7f7314074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f731941b640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314101c00 0x7f7314075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7318c1a640 1 -- 172.21.15.143:0/2963719160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73141037c0 msgr2=0x7f7314074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7318c1a640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73141037c0 0x7f7314074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.402+0000 7f7318c1a640 1 -- 172.21.15.143:0/2963719160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314101c00 msgr2=0x7f7314075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f7318c1a640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314101c00 0x7f7314075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f7318c1a640 1 -- 172.21.15.143:0/2963719160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73140753e0 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f7318c1a640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 0x7f73140745e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f730800d770 tx=0x7f730800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308017070 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73080054d0 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73141a1fb0 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004cf0 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.403+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141a2410 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.404+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73141090a0 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.405+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7308004e90 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.406+0000 7f73067fc640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f73000a2c90 0x7f73000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.406+0000 7f731941b640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f73000a2c90 0x7f73000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.406+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f73080cf390 con 0x7f7314102e00 2024-09-10T10:44:54.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.406+0000 7f731941b640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f73000a2c90 0x7f73000a5150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7310006010 tx=0x7f7310000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.409+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f730809beb0 con 0x7f7314102e00 2024-09-10T10:44:54.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.614+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7314062630 con 0x7f73000a2c90 2024-09-10T10:44:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.615+0000 7f73067fc640 1 -- 172.21.15.143:0/2963719160 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7314062630 con 0x7f73000a2c90 2024-09-10T10:44:54.618 INFO:teuthology.orchestra.run.smithi143.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T10:44:54.618 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330073400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T10:44:54.618 INFO:teuthology.orchestra.run.smithi143.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T10:44:54.618 INFO:teuthology.orchestra.run.smithi143.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-10T10:44:54.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f73000a2c90 msgr2=0x7f73000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f73000a2c90 0x7f73000a5150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7310006010 tx=0x7f7310000f40 comp rx=0 tx=0).stop 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 msgr2=0x7f73140745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 0x7f73140745e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f730800d770 tx=0x7f730800dc30 comp rx=0 tx=0).stop 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 shutdown_connections 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f73000a2c90 0x7f73000a5150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73141037c0 0x7f7314074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7314102e00 0x7f73140745e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 --2- 172.21.15.143:0/2963719160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7314101c00 0x7f7314075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 >> 172.21.15.143:0/2963719160 conn(0x7f73140fd390 msgr2=0x7f7314106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 shutdown_connections 2024-09-10T10:44:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:54.618+0000 7f731b67f640 1 -- 172.21.15.143:0/2963719160 wait complete. 2024-09-10T10:44:54.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:54 smithi143 ceph-mon[31057]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:44:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:54 smithi178 ceph-mon[34813]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:44:54.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:54 smithi144 ceph-mon[34230]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T10:44:55.321 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T10:44:55.330 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T10:44:55.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T10:44:55.602 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 -- 172.21.15.143:0/2565776651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 msgr2=0x7f85681018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 --2- 172.21.15.143:0/2565776651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681018f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f855800aa10 tx=0x7f855802b720 comp rx=0 tx=0).stop 2024-09-10T10:44:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 -- 172.21.15.143:0/2565776651 shutdown_connections 2024-09-10T10:44:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 --2- 172.21.15.143:0/2565776651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568101e30 0x7f85681086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 --2- 172.21.15.143:0/2565776651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681018f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 --2- 172.21.15.143:0/2565776651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568100270 0x7f8568100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 -- 172.21.15.143:0/2565776651 >> 172.21.15.143:0/2565776651 conn(0x7f85680fba40 msgr2=0x7f85680fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 -- 172.21.15.143:0/2565776651 shutdown_connections 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.068+0000 7f856d6c7640 1 -- 172.21.15.143:0/2565776651 wait complete. 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 Processor -- start 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 -- start start 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568100270 0x7f8568197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568101e30 0x7f856819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f856810b620 con 0x7f8568101470 2024-09-10T10:44:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f856810b4a0 con 0x7f8568100270 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f856d6c7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f856810b7a0 con 0x7f8568101e30 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43890/0 (socket says 172.21.15.143:43890) 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 -- 172.21.15.143:0/3704985874 learned_addr learned my addr 172.21.15.143:0/3704985874 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f8566ffd640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568100270 0x7f8568197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 -- 172.21.15.143:0/3704985874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568101e30 msgr2=0x7f856819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85677fe640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568101e30 0x7f856819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568101e30 0x7f856819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 -- 172.21.15.143:0/3704985874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568100270 msgr2=0x7f8568197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568100270 0x7f8568197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85667fc640 1 -- 172.21.15.143:0/3704985874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856819fbd0 con 0x7f8568101470 2024-09-10T10:44:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f8566ffd640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568100270 0x7f8568197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.069+0000 7f85677fe640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568101e30 0x7f856819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.070+0000 7f85667fc640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681984e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8558005ec0 tx=0x7f8558004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.070+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855803f070 con 0x7f8568101470 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.070+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856819fe60 con 0x7f8568101470 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.070+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8558032400 con 0x7f8568101470 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.070+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558036040 con 0x7f8568101470 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.070+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a0340 con 0x7f8568101470 2024-09-10T10:44:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.071+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85681077b0 con 0x7f8568101470 2024-09-10T10:44:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.072+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8558042070 con 0x7f8568101470 2024-09-10T10:44:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.073+0000 7f854ffff640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f85500a2c40 0x7f85500a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.073+0000 7f8566ffd640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f85500a2c40 0x7f85500a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.073+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85580f0520 con 0x7f8568101470 2024-09-10T10:44:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.073+0000 7f8566ffd640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f85500a2c40 0x7f85500a5100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f855c006fd0 tx=0x7f855c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.076+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85580bcfc0 con 0x7f8568101470 2024-09-10T10:44:56.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.287+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f856802d990 con 0x7f8568101470 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.287+0000 7f854ffff640 1 -- 172.21.15.143:0/3704985874 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3146 (secure 0 0 0) 0x7f85580bc8c0 con 0x7f8568101470 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stdout:e28 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stdout:btime 2024-09-10T10:43:32:550770+0000 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stdout:legacy client fscid: 1 2024-09-10T10:44:56.290 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:Filesystem 'cephfs' (1) 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:fs_name cephfs 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 28 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:created 2024-09-10T10:42:53.956292+0000 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:modified 2024-09-10T10:43:32.550735+0000 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:tableserver 0 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:root 0 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:session_timeout 300 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:session_autoclose 300 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:max_file_size 1099511627776 2024-09-10T10:44:56.291 INFO:teuthology.orchestra.run.smithi143.stdout:max_xattr_size 65536 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:required_client_features {} 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:last_failure 0 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:last_failure_osd_epoch 0 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:max_mds 5 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:in 0,1,2,3,4 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:up {0=24502,1=24485,2=24481,3=24475,4=24458} 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:failed 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:damaged 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:stopped 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:data_pools [3,4] 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:metadata_pool 2 2024-09-10T10:44:56.292 INFO:teuthology.orchestra.run.smithi143.stdout:inline_data disabled 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:balancer 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:bal_rank_mask -1 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:standby_count_wanted 0 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:qdb_cluster leader: 24502 members: 24502,24485,24481,24475,24458 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.i{0:24502} state up:active seq 7 addr [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.e{0:24434} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/3190468681,v1:172.21.15.144:6835/3190468681] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b{1:24485} state up:active seq 13 addr [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.l{1:24433} state up:standby-replay seq 1 addr [v2:172.21.15.178:6832/1016684638,v1:172.21.15.178:6833/1016684638] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c{2:24481} state up:active seq 17 addr [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.k{2:24407} state up:standby-replay seq 1 addr [v2:172.21.15.144:6832/3121359191,v1:172.21.15.144:6833/3121359191] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.f{3:24475} state up:active seq 18 addr [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a{3:14595} state up:standby-replay seq 1 addr [v2:172.21.15.143:6840/1083221625,v1:172.21.15.143:6841/1083221625] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.293 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.h{4:24458} state up:active seq 17 addr [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.g{4:14589} state up:standby-replay seq 1 addr [v2:172.21.15.143:6838/2518540467,v1:172.21.15.143:6839/2518540467] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout:Standby daemons: 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.143:6834/3176590385,v1:172.21.15.143:6835/3176590385] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.294 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.143:6836/2245538131,v1:172.21.15.143:6837/2245538131] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f85500a2c40 msgr2=0x7f85500a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f85500a2c40 0x7f85500a5100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f855c006fd0 tx=0x7f855c008040 comp rx=0 tx=0).stop 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 msgr2=0x7f85681984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681984e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8558005ec0 tx=0x7f8558004430 comp rx=0 tx=0).stop 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 shutdown_connections 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f85500a2c40 0x7f85500a5100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8568101e30 0x7f856819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8568101470 0x7f85681984e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 --2- 172.21.15.143:0/3704985874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568100270 0x7f8568197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 >> 172.21.15.143:0/3704985874 conn(0x7f85680fba40 msgr2=0x7f8568104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 shutdown_connections 2024-09-10T10:44:56.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:56.290+0000 7f856d6c7640 1 -- 172.21.15.143:0/3704985874 wait complete. 2024-09-10T10:44:56.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:56 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3704985874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:44:56.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:56 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3704985874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:44:56.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:44:56.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:56 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3704985874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:44:57.008 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T10:44:57.254 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:57.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2772914523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014101c00 msgr2=0x7fe014102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2772914523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014101c00 0x7fe014102000 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe004009900 tx=0x7fe00402a410 comp rx=0 tx=0).stop 2024-09-10T10:44:57.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2772914523 shutdown_connections 2024-09-10T10:44:57.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2772914523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0141037c0 0x7fe01410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2772914523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe014102e00 0x7fe014103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2772914523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014101c00 0x7fe014102000 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2772914523 >> 172.21.15.143:0/2772914523 conn(0x7fe0140fd390 msgr2=0x7fe0140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2772914523 shutdown_connections 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.763+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2772914523 wait complete. 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 Processor -- start 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 -- start start 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe014101c00 0x7fe014075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 0x7fe0140745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0141037c0 0x7fe014074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe01410d790 con 0x7fe014102e00 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe01410d610 con 0x7fe0141037c0 2024-09-10T10:44:57.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01c7ea640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe01410d910 con 0x7fe014101c00 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 0x7fe0140745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 0x7fe0140745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52640/0 (socket says 172.21.15.143:52640) 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 -- 172.21.15.143:0/2800073675 learned_addr learned my addr 172.21.15.143:0/2800073675 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01ad87640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0141037c0 0x7fe014074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe01a586640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe014101c00 0x7fe014075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 -- 172.21.15.143:0/2800073675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe014101c00 msgr2=0x7fe014075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe014101c00 0x7fe014075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 -- 172.21.15.143:0/2800073675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0141037c0 msgr2=0x7fe014074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0141037c0 0x7fe014074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 -- 172.21.15.143:0/2800073675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0140753e0 con 0x7fe014102e00 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.764+0000 7fe019d85640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 0x7fe0140745e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c890 tx=0x7fe01000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.765+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010004220 con 0x7fe014102e00 2024-09-10T10:44:57.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.765+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0100043c0 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.765+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0141a1fc0 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.765+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010004a00 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.765+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a24f0 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.766+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe014109090 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.767+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe010004ba0 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.767+0000 7fe00b7fe640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe0000a2c10 0x7fe0000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.767+0000 7fe01a586640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe0000a2c10 0x7fe0000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.767+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe0100cf520 con 0x7fe014102e00 2024-09-10T10:44:57.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.768+0000 7fe01a586640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe0000a2c10 0x7fe0000a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe004006010 tx=0x7fe004000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:57.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.770+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe010014030 con 0x7fe014102e00 2024-09-10T10:44:57.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.940+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe01402d9c0 con 0x7fe014102e00 2024-09-10T10:44:57.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.941+0000 7fe00b7fe640 1 -- 172.21.15.143:0/2800073675 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5638 (secure 0 0 0) 0x7fe01009b970 con 0x7fe014102e00 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 89 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:fsid 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:created 2024-09-10T10:35:07.959463+0000 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:modified 2024-09-10T10:44:00.496675+0000 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:crush_version 27 2024-09-10T10:44:57.943 INFO:teuthology.orchestra.run.smithi143.stdout:full_ratio 0.95 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:backfillfull_ratio 0.9 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:nearfull_ratio 0.85 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:require_min_compat_client luminous 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:min_compat_client luminous 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:require_osd_release squid 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:stretch_mode_enabled false 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:max_osd 12 2024-09-10T10:44:57.944 INFO:teuthology.orchestra.run.smithi143.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6802/3953963737,v1:172.21.15.143:6803/3953963737] [v2:172.21.15.143:6804/3953963737,v1:172.21.15.143:6805/3953963737] exists,up b681e90c-4078-4a6f-8d49-607d1b7505b1 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/3635948930,v1:172.21.15.144:6801/3635948930] [v2:172.21.15.144:6802/3635948930,v1:172.21.15.144:6803/3635948930] exists,up d1c81ecf-0a81-4ef2-8abd-cc38f88f5482 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/3522332994,v1:172.21.15.178:6801/3522332994] [v2:172.21.15.178:6802/3522332994,v1:172.21.15.178:6803/3522332994] exists,up b5d7e27a-4fb6-4951-86f4-b0f40050cdd2 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.3 up in weight 1 up_from 24 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6810/3687852925,v1:172.21.15.143:6811/3687852925] [v2:172.21.15.143:6812/3687852925,v1:172.21.15.143:6813/3687852925] exists,up 4a60f699-6beb-43e3-a94b-e5544e39fec5 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.4 up in weight 1 up_from 32 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/2906236694,v1:172.21.15.144:6809/2906236694] [v2:172.21.15.144:6810/2906236694,v1:172.21.15.144:6811/2906236694] exists,up d383b3ca-df71-4034-ac1f-bcb431e38265 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.5 up in weight 1 up_from 37 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/1007276569,v1:172.21.15.178:6809/1007276569] [v2:172.21.15.178:6810/1007276569,v1:172.21.15.178:6811/1007276569] exists,up bb5db187-d7f1-4f74-a4ec-ff8e4e686270 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.6 up in weight 1 up_from 44 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6818/148519296,v1:172.21.15.143:6819/148519296] [v2:172.21.15.143:6820/148519296,v1:172.21.15.143:6821/148519296] exists,up 3b57aa64-3a10-4155-a2c6-984a15789ab4 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.7 up in weight 1 up_from 49 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/3746149853,v1:172.21.15.144:6817/3746149853] [v2:172.21.15.144:6818/3746149853,v1:172.21.15.144:6819/3746149853] exists,up 680d5255-bade-46c7-a4be-00bfb073eafb 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.8 up in weight 1 up_from 56 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/3485367257,v1:172.21.15.178:6817/3485367257] [v2:172.21.15.178:6818/3485367257,v1:172.21.15.178:6819/3485367257] exists,up f707e74d-dfb8-4966-802f-5df28f3ac0e9 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.9 up in weight 1 up_from 63 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6826/4014720297,v1:172.21.15.143:6827/4014720297] [v2:172.21.15.143:6828/4014720297,v1:172.21.15.143:6829/4014720297] exists,up 3005ede3-bfda-4cfd-a562-8c1540c829af 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.10 up in weight 1 up_from 68 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/2444988938,v1:172.21.15.144:6825/2444988938] [v2:172.21.15.144:6826/2444988938,v1:172.21.15.144:6827/2444988938] exists,up 1f83408d-f6e1-4760-8b33-81493fb716f9 2024-09-10T10:44:57.945 INFO:teuthology.orchestra.run.smithi143.stdout:osd.11 up in weight 1 up_from 75 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/2309120918,v1:172.21.15.178:6825/2309120918] [v2:172.21.15.178:6826/2309120918,v1:172.21.15.178:6827/2309120918] exists,up 38c6f977-35b1-4373-95af-491a20581f8f 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 2.b [8,11] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 2.13 [2,11] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 2.14 [2,4] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 3.2 [3,9] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 3.13 [4,9] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 3.34 [4,9] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 4.c [9,10] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 4.16 [9,8] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 4.1b [5,8] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:pg_upmap_items 4.1f [9,8] 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:0/3629809110 expires 2024-09-11T10:35:59.338289+0000 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:0/212886119 expires 2024-09-11T10:35:59.338289+0000 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:6801/1050213403 expires 2024-09-11T10:35:59.338289+0000 2024-09-10T10:44:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:0/3153560299 expires 2024-09-11T10:35:32.440973+0000 2024-09-10T10:44:57.947 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:6800/1050213403 expires 2024-09-11T10:35:59.338289+0000 2024-09-10T10:44:57.947 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:6800/3171173333 expires 2024-09-11T10:35:32.440973+0000 2024-09-10T10:44:57.947 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:6801/3171173333 expires 2024-09-11T10:35:32.440973+0000 2024-09-10T10:44:57.947 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:0/1093984961 expires 2024-09-11T10:35:32.440973+0000 2024-09-10T10:44:57.947 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:0/2707450669 expires 2024-09-11T10:35:59.338289+0000 2024-09-10T10:44:57.947 INFO:teuthology.orchestra.run.smithi143.stdout:blocklist 172.21.15.143:0/3503275671 expires 2024-09-11T10:35:32.440973+0000 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe0000a2c10 msgr2=0x7fe0000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe0000a2c10 0x7fe0000a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe004006010 tx=0x7fe004000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 msgr2=0x7fe0140745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 0x7fe0140745e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c890 tx=0x7fe01000cd50 comp rx=0 tx=0).stop 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 shutdown_connections 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe0000a2c10 0x7fe0000a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0141037c0 0x7fe014074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe014102e00 0x7fe0140745e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 --2- 172.21.15.143:0/2800073675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe014101c00 0x7fe014075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 >> 172.21.15.143:0/2800073675 conn(0x7fe0140fd390 msgr2=0x7fe014106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 shutdown_connections 2024-09-10T10:44:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:57.943+0000 7fe01c7ea640 1 -- 172.21.15.143:0/2800073675 wait complete. 2024-09-10T10:44:58.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:58 smithi143 ceph-mon[31057]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:58.740 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:44:58 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2800073675' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:44:58.774 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T10:44:58.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:44:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:58 smithi178 ceph-mon[34813]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:44:58 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2800073675' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:44:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:58 smithi144 ceph-mon[34230]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:44:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:44:58 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2800073675' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:44:59.038 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:59.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 -- 172.21.15.143:0/2866964171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981082a0 msgr2=0x7f729810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 --2- 172.21.15.143:0/2866964171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981082a0 0x7f729810ec40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7290009cd0 tx=0x7f729002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 -- 172.21.15.143:0/2866964171 shutdown_connections 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 --2- 172.21.15.143:0/2866964171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981082a0 0x7f729810ec40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 --2- 172.21.15.143:0/2866964171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981036e0 0x7f7298107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 --2- 172.21.15.143:0/2866964171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 0x7f7298103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 -- 172.21.15.143:0/2866964171 >> 172.21.15.143:0/2866964171 conn(0x7f72980fc7d0 msgr2=0x7f72980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 -- 172.21.15.143:0/2866964171 shutdown_connections 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 -- 172.21.15.143:0/2866964171 wait complete. 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.527+0000 7f729ef12640 1 Processor -- start 2024-09-10T10:44:59.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 -- start start 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 0x7f7298100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 0x7f7298100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981082a0 0x7f7298101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7298111f50 con 0x7f72981036e0 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7298111dd0 con 0x7f72981082a0 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ef12640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72981120d0 con 0x7f7298102d30 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 0x7f7298100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 0x7f7298100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52658/0 (socket says 172.21.15.143:52658) 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 -- 172.21.15.143:0/1752328412 learned_addr learned my addr 172.21.15.143:0/1752328412 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729d4af640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981082a0 0x7f7298101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ccae640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 0x7f7298100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 -- 172.21.15.143:0/1752328412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 msgr2=0x7f7298100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 0x7f7298100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 -- 172.21.15.143:0/1752328412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981082a0 msgr2=0x7f7298101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981082a0 0x7f7298101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f728ffff640 1 -- 172.21.15.143:0/1752328412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72981a6930 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729d4af640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981082a0 0x7f7298101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.528+0000 7f729ccae640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 0x7f7298100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.529+0000 7f728ffff640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 0x7f7298100ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f728800d950 tx=0x7f728800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.529+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288017070 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.529+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7288009c30 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.529+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72981a6c20 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.529+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72880047a0 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.529+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981a7160 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.530+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298103110 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.531+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7288004020 con 0x7f72981036e0 2024-09-10T10:44:59.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.532+0000 7f728dffb640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72800a2d60 0x7f72800a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.532+0000 7f729ccae640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72800a2d60 0x7f72800a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.532+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f72880cf4a0 con 0x7f72981036e0 2024-09-10T10:44:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.532+0000 7f729ccae640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72800a2d60 0x7f72800a5220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f72940058c0 tx=0x7f7294005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.539+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7288020020 con 0x7f72981036e0 2024-09-10T10:44:59.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.747+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f729810a2b0 con 0x7f72981036e0 2024-09-10T10:44:59.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.748+0000 7f728dffb640 1 -- 172.21.15.143:0/1752328412 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f728809c040 con 0x7f72981036e0 2024-09-10T10:44:59.749 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:44:59.749 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"btime":"2024-09-10T10:43:32:550770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:32.550735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:44:59.749 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:44:59.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72800a2d60 msgr2=0x7f72800a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72800a2d60 0x7f72800a5220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f72940058c0 tx=0x7f7294005850 comp rx=0 tx=0).stop 2024-09-10T10:44:59.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 msgr2=0x7f7298100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 0x7f7298100ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f728800d950 tx=0x7f728800de10 comp rx=0 tx=0).stop 2024-09-10T10:44:59.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 shutdown_connections 2024-09-10T10:44:59.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72800a2d60 0x7f72800a5220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72981082a0 0x7f7298101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f72981036e0 0x7f7298100ed0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 --2- 172.21.15.143:0/1752328412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7298102d30 0x7f7298100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 >> 172.21.15.143:0/1752328412 conn(0x7f72980fc7d0 msgr2=0x7f7298105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 shutdown_connections 2024-09-10T10:44:59.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:44:59.750+0000 7f729ef12640 1 -- 172.21.15.143:0/1752328412 wait complete. 2024-09-10T10:45:00.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:00.733 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:00.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:00 smithi143 ceph-mon[31057]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:45:00.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:00 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1752328412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:00 smithi178 ceph-mon[34813]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:45:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:00 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1752328412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:00 smithi144 ceph-mon[34230]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:45:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:00 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1752328412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 -- 172.21.15.143:0/1544000481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b41082a0 msgr2=0x7f84b410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 --2- 172.21.15.143:0/1544000481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b41082a0 0x7f84b410ec40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009cd0 tx=0x7f84ac02a910 comp rx=0 tx=0).stop 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 -- 172.21.15.143:0/1544000481 shutdown_connections 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 --2- 172.21.15.143:0/1544000481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b41082a0 0x7f84b410ec40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 --2- 172.21.15.143:0/1544000481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b41036e0 0x7f84b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 --2- 172.21.15.143:0/1544000481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b4102d30 0x7f84b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 -- 172.21.15.143:0/1544000481 >> 172.21.15.143:0/1544000481 conn(0x7f84b40fc7d0 msgr2=0x7f84b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 -- 172.21.15.143:0/1544000481 shutdown_connections 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 -- 172.21.15.143:0/1544000481 wait complete. 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 Processor -- start 2024-09-10T10:45:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.283+0000 7f84bb228640 1 -- start start 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 0x7f84b4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b41036e0 0x7f84b4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b41082a0 0x7f84b4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112010 con 0x7f84b4102d30 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4111e90 con 0x7f84b41082a0 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4112190 con 0x7f84b41036e0 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 0x7f84b4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 0x7f84b4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52688/0 (socket says 172.21.15.143:52688) 2024-09-10T10:45:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 -- 172.21.15.143:0/1068927385 learned_addr learned my addr 172.21.15.143:0/1068927385 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b97c5640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b41082a0 0x7f84b4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b3fff640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b41036e0 0x7f84b4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 -- 172.21.15.143:0/1068927385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b41036e0 msgr2=0x7f84b4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b41036e0 0x7f84b4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 -- 172.21.15.143:0/1068927385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b41082a0 msgr2=0x7f84b4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b41082a0 0x7f84b4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 -- 172.21.15.143:0/1068927385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b41a6a90 con 0x7f84b4102d30 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b97c5640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b41082a0 0x7f84b4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b8fc4640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 0x7f84b4100990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f84a400b4a0 tx=0x7f84a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4014070 con 0x7f84b4102d30 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a4010ac0 con 0x7f84b4102d30 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41a6d80 con 0x7f84b4102d30 2024-09-10T10:45:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.284+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4017c30 con 0x7f84b4102d30 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.285+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a7240 con 0x7f84b4102d30 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.286+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b406a850 con 0x7f84b4102d30 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.287+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f84a4017070 con 0x7f84b4102d30 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.287+0000 7f84b1ffb640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f849c0a2c00 0x7f849c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.288+0000 7f84b3fff640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f849c0a2c00 0x7f849c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.288+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f84a40cea80 con 0x7f84b4102d30 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.288+0000 7f84b3fff640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f849c0a2c00 0x7f849c0a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84b4102420 tx=0x7f84a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.292+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84a409b680 con 0x7f84b4102d30 2024-09-10T10:45:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.494+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84b410a2b0 con 0x7f84b4102d30 2024-09-10T10:45:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.494+0000 7f84b1ffb640 1 -- 172.21.15.143:0/1068927385 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11998 (secure 0 0 0) 0x7f84a409af80 con 0x7f84b4102d30 2024-09-10T10:45:01.496 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:01.496 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":28,"btime":"2024-09-10T10:43:32:550770+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:43:32.550735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:45:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 28 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f849c0a2c00 msgr2=0x7f849c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f849c0a2c00 0x7f849c0a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84b4102420 tx=0x7f84a8008040 comp rx=0 tx=0).stop 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 msgr2=0x7f84b4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 0x7f84b4100990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f84a400b4a0 tx=0x7f84a400b960 comp rx=0 tx=0).stop 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 shutdown_connections 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f849c0a2c00 0x7f849c0a50c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.496+0000 7f84bb228640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b41082a0 0x7f84b4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.497+0000 7f84bb228640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b41036e0 0x7f84b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.497+0000 7f84bb228640 1 --2- 172.21.15.143:0/1068927385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84b4102d30 0x7f84b4100990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.497+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 >> 172.21.15.143:0/1068927385 conn(0x7f84b40fc7d0 msgr2=0x7f84b4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.497+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 shutdown_connections 2024-09-10T10:45:01.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:01.497+0000 7f84bb228640 1 -- 172.21.15.143:0/1068927385 wait complete. 2024-09-10T10:45:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:01 smithi143 ceph-mon[31057]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T10:45:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:01 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1068927385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:01.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:01 smithi178 ceph-mon[34813]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T10:45:01.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:01 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1068927385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:01 smithi144 ceph-mon[34230]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T10:45:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:01 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1068927385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:02.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:45:02.485 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:02.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:02 smithi143 ceph-mon[31057]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T10:45:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:02 smithi178 ceph-mon[34813]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T10:45:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:02 smithi144 ceph-mon[34230]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T10:45:03.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 -- 172.21.15.143:0/1837744058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408101de0 msgr2=0x7fc4081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 --2- 172.21.15.143:0/1837744058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408101de0 0x7fc4081021c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009900 tx=0x7fc3f802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:03.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 -- 172.21.15.143:0/1837744058 shutdown_connections 2024-09-10T10:45:03.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 --2- 172.21.15.143:0/1837744058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc408106f10 0x7fc40810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 --2- 172.21.15.143:0/1837744058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408102790 0x7fc4081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 --2- 172.21.15.143:0/1837744058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408101de0 0x7fc4081021c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 -- 172.21.15.143:0/1837744058 >> 172.21.15.143:0/1837744058 conn(0x7fc4080fd8c0 msgr2=0x7fc4080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 -- 172.21.15.143:0/1837744058 shutdown_connections 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.987+0000 7fc40db9d640 1 -- 172.21.15.143:0/1837744058 wait complete. 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 Processor -- start 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 -- start start 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408101de0 0x7fc4081982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 0x7fc408198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc408106f10 0x7fc40819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc408111e20 con 0x7fc408102790 2024-09-10T10:45:03.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc408111ca0 con 0x7fc408101de0 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc40db9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc408111fa0 con 0x7fc408106f10 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 0x7fc408198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 0x7fc408198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52704/0 (socket says 172.21.15.143:52704) 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 -- 172.21.15.143:0/3319767460 learned_addr learned my addr 172.21.15.143:0/3319767460 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc407fff640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc408106f10 0x7fc40819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc4077fe640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408101de0 0x7fc4081982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 -- 172.21.15.143:0/3319767460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc408106f10 msgr2=0x7fc40819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc408106f10 0x7fc40819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 -- 172.21.15.143:0/3319767460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408101de0 msgr2=0x7fc4081982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408101de0 0x7fc4081982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc406ffd640 1 -- 172.21.15.143:0/3319767460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40819d290 con 0x7fc408102790 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.988+0000 7fc4077fe640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408101de0 0x7fc4081982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:03.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.989+0000 7fc406ffd640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 0x7fc408198820 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c920 tx=0x7fc3fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.989+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc007c70 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.989+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40819d580 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.989+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3fc007e10 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.989+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc018820 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.989+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a4e60 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.990+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4081021c0 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.991+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc3fc0189c0 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.991+0000 7fc404ff9640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc3f00a2b80 0x7fc3f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.992+0000 7fc4077fe640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc3f00a2b80 0x7fc3f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.992+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc3fc0cf4d0 con 0x7fc408102790 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.992+0000 7fc4077fe640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc3f00a2b80 0x7fc3f00a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:03.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:02.996+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3fc014030 con 0x7fc408102790 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.167+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc40802d7c0 con 0x7fc408102790 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.168+0000 7fc404ff9640 1 -- 172.21.15.143:0/3319767460 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7fc3fc09c0d0 con 0x7fc408102790 2024-09-10T10:45:03.170 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:03.170 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc3f00a2b80 msgr2=0x7fc3f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc3f00a2b80 0x7fc3f00a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f8031400 comp rx=0 tx=0).stop 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 msgr2=0x7fc408198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 0x7fc408198820 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00c920 tx=0x7fc3fc00cde0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 shutdown_connections 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc3f00a2b80 0x7fc3f00a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc408106f10 0x7fc40819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc408102790 0x7fc408198820 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 --2- 172.21.15.143:0/3319767460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408101de0 0x7fc4081982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 >> 172.21.15.143:0/3319767460 conn(0x7fc4080fd8c0 msgr2=0x7fc4080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 shutdown_connections 2024-09-10T10:45:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:03.171+0000 7fc40db9d640 1 -- 172.21.15.143:0/3319767460 wait complete. 2024-09-10T10:45:03.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3319767460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:03.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3319767460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:03.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:42:53.956292+0000', 'modified': '2024-09-10T10:43:32.550735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24502, 'mds_1': 24485, 'mds_2': 24481, 'mds_3': 24475, 'mds_4': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6839/2518540467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2518540467}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2518540467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6841/1083221625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6840', 'nonce': 1083221625}, {'type': 'v1', 'addr': '172.21.15.143:6841', 'nonce': 1083221625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3121359191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3121359191}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3121359191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1016684638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1016684638}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1016684638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3190468681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3190468681}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3190468681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.144:6837/3174266316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 3174266316}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 3174266316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.178:6835/261851614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 261851614}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 261851614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.178:6837/2330516637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2330516637}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2330516637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.144:6839/1994626034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 1994626034}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 1994626034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.178:6839/774166191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 774166191}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 774166191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24502, 'qdb_cluster': [24502, 24485, 24481, 24475, 24458]} 2024-09-10T10:45:03.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T10:45:03.930 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T10:45:03.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3319767460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:04.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 -- 172.21.15.143:0/772292424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac101de0 msgr2=0x7ffbac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/772292424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac101de0 0x7ffbac1021c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009900 tx=0x7ffba002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:04.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 -- 172.21.15.143:0/772292424 shutdown_connections 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/772292424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbac106f10 0x7ffbac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/772292424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac102790 0x7ffbac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/772292424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac101de0 0x7ffbac1021c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 -- 172.21.15.143:0/772292424 >> 172.21.15.143:0/772292424 conn(0x7ffbac0fd8c0 msgr2=0x7ffbac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 -- 172.21.15.143:0/772292424 shutdown_connections 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 -- 172.21.15.143:0/772292424 wait complete. 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 Processor -- start 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.667+0000 7ffbb31e2640 1 -- start start 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbac101de0 0x7ffbac1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 0x7ffbac198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac106f10 0x7ffbac19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac111e60 con 0x7ffbac102790 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac111ce0 con 0x7ffbac106f10 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac111fe0 con 0x7ffbac101de0 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 0x7ffbac198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 0x7ffbac198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52718/0 (socket says 172.21.15.143:52718) 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 -- 172.21.15.143:0/2811592945 learned_addr learned my addr 172.21.15.143:0/2811592945 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac106f10 0x7ffbac19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbac101de0 0x7ffbac1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 -- 172.21.15.143:0/2811592945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbac101de0 msgr2=0x7ffbac1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbac101de0 0x7ffbac1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 -- 172.21.15.143:0/2811592945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac106f10 msgr2=0x7ffbac19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac106f10 0x7ffbac19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 -- 172.21.15.143:0/2811592945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac19d390 con 0x7ffbac102790 2024-09-10T10:45:04.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 0x7ffbac198920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800b4a0 tx=0x7ffb9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffba9ffb640 1 -- 172.21.15.143:0/2811592945 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98014070 con 0x7ffbac102790 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac19d680 con 0x7ffbac102790 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffba9ffb640 1 -- 172.21.15.143:0/2811592945 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb98010ac0 con 0x7ffbac102790 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.668+0000 7ffba9ffb640 1 -- 172.21.15.143:0/2811592945 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98017c30 con 0x7ffbac102790 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.669+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac1a4f60 con 0x7ffbac102790 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffba9ffb640 1 -- 172.21.15.143:0/2811592945 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffb98017070 con 0x7ffbac102790 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 Processor -- start 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 -- start start 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffba9ffb640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffb9c0a2b80 0x7ffb9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 0x561dfa6f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa706a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa706fc0 0x561dfa7093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561dfa65e090 con 0x561dfa6f78d0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561dfa605a70 con 0x561dfa6f81f0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.671+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561dfa65a050 con 0x561dfa706fc0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 0x561dfa6f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffb9c0a2b80 0x7ffb9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 0x561dfa6f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52734/0 (socket says 172.21.15.143:52734) 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffba9ffb640 1 -- 172.21.15.143:0/2811592945 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffb980ce9b0 con 0x7ffbac102790 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 -- 172.21.15.143:0/1935462672 learned_addr learned my addr 172.21.15.143:0/1935462672 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb177f640 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa706fc0 0x561dfa7093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbabfff640 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa706a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 -- 172.21.15.143:0/1935462672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa706fc0 msgr2=0x561dfa7093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa706fc0 0x561dfa7093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 -- 172.21.15.143:0/1935462672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 msgr2=0x561dfa706a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa706a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 -- 172.21.15.143:0/1935462672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfa612130 con 0x561dfa6f78d0 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffb9c0a2b80 0x7ffb9c0a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffba4005fd0 tx=0x7ffba4014bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 0x561dfa6f7cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009900 tx=0x7ffba0032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b57fa014c91a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffb96ffd640 1 -- 172.21.15.143:0/1935462672 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0040070 con 0x561dfa6f78d0 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffb96ffd640 1 -- 172.21.15.143:0/1935462672 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba00316a0 con 0x561dfa6f78d0 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.672+0000 7ffb96ffd640 1 -- 172.21.15.143:0/1935462672 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0035040 con 0x561dfa6f78d0 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1935462672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 msgr2=0x561dfa6f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 0x561dfa6f7cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009900 tx=0x7ffba0032e40 comp rx=0 tx=0).stop 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1935462672 shutdown_connections 2024-09-10T10:45:04.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa706fc0 0x561dfa7093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa706a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1935462672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f78d0 0x561dfa6f7cb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1935462672 >> 172.21.15.143:0/1935462672 conn(0x561dfa6f3f60 msgr2=0x561dfa6f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1935462672 shutdown_connections 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1935462672 wait complete. 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 -- start start 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.673+0000 7ffbb55bb740 1 Processor -- start 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa7068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 0x561dfa703bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa706fc0 0x561dfa704130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561dfa709b90 con 0x561dfa6f81f0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561dfa618f20 con 0x561dfa706fc0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561dfa709d10 con 0x561dfa6f78d0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb55bb740 10 client.0 _ll_register_callbacks cb 0x561dfa5e2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 0x561dfa703bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 0x561dfa703bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52738/0 (socket says 172.21.15.143:52738) 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 -- 172.21.15.143:0/2780015089 learned_addr learned my addr 172.21.15.143:0/2780015089 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa706fc0 0x561dfa704130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa7068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 -- 172.21.15.143:0/2780015089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 msgr2=0x561dfa7068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa7068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 -- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa706fc0 msgr2=0x561dfa704130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa706fc0 0x561dfa704130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 -- 172.21.15.143:0/2780015089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfa3cf2b0 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 0x561dfa703bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb98097930 tx=0x7ffb9809aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09eb5fae5031ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.674+0000 7ffb8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2780015089 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9801e070 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2780015089 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffb98099260 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba00316a0 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2780015089 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb9801a400 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 10 client.15201 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561dfa7155a0 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2780015089 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba00316a0 con 0x561dfa6f81f0 2024-09-10T10:45:04.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2780015089 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffb98010c70 con 0x561dfa6f81f0 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 10 client.15201 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 10 client.15201 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 20 client.15201 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa717c30 0x561dfa7190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 4 client.15201 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dfa66b600 con 0x561dfa717c30 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.675+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa717c30 0x561dfa7190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.676+0000 7ffb8bfff640 10 client.15201 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.676+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa717c30 0x561dfa7190a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=108e40b8143d2fd2 server_cookie=6e2510894e2edd79 in_seq=0 out_seq=0 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.676+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2780015089 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dfa66b600 con 0x561dfa717c30 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.676+0000 7ffb8bfff640 10 client.15201 handle_command_reply: tid=0 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 client.15201 shutdown 2024-09-10T10:45:04.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 msgr2=0x561dfa703bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 0x561dfa703bf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb98097930 tx=0x7ffb9809aaa0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 shutdown_connections 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa717c30 0x561dfa7190a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa706fc0 0x561dfa704130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa6f81f0 0x561dfa703bf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2780015089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa7068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 >> 172.21.15.143:0/2780015089 conn(0x561dfa6f3f60 msgr2=0x561dfa6f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffb8bfff640 0 client.15201 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 shutdown_connections 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2780015089 wait complete. 2024-09-10T10:45:04.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.677+0000 7ffbb55bb740 20 client.15201 trim_cache size 0 max 16384 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.694+0000 7ffbb55bb740 1 Processor -- start 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.694+0000 7ffbb55bb740 1 -- start start 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa717560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa7189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa7034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561dfa7173e0 con 0x561dfa706fc0 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561dfa717ab0 con 0x561dfa6f81f0 2024-09-10T10:45:04.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561dfa718180 con 0x561dfa6f78d0 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa7034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa7034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52748/0 (socket says 172.21.15.143:52748) 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 -- 172.21.15.143:0/1640855770 learned_addr learned my addr 172.21.15.143:0/1640855770 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbabfff640 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa7189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa717560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 -- 172.21.15.143:0/1640855770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 msgr2=0x561dfa717560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa717560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 -- 172.21.15.143:0/1640855770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 msgr2=0x561dfa7189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa7189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 -- 172.21.15.143:0/1640855770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dfa6f50f0 con 0x561dfa706fc0 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffbb177f640 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa7034b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffba4012c80 tx=0x7ffba401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc323ea19c95cf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffb967fc640 1 -- 172.21.15.143:0/1640855770 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4016e60 con 0x561dfa706fc0 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.695+0000 7ffb967fc640 1 -- 172.21.15.143:0/1640855770 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba4026070 con 0x561dfa706fc0 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffb967fc640 1 -- 172.21.15.143:0/1640855770 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4022e50 con 0x561dfa706fc0 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1640855770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 msgr2=0x561dfa7034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa7034b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffba4012c80 tx=0x7ffba401d000 comp rx=0 tx=0).stop 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1640855770 shutdown_connections 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa7034b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa7189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/1640855770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa717560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1640855770 >> 172.21.15.143:0/1640855770 conn(0x561dfa6f3f60 msgr2=0x561dfa6f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1640855770 shutdown_connections 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.696+0000 7ffbb55bb740 1 -- 172.21.15.143:0/1640855770 wait complete. 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 -- start start 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 Processor -- start 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa721bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa71a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa71a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561dfa718f20 con 0x561dfa706fc0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561dfa718850 con 0x561dfa6f81f0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561dfa7195f0 con 0x561dfa6f78d0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb55bb740 10 client.0 _ll_register_callbacks cb 0x7ffb980174e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb177f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa71a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb177f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa71a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52760/0 (socket says 172.21.15.143:52760) 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb177f640 1 -- 172.21.15.143:0/2678009524 learned_addr learned my addr 172.21.15.143:0/2678009524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbabfff640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa71a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.697+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa721bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb177f640 1 -- 172.21.15.143:0/2678009524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 msgr2=0x561dfa721bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa721bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb177f640 1 -- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 msgr2=0x561dfa71a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa71a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb177f640 1 -- 172.21.15.143:0/2678009524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dfa780330 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb177f640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa71a970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffba4012c50 tx=0x7ffba4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4e815f3ae65754 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffb8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2678009524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4022dd0 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2678009524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffba4022400 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba4026070 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2678009524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffba4022600 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb55bb740 10 client.15207 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561dfa7814b0 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2678009524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba4026070 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2678009524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffba400f040 con 0x561dfa706fc0 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb55bb740 10 client.15207 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.698+0000 7ffbb55bb740 10 client.15207 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T10:45:04.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 20 client.15207 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa785e70 0x561dfa784b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 4 client.15207 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "status"}) -- 0x561dfa665b10 con 0x561dfa785e70 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa785e70 0x561dfa784b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb0f7e640 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa785e70 0x561dfa784b80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fcf7d073ef517326 server_cookie=c320f3f5c4607e1b in_seq=0 out_seq=0 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffb8bfff640 10 client.15207 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffb8bfff640 1 -- 172.21.15.143:0/2678009524 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561dfa665b10 con 0x561dfa785e70 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffb8bfff640 10 client.15207 handle_command_reply: tid=0 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 client.15207 shutdown 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 msgr2=0x561dfa71a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa71a970 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffba4012c50 tx=0x7ffba4005a50 comp rx=0 tx=0).stop 2024-09-10T10:45:04.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 shutdown_connections 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561dfa785e70 0x561dfa784b80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561dfa706fc0 0x561dfa71a970 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561dfa6f81f0 0x561dfa71a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffb8bfff640 0 client.15207 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 --2- 172.21.15.143:0/2678009524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561dfa6f78d0 0x561dfa721bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.699+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 >> 172.21.15.143:0/2678009524 conn(0x561dfa6f3f60 msgr2=0x561dfa6fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 shutdown_connections 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb55bb740 1 -- 172.21.15.143:0/2678009524 wait complete. 2024-09-10T10:45:04.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb55bb740 20 client.15207 trim_cache size 0 max 16384 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 4, 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24458, 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 93.157976005999998, 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:45:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 153.19415603499999 2024-09-10T10:45:04.712 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:04.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffb9c0a2b80 msgr2=0x7ffb9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffb9c0a2b80 0x7ffb9c0a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffba4005fd0 tx=0x7ffba4014bf0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 msgr2=0x7ffbac198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.700+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 0x7ffbac198920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800b4a0 tx=0x7ffb9800b960 comp rx=0 tx=0).stop 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 shutdown_connections 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffb9c0a2b80 0x7ffb9c0a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac106f10 0x7ffbac19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbac102790 0x7ffbac198920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 --2- 172.21.15.143:0/2811592945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbac101de0 0x7ffbac1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 >> 172.21.15.143:0/2811592945 conn(0x7ffbac0fd8c0 msgr2=0x7ffbac0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 shutdown_connections 2024-09-10T10:45:04.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:04.701+0000 7ffbb31e2640 1 -- 172.21.15.143:0/2811592945 wait complete. 2024-09-10T10:45:04.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:04 smithi143 ceph-mon[31057]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T10:45:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:04 smithi178 ceph-mon[34813]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T10:45:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:04 smithi144 ceph-mon[34230]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T10:45:05.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T10:45:05.761 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 -- 172.21.15.143:0/2749982197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 msgr2=0x7fbcb810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 --2- 172.21.15.143:0/2749982197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb810d950 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009cd0 tx=0x7fbcb002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 -- 172.21.15.143:0/2749982197 shutdown_connections 2024-09-10T10:45:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 --2- 172.21.15.143:0/2749982197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb810d950 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 --2- 172.21.15.143:0/2749982197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb8100a80 0x7fbcb8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 --2- 172.21.15.143:0/2749982197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb8106a80 0x7fbcb8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 -- 172.21.15.143:0/2749982197 >> 172.21.15.143:0/2749982197 conn(0x7fbcb80fc7a0 msgr2=0x7fbcb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 -- 172.21.15.143:0/2749982197 shutdown_connections 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.301+0000 7fbcc059b640 1 -- 172.21.15.143:0/2749982197 wait complete. 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 Processor -- start 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 -- start start 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb8100a80 0x7fbcb8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb8106a80 0x7fbcb819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8111e50 con 0x7fbcb8101420 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8111cd0 con 0x7fbcb8100a80 2024-09-10T10:45:06.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcc059b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8111fd0 con 0x7fbcb8106a80 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcbdb36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcbdb36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52774/0 (socket says 172.21.15.143:52774) 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.302+0000 7fbcbdb36640 1 -- 172.21.15.143:0/3089664429 learned_addr learned my addr 172.21.15.143:0/3089664429 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb8100a80 0x7fbcb8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbeb38640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb8106a80 0x7fbcb819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbdb36640 1 -- 172.21.15.143:0/3089664429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb8106a80 msgr2=0x7fbcb819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb8106a80 0x7fbcb819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbdb36640 1 -- 172.21.15.143:0/3089664429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb8100a80 msgr2=0x7fbcb8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb8100a80 0x7fbcb8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbdb36640 1 -- 172.21.15.143:0/3089664429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb819d300 con 0x7fbcb8101420 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb8198890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbca800c920 tx=0x7fbca800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcaf7fe640 1 -- 172.21.15.143:0/3089664429 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8007c70 con 0x7fbcb8101420 2024-09-10T10:45:06.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcaf7fe640 1 -- 172.21.15.143:0/3089664429 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbca8007e10 con 0x7fbcb8101420 2024-09-10T10:45:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb819d5f0 con 0x7fbcb8101420 2024-09-10T10:45:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcaf7fe640 1 -- 172.21.15.143:0/3089664429 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8018820 con 0x7fbcb8101420 2024-09-10T10:45:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.303+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81a4ed0 con 0x7fbcb8101420 2024-09-10T10:45:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.305+0000 7fbcaf7fe640 1 -- 172.21.15.143:0/3089664429 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbca80189c0 con 0x7fbcb8101420 2024-09-10T10:45:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 Processor -- start 2024-09-10T10:45:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 -- start start 2024-09-10T10:45:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcaf7fe640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2b80 0x7fbca40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 0x558916dcdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916ddca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcaf7fe640 1 -- 172.21.15.143:0/3089664429 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbca80cf4d0 con 0x7fbcb8101420 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916ddf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558916d34090 con 0x558916dce1b0 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558916cdba70 con 0x558916dcd890 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcc2974740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558916d30050 con 0x558916ddcfc0 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.306+0000 7fbcbeb38640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2b80 0x7fbca40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 0x558916dcdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbdb36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916ddca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 0x558916dcdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:53636/0 (socket says 172.21.15.143:53636) 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 -- 172.21.15.143:0/3395629099 learned_addr learned my addr 172.21.15.143:0/3395629099 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbeb38640 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916ddf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 -- 172.21.15.143:0/3395629099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 msgr2=0x558916ddf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916ddf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 -- 172.21.15.143:0/3395629099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 msgr2=0x558916ddca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916ddca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 -- 172.21.15.143:0/3395629099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558916ce8130 con 0x558916dcd890 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbeb38640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2b80 0x7fbca40a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009ca0 tx=0x7fbcb0031710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 0x558916dcdc70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400ab60 tx=0x7fbcb400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d79e98d37c0fd54c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcacff9640 1 -- 172.21.15.143:0/3395629099 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb401d070 con 0x558916dcd890 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcacff9640 1 -- 172.21.15.143:0/3395629099 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcb4018af0 con 0x558916dcd890 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.307+0000 7fbcacff9640 1 -- 172.21.15.143:0/3395629099 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40109e0 con 0x558916dcd890 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 -- 172.21.15.143:0/3395629099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 msgr2=0x558916dcdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 0x558916dcdc70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400ab60 tx=0x7fbcb400f790 comp rx=0 tx=0).stop 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 -- 172.21.15.143:0/3395629099 shutdown_connections 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916ddf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916ddca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3395629099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dcd890 0x558916dcdc70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 -- 172.21.15.143:0/3395629099 >> 172.21.15.143:0/3395629099 conn(0x558916dc9f60 msgr2=0x558916dca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 -- 172.21.15.143:0/3395629099 shutdown_connections 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 -- 172.21.15.143:0/3395629099 wait complete. 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.308+0000 7fbcc2974740 1 -- start start 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 Processor -- start 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916dcd890 0x558916ddc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916dd9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916ddcfc0 0x558916dda0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558916ddfb90 con 0x558916dce1b0 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558916ceef20 con 0x558916ddcfc0 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558916ddfd10 con 0x558916dcd890 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcc2974740 10 client.0 _ll_register_callbacks cb 0x558916cb8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916dd9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916dd9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52796/0 (socket says 172.21.15.143:52796) 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 -- 172.21.15.143:0/2824027379 learned_addr learned my addr 172.21.15.143:0/2824027379 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbeb38640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916ddcfc0 0x558916dda0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbe337640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916dcd890 0x558916ddc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 -- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916dcd890 msgr2=0x558916ddc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916dcd890 0x558916ddc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 -- 172.21.15.143:0/2824027379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916ddcfc0 msgr2=0x558916dda0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916ddcfc0 0x558916dda0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 -- 172.21.15.143:0/2824027379 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558916aa52b0 con 0x558916dce1b0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916dd9bb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbca80983a0 tx=0x7fbca800fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9e3e01a97a699e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.309+0000 7fbca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbca1ffb640 1 -- 172.21.15.143:0/2824027379 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca809fd80 con 0x558916dce1b0 2024-09-10T10:45:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbca1ffb640 1 -- 172.21.15.143:0/2824027379 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbca80a41b0 con 0x558916dce1b0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb4018af0 con 0x558916dce1b0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbca1ffb640 1 -- 172.21.15.143:0/2824027379 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbca80a4640 con 0x558916dce1b0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 10 client.15219 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558916deb570 con 0x558916dce1b0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbca1ffb640 1 -- 172.21.15.143:0/2824027379 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcb4018af0 con 0x558916dce1b0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbca1ffb640 1 -- 172.21.15.143:0/2824027379 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbca80860b0 con 0x558916dce1b0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 10 client.15219 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 10 client.15219 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 20 client.15219 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916dedb60 0x558916deefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 4 client.15219 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.310+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558916d41600 con 0x558916dedb60 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.311+0000 7fbcbe337640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916dedb60 0x558916deefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.311+0000 7fbcbe337640 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916dedb60 0x558916deefd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69d0e00df909a2d5 server_cookie=5233076adf8d40d8 in_seq=0 out_seq=0 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.311+0000 7fbca1ffb640 10 client.15219 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbca1ffb640 1 -- 172.21.15.143:0/2824027379 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558916d41600 con 0x558916dedb60 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbca1ffb640 10 client.15219 handle_command_reply: tid=0 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 client.15219 shutdown 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 msgr2=0x558916dd9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916dd9bb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbca80983a0 tx=0x7fbca800fbc0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 shutdown_connections 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916dedb60 0x558916deefd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916ddcfc0 0x558916dda0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dce1b0 0x558916dd9bb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbca1ffb640 0 client.15219 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2824027379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916dcd890 0x558916ddc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 >> 172.21.15.143:0/2824027379 conn(0x558916dc9f60 msgr2=0x558916dcfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 shutdown_connections 2024-09-10T10:45:06.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 1 -- 172.21.15.143:0/2824027379 wait complete. 2024-09-10T10:45:06.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.312+0000 7fbcc2974740 20 client.15219 trim_cache size 0 max 16384 2024-09-10T10:45:06.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.329+0000 7fbcc2974740 1 Processor -- start 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 -- start start 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916deefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916dd8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558916ded310 con 0x558916dcd890 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558916ded9e0 con 0x558916dce1b0 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcc2974740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558916dee0b0 con 0x558916ddcfc0 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916deefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916deefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52802/0 (socket says 172.21.15.143:52802) 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 -- 172.21.15.143:0/2331479118 learned_addr learned my addr 172.21.15.143:0/2331479118 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:06.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916dd8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 -- 172.21.15.143:0/2331479118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 msgr2=0x558916dd89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 -- 172.21.15.143:0/2331479118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 msgr2=0x558916dd8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916dd8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 -- 172.21.15.143:0/2331479118 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558916de9830 con 0x558916dcd890 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbcbe337640 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916deefd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbcb401b040 tx=0x7fbcb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8b758e1b78591d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.330+0000 7fbca3fff640 1 -- 172.21.15.143:0/2331479118 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb401d040 con 0x558916dcd890 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbca3fff640 1 -- 172.21.15.143:0/2331479118 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcb400a5e0 con 0x558916dcd890 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbca3fff640 1 -- 172.21.15.143:0/2331479118 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4010860 con 0x558916dcd890 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 -- 172.21.15.143:0/2331479118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 msgr2=0x558916deefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916deefd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbcb401b040 tx=0x7fbcb40046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 -- 172.21.15.143:0/2331479118 shutdown_connections 2024-09-10T10:45:06.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916dd8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 --2- 172.21.15.143:0/2331479118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916deefd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 -- 172.21.15.143:0/2331479118 >> 172.21.15.143:0/2331479118 conn(0x558916dc9f60 msgr2=0x558916dc4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 -- 172.21.15.143:0/2331479118 shutdown_connections 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 -- 172.21.15.143:0/2331479118 wait complete. 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.331+0000 7fbcc2974740 1 -- start start 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 Processor -- start 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916df7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916df7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558916deee50 con 0x558916dcd890 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558916dee780 con 0x558916dce1b0 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558916def520 con 0x558916ddcfc0 2024-09-10T10:45:06.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcc2974740 10 client.0 _ll_register_callbacks cb 0x7fbca809b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916df7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916df7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52804/0 (socket says 172.21.15.143:52804) 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 -- 172.21.15.143:0/3513651011 learned_addr learned my addr 172.21.15.143:0/3513651011 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916df7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbeb38640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 -- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 msgr2=0x558916dd6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 -- 172.21.15.143:0/3513651011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 msgr2=0x558916df7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916df7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 -- 172.21.15.143:0/3513651011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558916dd6d30 con 0x558916dcd890 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbcbe337640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916df7420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4005c30 tx=0x7fbcb4004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d2ab35e6858c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.332+0000 7fbca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbca1ffb640 1 -- 172.21.15.143:0/3513651011 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40186c0 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbca1ffb640 1 -- 172.21.15.143:0/3513651011 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbcb4018860 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb400a5e0 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbca1ffb640 1 -- 172.21.15.143:0/3513651011 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbcb4045050 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 10 client.15225 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558916df6c60 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbca1ffb640 1 -- 172.21.15.143:0/3513651011 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcb400a5e0 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbca1ffb640 1 -- 172.21.15.143:0/3513651011 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbcb4010580 con 0x558916dcd890 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 10 client.15225 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 20 client.15225 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916e5c220 0x558916e5af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 4 client.15225 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:06.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "status"}) -- 0x558916d3bb10 con 0x558916e5c220 2024-09-10T10:45:06.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.333+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916e5c220 0x558916e5af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbca1ffb640 10 client.15225 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:06.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcbdb36640 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916e5c220 0x558916e5af30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0885771ac2c13c2 server_cookie=2345c0eae3e26ac3 in_seq=0 out_seq=0 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbca1ffb640 1 -- 172.21.15.143:0/3513651011 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558916d3bb10 con 0x558916e5c220 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbca1ffb640 10 client.15225 handle_command_reply: tid=0 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 client.15225 shutdown 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 msgr2=0x558916df7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916df7420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4005c30 tx=0x7fbcb4004520 comp rx=0 tx=0).stop 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 shutdown_connections 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x558916e5c220 0x558916e5af30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558916ddcfc0 0x558916dd6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbca1ffb640 0 client.15225 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:06.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558916dce1b0 0x558916df7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 --2- 172.21.15.143:0/3513651011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558916dcd890 0x558916df7420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 >> 172.21.15.143:0/3513651011 conn(0x558916dc9f60 msgr2=0x558916dd9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 shutdown_connections 2024-09-10T10:45:06.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 1 -- 172.21.15.143:0/3513651011 wait complete. 2024-09-10T10:45:06.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.334+0000 7fbcc2974740 20 client.15225 trim_cache size 0 max 16384 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 3, 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24475, 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 95.802828864000006, 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:45:06.343 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 160.09733348 2024-09-10T10:45:06.344 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:06.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2b80 msgr2=0x7fbca40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2b80 0x7fbca40a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009ca0 tx=0x7fbcb0031710 comp rx=0 tx=0).stop 2024-09-10T10:45:06.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 msgr2=0x7fbcb8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb8198890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbca800c920 tx=0x7fbca800cde0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 shutdown_connections 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2b80 0x7fbca40a5040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb8106a80 0x7fbcb819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb8101420 0x7fbcb8198890 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 --2- 172.21.15.143:0/3089664429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb8100a80 0x7fbcb8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 >> 172.21.15.143:0/3089664429 conn(0x7fbcb80fc7a0 msgr2=0x7fbcb80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 shutdown_connections 2024-09-10T10:45:06.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:06.335+0000 7fbcc059b640 1 -- 172.21.15.143:0/3089664429 wait complete. 2024-09-10T10:45:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:06 smithi178 ceph-mon[34813]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:45:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:06 smithi144 ceph-mon[34230]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:45:06.939 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:06 smithi143 ceph-mon[31057]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:45:07.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T10:45:07.412 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:07.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/1119607392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 msgr2=0x7fd7d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/1119607392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d01029b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009900 tx=0x7fd7c002a480 comp rx=0 tx=0).stop 2024-09-10T10:45:07.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/1119607392 shutdown_connections 2024-09-10T10:45:07.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/1119607392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0102ef0 0x7fd7d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/1119607392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d01029b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/1119607392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0100e20 0x7fd7d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/1119607392 >> 172.21.15.143:0/1119607392 conn(0x7fd7d00fc7d0 msgr2=0x7fd7d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/1119607392 shutdown_connections 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.941+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/1119607392 wait complete. 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 Processor -- start 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 -- start start 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0100e20 0x7fd7d0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0102ef0 0x7fd7d019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0110e80 con 0x7fd7d0102570 2024-09-10T10:45:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0110d00 con 0x7fd7d0100e20 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d6d9e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0111000 con 0x7fd7d0102ef0 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41268/0 (socket says 172.21.15.143:41268) 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 -- 172.21.15.143:0/877237503 learned_addr learned my addr 172.21.15.143:0/877237503 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0100e20 0x7fd7d0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d533b640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0102ef0 0x7fd7d019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 -- 172.21.15.143:0/877237503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0102ef0 msgr2=0x7fd7d019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0102ef0 0x7fd7d019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 -- 172.21.15.143:0/877237503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0100e20 msgr2=0x7fd7d0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0100e20 0x7fd7d0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 -- 172.21.15.143:0/877237503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d019d2e0 con 0x7fd7d0102570 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d533b640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0102ef0 0x7fd7d019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0100e20 0x7fd7d0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.942+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d0198870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006010 tx=0x7fd7c0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.943+0000 7fd7c5ffb640 1 -- 172.21.15.143:0/877237503 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c003f070 con 0x7fd7d0102570 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.943+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d019d570 con 0x7fd7d0102570 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.943+0000 7fd7c5ffb640 1 -- 172.21.15.143:0/877237503 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7c003bd20 con 0x7fd7d0102570 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.943+0000 7fd7c5ffb640 1 -- 172.21.15.143:0/877237503 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0035030 con 0x7fd7d0102570 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.943+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d01a4eb0 con 0x7fd7d0102570 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.945+0000 7fd7c5ffb640 1 -- 172.21.15.143:0/877237503 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd7c0042070 con 0x7fd7d0102570 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.945+0000 7fd7d9177740 1 Processor -- start 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.945+0000 7fd7d9177740 1 -- start start 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.945+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x56289183adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.945+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891845670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891847fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d9177740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628917a1090 con 0x56289183b2f0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d9177740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562891748a70 con 0x56289183a9d0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d9177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56289179d050 con 0x562891845bb0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891845670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7c5ffb640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7b80a2b80 0x7fd7b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891845670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41282/0 (socket says 172.21.15.143:41282) 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 -- 172.21.15.143:0/505302356 learned_addr learned my addr 172.21.15.143:0/505302356 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d533b640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7b80a2b80 0x7fd7b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d533b640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x56289183adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7c5ffb640 1 -- 172.21.15.143:0/877237503 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd7c00efda0 con 0x7fd7d0102570 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891847fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 -- 172.21.15.143:0/505302356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 msgr2=0x562891847fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891847fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 -- 172.21.15.143:0/505302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 msgr2=0x56289183adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x56289183adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 -- 172.21.15.143:0/505302356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562891755130 con 0x56289183b2f0 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d533b640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x56289183adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891847fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891845670 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc008c60 tx=0x7fd7cc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65148596384c0f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7d533b640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7b80a2b80 0x7fd7b80a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80104f0 tx=0x7fd7c80074c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7b6ffd640 1 -- 172.21.15.143:0/505302356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc014070 con 0x56289183b2f0 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7b6ffd640 1 -- 172.21.15.143:0/505302356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7cc0047a0 con 0x56289183b2f0 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.946+0000 7fd7b6ffd640 1 -- 172.21.15.143:0/505302356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc010670 con 0x56289183b2f0 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 -- 172.21.15.143:0/505302356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 msgr2=0x562891845670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891845670 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc008c60 tx=0x7fd7cc00e720 comp rx=0 tx=0).stop 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 -- 172.21.15.143:0/505302356 shutdown_connections 2024-09-10T10:45:07.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891847fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891845670 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 --2- 172.21.15.143:0/505302356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x56289183adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 -- 172.21.15.143:0/505302356 >> 172.21.15.143:0/505302356 conn(0x562891836f60 msgr2=0x562891837340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 -- 172.21.15.143:0/505302356 shutdown_connections 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 -- 172.21.15.143:0/505302356 wait complete. 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.947+0000 7fd7d9177740 1 -- start start 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 Processor -- start 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x5628918453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891842700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891842c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562891848780 con 0x56289183b2f0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56289175bf20 con 0x56289183a9d0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562891848900 con 0x562891845bb0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d9177740 10 client.0 _ll_register_callbacks cb 0x562891725030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891842700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891842700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41296/0 (socket says 172.21.15.143:41296) 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3154815220 learned_addr learned my addr 172.21.15.143:0/3154815220 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d533b640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891842c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 msgr2=0x562891842c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891842c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3154815220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 msgr2=0x5628918453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x5628918453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x5628918453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3154815220 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628915122b0 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7d533b640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891842c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891842700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00c49e0 tx=0x7fd7c00bc130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b497b1efd6b707 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.948+0000 7fd7affff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7affff640 1 -- 172.21.15.143:0/3154815220 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0049040 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7affff640 1 -- 172.21.15.143:0/3154815220 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd7c00b8910 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc0047a0 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7affff640 1 -- 172.21.15.143:0/3154815220 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7c00bc9a0 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 10 client.15237 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562891853f80 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7affff640 1 -- 172.21.15.143:0/3154815220 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7cc0047a0 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7affff640 1 -- 172.21.15.143:0/3154815220 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd7c00d50d0 con 0x56289183b2f0 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:07.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 10 client.15237 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 20 client.15237 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562891856660 0x562891857ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 4 client.15237 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628917ae600 con 0x562891856660 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.949+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562891856660 0x562891857ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.950+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562891856660 0x562891857ad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=748ba44a25711b2e server_cookie=576a30001b434043 in_seq=0 out_seq=0 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.950+0000 7fd7affff640 10 client.15237 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7affff640 1 -- 172.21.15.143:0/3154815220 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628917ae600 con 0x562891856660 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7affff640 10 client.15237 handle_command_reply: tid=0 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 client.15237 shutdown 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 msgr2=0x562891842700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891842700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00c49e0 tx=0x7fd7c00bc130 comp rx=0 tx=0).stop 2024-09-10T10:45:07.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 shutdown_connections 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562891856660 0x562891857ad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562891845bb0 0x562891842c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891842700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3154815220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x5628918453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 >> 172.21.15.143:0/3154815220 conn(0x562891836f60 msgr2=0x56289183cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7affff640 0 client.15237 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 shutdown_connections 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 1 -- 172.21.15.143:0/3154815220 wait complete. 2024-09-10T10:45:07.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.951+0000 7fd7d9177740 20 client.15237 trim_cache size 0 max 16384 2024-09-10T10:45:07.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.968+0000 7fd7d9177740 1 Processor -- start 2024-09-10T10:45:07.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 -- start start 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183a9d0 0x562891852000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891857ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562891845bb0 0x562891855f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562891855e10 con 0x56289183b2f0 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628918564e0 con 0x562891845bb0 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d9177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562891856bb0 con 0x56289183a9d0 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891857ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891857ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41302/0 (socket says 172.21.15.143:41302) 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3713111019 learned_addr learned my addr 172.21.15.143:0/3713111019 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d533b640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562891845bb0 0x562891855f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183a9d0 0x562891852000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3713111019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183a9d0 msgr2=0x562891852000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183a9d0 0x562891852000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3713111019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562891845bb0 msgr2=0x562891855f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562891845bb0 0x562891855f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 -- 172.21.15.143:0/3713111019 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562891857060 con 0x56289183b2f0 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d533b640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562891845bb0 0x562891855f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183a9d0 0x562891852000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.969+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891857ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0031940 tx=0x7fd7c00ba7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c54be660defa762 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7b67fc640 1 -- 172.21.15.143:0/3713111019 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0049070 con 0x56289183b2f0 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7b67fc640 1 -- 172.21.15.143:0/3713111019 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7c00c0650 con 0x56289183b2f0 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7b67fc640 1 -- 172.21.15.143:0/3713111019 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00d50d0 con 0x56289183b2f0 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 -- 172.21.15.143:0/3713111019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 msgr2=0x562891857ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891857ad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0031940 tx=0x7fd7c00ba7b0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 -- 172.21.15.143:0/3713111019 shutdown_connections 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562891845bb0 0x562891855f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56289183b2f0 0x562891857ad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 --2- 172.21.15.143:0/3713111019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183a9d0 0x562891852000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 -- 172.21.15.143:0/3713111019 >> 172.21.15.143:0/3713111019 conn(0x562891836f60 msgr2=0x5628918552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 -- 172.21.15.143:0/3713111019 shutdown_connections 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.970+0000 7fd7d9177740 1 -- 172.21.15.143:0/3713111019 wait complete. 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 -- start start 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 Processor -- start 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x562891860570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183b2f0 0x56289185a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 0x56289185aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562891857950 con 0x562891845bb0 2024-09-10T10:45:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562891857280 con 0x56289183a9d0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562891858020 con 0x56289183b2f0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d9177740 10 client.0 _ll_register_callbacks cb 0x7fd7c0031890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d533b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 0x56289185aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d533b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 0x56289185aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41316/0 (socket says 172.21.15.143:41316) 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.971+0000 7fd7d533b640 1 -- 172.21.15.143:0/1793152841 learned_addr learned my addr 172.21.15.143:0/1793152841 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183b2f0 0x56289185a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x562891860570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d533b640 1 -- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183b2f0 msgr2=0x56289185a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d533b640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183b2f0 0x56289185a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d533b640 1 -- 172.21.15.143:0/1793152841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 msgr2=0x562891860570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d533b640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x562891860570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d533b640 1 -- 172.21.15.143:0/1793152841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289185b6d0 con 0x562891845bb0 2024-09-10T10:45:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7c7fff640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183b2f0 0x56289185a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d533b640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 0x56289185aea0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8011fb0 tx=0x7fd7c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf92ed90d03f97e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7affff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7affff640 1 -- 172.21.15.143:0/1793152841 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c801a830 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7affff640 1 -- 172.21.15.143:0/1793152841 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd7c801a9d0 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c00c0650 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7affff640 1 -- 172.21.15.143:0/1793152841 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7c8030050 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d9177740 10 client.15243 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628918bfe30 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.972+0000 7fd7affff640 1 -- 172.21.15.143:0/1793152841 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd7c00c0650 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7affff640 1 -- 172.21.15.143:0/1793152841 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd7c8025020 con 0x562891845bb0 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d9177740 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d9177740 10 client.15243 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d9177740 20 client.15243 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d9177740 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628918c4a90 0x5628918c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d9177740 4 client.15243 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "status"}) -- 0x5628917a8b10 con 0x5628918c4a90 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628918c4a90 0x5628918c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7affff640 10 client.15243 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.973+0000 7fd7d4b3a640 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628918c4a90 0x5628918c37a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17812317e049c6b3 server_cookie=c4bf1d892b5efee7 in_seq=0 out_seq=0 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7affff640 1 -- 172.21.15.143:0/1793152841 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5628917a8b10 con 0x5628918c4a90 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7affff640 10 client.15243 handle_command_reply: tid=0 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 client.15243 shutdown 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 msgr2=0x56289185aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 0x56289185aea0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8011fb0 tx=0x7fd7c8004ab0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 shutdown_connections 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628918c4a90 0x5628918c37a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562891845bb0 0x56289185aea0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56289183b2f0 0x56289185a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 --2- 172.21.15.143:0/1793152841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56289183a9d0 0x562891860570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7affff640 0 client.15243 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 >> 172.21.15.143:0/1793152841 conn(0x562891836f60 msgr2=0x5628918552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 shutdown_connections 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 1 -- 172.21.15.143:0/1793152841 wait complete. 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.974+0000 7fd7d9177740 20 client.15243 trim_cache size 0 max 16384 2024-09-10T10:45:07.982 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 2, 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24481, 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 98.448386936000006, 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 159.133037156 2024-09-10T10:45:07.983 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7b80a2b80 msgr2=0x7fd7b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7b80a2b80 0x7fd7b80a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80104f0 tx=0x7fd7c80074c0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 msgr2=0x7fd7d0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d0198870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006010 tx=0x7fd7c0003d50 comp rx=0 tx=0).stop 2024-09-10T10:45:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 shutdown_connections 2024-09-10T10:45:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd7b80a2b80 0x7fd7b80a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0102ef0 0x7fd7d019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d0102570 0x7fd7d0198870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 --2- 172.21.15.143:0/877237503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7d0100e20 0x7fd7d0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 >> 172.21.15.143:0/877237503 conn(0x7fd7d00fc7d0 msgr2=0x7fd7d006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 shutdown_connections 2024-09-10T10:45:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:07.975+0000 7fd7d6d9e640 1 -- 172.21.15.143:0/877237503 wait complete. 2024-09-10T10:45:08.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:45:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:08 smithi178 ceph-mon[34813]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T10:45:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:08 smithi144 ceph-mon[34230]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T10:45:08.964 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:09.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:08 smithi143 ceph-mon[31057]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T10:45:09.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 -- 172.21.15.143:0/1170883785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840106f10 msgr2=0x7f084010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 --2- 172.21.15.143:0/1170883785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840106f10 0x7f084010ea90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0838009900 tx=0x7f083802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 -- 172.21.15.143:0/1170883785 shutdown_connections 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 --2- 172.21.15.143:0/1170883785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840106f10 0x7f084010ea90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 --2- 172.21.15.143:0/1170883785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0840102790 0x7f08401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 --2- 172.21.15.143:0/1170883785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0840101de0 0x7f08401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 -- 172.21.15.143:0/1170883785 >> 172.21.15.143:0/1170883785 conn(0x7f08400fd880 msgr2=0x7f08400ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 -- 172.21.15.143:0/1170883785 shutdown_connections 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.467+0000 7f084768b640 1 -- 172.21.15.143:0/1170883785 wait complete. 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 Processor -- start 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 -- start start 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 0x7f0840198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0840102790 0x7f0840198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0840106f10 0x7f084019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0840111e20 con 0x7f0840101de0 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0840111ca0 con 0x7f0840102790 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f084768b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0840111fa0 con 0x7f0840106f10 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 0x7f0840198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 0x7f0840198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41338/0 (socket says 172.21.15.143:41338) 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 -- 172.21.15.143:0/132739812 learned_addr learned my addr 172.21.15.143:0/132739812 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0844c26640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0840102790 0x7f0840198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845c28640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0840106f10 0x7f084019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 -- 172.21.15.143:0/132739812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0840106f10 msgr2=0x7f084019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0840106f10 0x7f084019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 -- 172.21.15.143:0/132739812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0840102790 msgr2=0x7f0840198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0840102790 0x7f0840198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 -- 172.21.15.143:0/132739812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084019d2f0 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.468+0000 7f0845427640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 0x7f0840198340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f083000ea80 tx=0x7f083000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.469+0000 7f083e7fc640 1 -- 172.21.15.143:0/132739812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083000ce40 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.469+0000 7f083e7fc640 1 -- 172.21.15.143:0/132739812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0830005020 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.469+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084019d5e0 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.469+0000 7f083e7fc640 1 -- 172.21.15.143:0/132739812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08300116d0 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.469+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08401a4ec0 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.471+0000 7f083e7fc640 1 -- 172.21.15.143:0/132739812 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f08300051d0 con 0x7f0840101de0 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.471+0000 7f0849a64740 1 Processor -- start 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.471+0000 7f0849a64740 1 -- start start 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0849a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2497bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0849a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24986670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0849a64740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da24988fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0849a64740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55da248e2090 con 0x55da24986bb0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0849a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da24889a70 con 0x55da2497c2f0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0849a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da248de050 con 0x55da2497b9d0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f083e7fc640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08280a2b80 0x7f08280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da24988fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0845427640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24986670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da24988fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41348/0 (socket says 172.21.15.143:41348) 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 -- 172.21.15.143:0/3988062924 learned_addr learned my addr 172.21.15.143:0/3988062924 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0845c28640 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2497bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0845c28640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08280a2b80 0x7f08280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f083e7fc640 1 -- 172.21.15.143:0/132739812 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f08300cf620 con 0x7f0840101de0 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 -- 172.21.15.143:0/3988062924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 msgr2=0x55da2497bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2497bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 -- 172.21.15.143:0/3988062924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 msgr2=0x55da24986670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24986670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 -- 172.21.15.143:0/3988062924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da24896130 con 0x55da24986bb0 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0845427640 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24986670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0845c28640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08280a2b80 0x7f08280a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0838006010 tx=0x7f08380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.472+0000 7f0844c26640 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da24988fa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0834009c10 tx=0x7f0834011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a5a846050029d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f08277fe640 1 -- 172.21.15.143:0/3988062924 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834004230 con 0x55da24986bb0 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f08277fe640 1 -- 172.21.15.143:0/3988062924 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08340043d0 con 0x55da24986bb0 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f08277fe640 1 -- 172.21.15.143:0/3988062924 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834004d30 con 0x55da24986bb0 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 -- 172.21.15.143:0/3988062924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 msgr2=0x55da24988fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da24988fa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0834009c10 tx=0x7f0834011810 comp rx=0 tx=0).stop 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 -- 172.21.15.143:0/3988062924 shutdown_connections 2024-09-10T10:45:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da24988fa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24986670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 --2- 172.21.15.143:0/3988062924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2497bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 -- 172.21.15.143:0/3988062924 >> 172.21.15.143:0/3988062924 conn(0x55da24977f60 msgr2=0x55da24978340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 -- 172.21.15.143:0/3988062924 shutdown_connections 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.473+0000 7f0849a64740 1 -- 172.21.15.143:0/3988062924 wait complete. 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 -- start start 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 Processor -- start 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 0x55da24986400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24983750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da24986bb0 0x55da24983c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55da24989780 con 0x55da2497b9d0 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da2489cf20 con 0x55da2497c2f0 2024-09-10T10:45:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da24989900 con 0x55da24986bb0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0849a64740 10 client.0 _ll_register_callbacks cb 0x55da24866030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.474+0000 7f0845427640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 0x55da24986400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 0x55da24986400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41356/0 (socket says 172.21.15.143:41356) 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 -- 172.21.15.143:0/668786045 learned_addr learned my addr 172.21.15.143:0/668786045 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0844c26640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24983750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845c28640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da24986bb0 0x55da24983c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 -- 172.21.15.143:0/668786045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da24986bb0 msgr2=0x55da24983c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da24986bb0 0x55da24983c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 -- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 msgr2=0x55da24983750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24983750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 -- 172.21.15.143:0/668786045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da246532b0 con 0x55da2497b9d0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0845427640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 0x55da24986400 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f083000efd0 tx=0x7f083009b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d07fe50dba8de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0824ff9640 1 -- 172.21.15.143:0/668786045 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083001f040 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0824ff9640 1 -- 172.21.15.143:0/668786045 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0830005540 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08340043d0 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0824ff9640 1 -- 172.21.15.143:0/668786045 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f083009fed0 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0849a64740 10 client.15255 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da24995050 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.475+0000 7f0824ff9640 1 -- 172.21.15.143:0/668786045 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08340043d0 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0824ff9640 1 -- 172.21.15.143:0/668786045 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0830064020 con 0x55da2497b9d0 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0849a64740 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0849a64740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0849a64740 20 client.15255 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0849a64740 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da249976d0 0x55da24998b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0849a64740 4 client.15255 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da248ef600 con 0x55da249976d0 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0844c26640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da249976d0 0x55da24998b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0844c26640 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da249976d0 0x55da24998b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d22c92040f9da93b server_cookie=41eab4a428496d9e in_seq=0 out_seq=0 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.476+0000 7f0824ff9640 10 client.15255 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0824ff9640 1 -- 172.21.15.143:0/668786045 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da248ef600 con 0x55da249976d0 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0824ff9640 10 client.15255 handle_command_reply: tid=0 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 client.15255 shutdown 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 msgr2=0x55da24986400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 0x55da24986400 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f083000efd0 tx=0x7f083009b5f0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 shutdown_connections 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da24986bb0 0x55da24983c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da249976d0 0x55da24998b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da24983750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 --2- 172.21.15.143:0/668786045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497b9d0 0x55da24986400 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 >> 172.21.15.143:0/668786045 conn(0x55da24977f60 msgr2=0x55da2497dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0824ff9640 0 client.15255 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 shutdown_connections 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 1 -- 172.21.15.143:0/668786045 wait complete. 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.477+0000 7f0849a64740 20 client.15255 trim_cache size 0 max 16384 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 Processor -- start 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 -- start start 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da24997000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 0x55da24997da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da24986bb0 0x55da24983710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55da24996e80 con 0x55da2497c2f0 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da24997550 con 0x55da24986bb0 2024-09-10T10:45:09.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0849a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da24997c20 con 0x55da2497b9d0 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0844c26640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 0x55da24997da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.495+0000 7f0844c26640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 0x55da24997da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41366/0 (socket says 172.21.15.143:41366) 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 -- 172.21.15.143:0/2890905735 learned_addr learned my addr 172.21.15.143:0/2890905735 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0845c28640 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da24986bb0 0x55da24983710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0845427640 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da24997000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 -- 172.21.15.143:0/2890905735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 msgr2=0x55da24997000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da24997000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 -- 172.21.15.143:0/2890905735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da24986bb0 msgr2=0x55da24983710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da24986bb0 0x55da24983710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 -- 172.21.15.143:0/2890905735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da24989890 con 0x55da2497c2f0 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0845427640 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da24997000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0844c26640 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 0x55da24997da0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55da249792e0 tx=0x7f0834004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768d9284b6e82b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0826ffd640 1 -- 172.21.15.143:0/2890905735 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083401e040 con 0x55da2497c2f0 2024-09-10T10:45:09.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0826ffd640 1 -- 172.21.15.143:0/2890905735 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f083401f040 con 0x55da2497c2f0 2024-09-10T10:45:09.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.496+0000 7f0826ffd640 1 -- 172.21.15.143:0/2890905735 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834003c70 con 0x55da2497c2f0 2024-09-10T10:45:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 -- 172.21.15.143:0/2890905735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 msgr2=0x55da24997da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 0x55da24997da0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55da249792e0 tx=0x7f0834004970 comp rx=0 tx=0).stop 2024-09-10T10:45:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 -- 172.21.15.143:0/2890905735 shutdown_connections 2024-09-10T10:45:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da24986bb0 0x55da24983710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da2497c2f0 0x55da24997da0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 --2- 172.21.15.143:0/2890905735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da24997000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 -- 172.21.15.143:0/2890905735 >> 172.21.15.143:0/2890905735 conn(0x55da24977f60 msgr2=0x55da24982cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 -- 172.21.15.143:0/2890905735 shutdown_connections 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 -- 172.21.15.143:0/2890905735 wait complete. 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 -- start start 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.497+0000 7f0849a64740 1 Processor -- start 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2499bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da2499c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da2499c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55da249989c0 con 0x55da24986bb0 2024-09-10T10:45:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da249982f0 con 0x55da2497c2f0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da24999090 con 0x55da2497b9d0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0849a64740 10 client.0 _ll_register_callbacks cb 0x7f0830009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da2499c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0844c26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da2499c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0844c26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da2499c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:41934/0 (socket says 172.21.15.143:41934) 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0844c26640 1 -- 172.21.15.143:0/1758646744 learned_addr learned my addr 172.21.15.143:0/1758646744 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845427640 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2499bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 -- 172.21.15.143:0/1758646744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 msgr2=0x55da2499bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2499bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 -- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 msgr2=0x55da2499c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da2499c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 -- 172.21.15.143:0/1758646744 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da249fff00 con 0x55da24986bb0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0845c28640 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da2499c7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0838005940 tx=0x7f0838009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7240f2b642fb49b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.498+0000 7f0824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0824ff9640 1 -- 172.21.15.143:0/1758646744 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838052070 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083401f040 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0824ff9640 1 -- 172.21.15.143:0/1758646744 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0838005220 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0824ff9640 1 -- 172.21.15.143:0/1758646744 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f083805d050 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da24a011e0 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0824ff9640 1 -- 172.21.15.143:0/1758646744 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f083401f040 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0824ff9640 1 -- 172.21.15.143:0/1758646744 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f08380035f0 con 0x55da24986bb0 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 20 client.15261 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da24a05d30 0x55da24a04a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 4 client.15261 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "status"}) -- 0x55da248e9b10 con 0x55da24a05d30 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.499+0000 7f0845427640 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da24a05d30 0x55da24a04a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0824ff9640 10 client.15261 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0845427640 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da24a05d30 0x55da24a04a40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9971df6b20410084 server_cookie=a5cb46bef1f8e777 in_seq=0 out_seq=0 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0824ff9640 1 -- 172.21.15.143:0/1758646744 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55da248e9b10 con 0x55da24a05d30 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0824ff9640 10 client.15261 handle_command_reply: tid=0 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 client.15261 shutdown 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 msgr2=0x55da2499c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da2499c7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0838005940 tx=0x7f0838009260 comp rx=0 tx=0).stop 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 shutdown_connections 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da24986bb0 0x55da2499c7d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55da24a05d30 0x55da24a04a40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2497c2f0 0x55da2499c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 --2- 172.21.15.143:0/1758646744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da2497b9d0 0x55da2499bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 >> 172.21.15.143:0/1758646744 conn(0x55da24977f60 msgr2=0x55da24983710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0824ff9640 0 client.15261 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 shutdown_connections 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 1 -- 172.21.15.143:0/1758646744 wait complete. 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.500+0000 7f0849a64740 20 client.15261 trim_cache size 0 max 16384 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:09.517 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24485, 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 100.982970913, 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 146.46296973299999 2024-09-10T10:45:09.518 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08280a2b80 msgr2=0x7f08280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08280a2b80 0x7f08280a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0838006010 tx=0x7f08380058d0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 msgr2=0x7f0840198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 0x7f0840198340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f083000ea80 tx=0x7f083000ef40 comp rx=0 tx=0).stop 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 shutdown_connections 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08280a2b80 0x7f08280a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0840106f10 0x7f084019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0840102790 0x7f0840198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 --2- 172.21.15.143:0/132739812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0840101de0 0x7f0840198340 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 >> 172.21.15.143:0/132739812 conn(0x7f08400fd880 msgr2=0x7f08400feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 shutdown_connections 2024-09-10T10:45:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:09.501+0000 7f084768b640 1 -- 172.21.15.143:0/132739812 wait complete. 2024-09-10T10:45:10.246 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T10:45:10.491 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:10 smithi178 ceph-mon[34813]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T10:45:10.931 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:10 smithi143 ceph-mon[31057]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T10:45:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:10 smithi144 ceph-mon[34230]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 -- 172.21.15.143:0/1370102405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 msgr2=0x7f41e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1370102405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e810da40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009ca0 tx=0x7f41d802a950 comp rx=0 tx=0).stop 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 -- 172.21.15.143:0/1370102405 shutdown_connections 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1370102405 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e810da40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1370102405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e8102570 0x7f41e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1370102405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41e8100e20 0x7f41e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 -- 172.21.15.143:0/1370102405 >> 172.21.15.143:0/1370102405 conn(0x7f41e80fc7d0 msgr2=0x7f41e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 -- 172.21.15.143:0/1370102405 shutdown_connections 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 -- 172.21.15.143:0/1370102405 wait complete. 2024-09-10T10:45:11.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 Processor -- start 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.010+0000 7f41f05ab640 1 -- start start 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e8100e20 0x7f41e8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41e8102570 0x7f41e81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8110e30 con 0x7f41e8102ef0 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8110cb0 con 0x7f41e8102570 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41e8110fb0 con 0x7f41e8100e20 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41406/0 (socket says 172.21.15.143:41406) 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 -- 172.21.15.143:0/1891885094 learned_addr learned my addr 172.21.15.143:0/1891885094 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41ee347640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e8100e20 0x7f41e8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41edb46640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41e8102570 0x7f41e81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 -- 172.21.15.143:0/1891885094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e8100e20 msgr2=0x7f41e8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e8100e20 0x7f41e8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 -- 172.21.15.143:0/1891885094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41e8102570 msgr2=0x7f41e81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41e8102570 0x7f41e81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 -- 172.21.15.143:0/1891885094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e819d340 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41eeb48640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e819cc60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f41d8004af0 tx=0x7f41d8031f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41e77fe640 1 -- 172.21.15.143:0/1891885094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8034040 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41e77fe640 1 -- 172.21.15.143:0/1891885094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41d80308f0 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.011+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e819d5d0 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.012+0000 7f41e77fe640 1 -- 172.21.15.143:0/1891885094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8004600 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.012+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e81a4f10 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41e77fe640 1 -- 172.21.15.143:0/1891885094 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f41d8037020 con 0x7f41e8102ef0 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 Processor -- start 2024-09-10T10:45:11.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 -- start start 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654d780b090 con 0x5654d78afbb0 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5654d77b2a70 con 0x5654d78a49d0 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41f2984740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654d7807050 con 0x5654d78a52f0 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.014+0000 7f41e77fe640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41d00a2b80 0x7f41d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41412/0 (socket says 172.21.15.143:41412) 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 -- 172.21.15.143:0/997367123 learned_addr learned my addr 172.21.15.143:0/997367123 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41ee347640 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41eeb48640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41d00a2b80 0x7f41d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41e77fe640 1 -- 172.21.15.143:0/1891885094 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f41d80f0190 con 0x7f41e8102ef0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 -- 172.21.15.143:0/997367123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 msgr2=0x5654d78af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 -- 172.21.15.143:0/997367123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 msgr2=0x5654d78a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 -- 172.21.15.143:0/997367123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d77bf130 con 0x5654d78afbb0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41edb46640 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78b1fa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009fa0 tx=0x7f41dc0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7457037428243d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41eeb48640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41d00a2b80 0x7f41d00a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f41d80bc620 tx=0x7f41d8087260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41e4ff9640 1 -- 172.21.15.143:0/997367123 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc004230 con 0x5654d78afbb0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41e4ff9640 1 -- 172.21.15.143:0/997367123 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41dc0043d0 con 0x5654d78afbb0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.015+0000 7f41e4ff9640 1 -- 172.21.15.143:0/997367123 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0109d0 con 0x5654d78afbb0 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 -- 172.21.15.143:0/997367123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 msgr2=0x5654d78b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78b1fa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009fa0 tx=0x7f41dc0097b0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 -- 172.21.15.143:0/997367123 shutdown_connections 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78b1fa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 --2- 172.21.15.143:0/997367123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 -- 172.21.15.143:0/997367123 >> 172.21.15.143:0/997367123 conn(0x5654d78a0f60 msgr2=0x5654d78a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 -- 172.21.15.143:0/997367123 shutdown_connections 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.016+0000 7f41f2984740 1 -- 172.21.15.143:0/997367123 wait complete. 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 -- start start 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 Processor -- start 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654d78b2780 con 0x5654d78afbb0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5654d77c5f20 con 0x5654d78a49d0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654d78b2900 con 0x5654d78a52f0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41f2984740 10 client.0 _ll_register_callbacks cb 0x5654d778f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41428/0 (socket says 172.21.15.143:41428) 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 -- 172.21.15.143:0/2264161526 learned_addr learned my addr 172.21.15.143:0/2264161526 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41ee347640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 -- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 msgr2=0x5654d78ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 -- 172.21.15.143:0/2264161526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 msgr2=0x5654d78af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 -- 172.21.15.143:0/2264161526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d757c2b0 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41ee347640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41edb46640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78acd30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f41dc013fd0 tx=0x7f41dc0105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5319d6f6be79e43d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.017+0000 7f41cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41cdffb640 1 -- 172.21.15.143:0/2264161526 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc005030 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41cdffb640 1 -- 172.21.15.143:0/2264161526 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f41dc0051d0 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41dc0043d0 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41cdffb640 1 -- 172.21.15.143:0/2264161526 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41dc032050 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 10 client.15273 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654d78be0f0 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41cdffb640 1 -- 172.21.15.143:0/2264161526 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41dc0043d0 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41cdffb640 1 -- 172.21.15.143:0/2264161526 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f41dc0109e0 con 0x5654d78afbb0 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 10 client.15273 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:11.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 20 client.15273 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d78c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 4 client.15273 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654d7818600 con 0x5654d78c05b0 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.018+0000 7f41ee347640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d78c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.019+0000 7f41ee347640 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d78c1a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bafcd989706b0b3d server_cookie=1fa4ccccfa707c5e in_seq=0 out_seq=0 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.019+0000 7f41cdffb640 10 client.15273 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41cdffb640 1 -- 172.21.15.143:0/2264161526 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654d7818600 con 0x5654d78c05b0 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41cdffb640 10 client.15273 handle_command_reply: tid=0 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 client.15273 shutdown 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 msgr2=0x5654d78acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78acd30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f41dc013fd0 tx=0x7f41dc0105b0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 shutdown_connections 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78acd30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d78c1a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41cdffb640 0 client.15273 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 --2- 172.21.15.143:0/2264161526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 >> 172.21.15.143:0/2264161526 conn(0x5654d78a0f60 msgr2=0x5654d78a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 shutdown_connections 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 1 -- 172.21.15.143:0/2264161526 wait complete. 2024-09-10T10:45:11.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.020+0000 7f41f2984740 20 client.15273 trim_cache size 0 max 16384 2024-09-10T10:45:11.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.037+0000 7f41f2984740 1 Processor -- start 2024-09-10T10:45:11.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.037+0000 7f41f2984740 1 -- start start 2024-09-10T10:45:11.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.037+0000 7f41f2984740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.037+0000 7f41f2984740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41f2984740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78ab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41f2984740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654d78bfd60 con 0x5654d78afbb0 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41f2984740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5654d78c0430 con 0x5654d78a49d0 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41f2984740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654d78c0b00 con 0x5654d78a52f0 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41436/0 (socket says 172.21.15.143:41436) 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 -- 172.21.15.143:0/1522486315 learned_addr learned my addr 172.21.15.143:0/1522486315 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41ee347640 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 -- 172.21.15.143:0/1522486315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 msgr2=0x5654d78ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 -- 172.21.15.143:0/1522486315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 msgr2=0x5654d78ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78ac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 -- 172.21.15.143:0/1522486315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d78a1b60 con 0x5654d78afbb0 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41edb46640 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78ab640 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f41dc015010 tx=0x7f41dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854816ca6a8248a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41cffff640 1 -- 172.21.15.143:0/1522486315 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc009630 con 0x5654d78afbb0 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41cffff640 1 -- 172.21.15.143:0/1522486315 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41dc004380 con 0x5654d78afbb0 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.038+0000 7f41cffff640 1 -- 172.21.15.143:0/1522486315 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0035f0 con 0x5654d78afbb0 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 -- 172.21.15.143:0/1522486315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 msgr2=0x5654d78ab640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78ab640 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f41dc015010 tx=0x7f41dc005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 -- 172.21.15.143:0/1522486315 shutdown_connections 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78afbb0 0x5654d78ab640 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78a52f0 0x5654d78ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 --2- 172.21.15.143:0/1522486315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a49d0 0x5654d78ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 -- 172.21.15.143:0/1522486315 >> 172.21.15.143:0/1522486315 conn(0x5654d78a0f60 msgr2=0x5654d78ac7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 -- 172.21.15.143:0/1522486315 shutdown_connections 2024-09-10T10:45:11.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 -- 172.21.15.143:0/1522486315 wait complete. 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 -- start start 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.039+0000 7f41f2984740 1 Processor -- start 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 0x5654d78a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a52f0 0x5654d78a9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78afbb0 0x5654d7928cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654d78c18a0 con 0x5654d78a49d0 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5654d78c11d0 con 0x5654d78a52f0 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5654d78c1f70 con 0x5654d78afbb0 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41f2984740 10 client.0 _ll_register_callbacks cb 0x7f41e00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 0x5654d78a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 0x5654d78a9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41450/0 (socket says 172.21.15.143:41450) 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 -- 172.21.15.143:0/154389483 learned_addr learned my addr 172.21.15.143:0/154389483 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:11.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78afbb0 0x5654d7928cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41ee347640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a52f0 0x5654d78a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 -- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78afbb0 msgr2=0x5654d7928cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78afbb0 0x5654d7928cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 -- 172.21.15.143:0/154389483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a52f0 msgr2=0x5654d78a9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a52f0 0x5654d78a9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 -- 172.21.15.143:0/154389483 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d79291f0 con 0x5654d78a49d0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41ee347640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a52f0 0x5654d78a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41edb46640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 0x5654d78a9240 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009ba0 tx=0x7f41dc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bcb7b26690eb30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.040+0000 7f41cdffb640 1 -- 172.21.15.143:0/154389483 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0035f0 con 0x5654d78a49d0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41cdffb640 1 -- 172.21.15.143:0/154389483 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f41dc003790 con 0x5654d78a49d0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41dc004380 con 0x5654d78a49d0 2024-09-10T10:45:11.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41cdffb640 1 -- 172.21.15.143:0/154389483 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41dc049050 con 0x5654d78a49d0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 10 client.15279 fetch_fsmap learned FSMap version 28 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654d792a420 con 0x5654d78a49d0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41cdffb640 1 -- 172.21.15.143:0/154389483 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41dc004380 con 0x5654d78a49d0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41cdffb640 1 -- 172.21.15.143:0/154389483 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f41dc02b820 con 0x5654d78a49d0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 20 client.15279 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d792da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 4 client.15279 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "status"}) -- 0x5654d7812b10 con 0x5654d78c05b0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.041+0000 7f41ee347640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d792da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41ee347640 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d792da40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83eff0186b3a01d0 server_cookie=db1b2e1cbe779c9f in_seq=0 out_seq=0 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41cdffb640 10 client.15279 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:11.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41cdffb640 1 -- 172.21.15.143:0/154389483 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5654d7812b10 con 0x5654d78c05b0 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41cdffb640 10 client.15279 handle_command_reply: tid=0 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 client.15279 shutdown 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 msgr2=0x5654d78a9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 0x5654d78a9240 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009ba0 tx=0x7f41dc004530 comp rx=0 tx=0).stop 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 shutdown_connections 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5654d78c05b0 0x5654d792da40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5654d78afbb0 0x5654d7928cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654d78a52f0 0x5654d78a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 --2- 172.21.15.143:0/154389483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654d78a49d0 0x5654d78a9240 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41cdffb640 0 client.15279 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 >> 172.21.15.143:0/154389483 conn(0x5654d78a0f60 msgr2=0x5654d78ab0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 shutdown_connections 2024-09-10T10:45:11.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 1 -- 172.21.15.143:0/154389483 wait complete. 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.042+0000 7f41f2984740 20 client.15279 trim_cache size 0 max 16384 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24502, 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 136.79784905899999, 2024-09-10T10:45:11.053 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 28, 2024-09-10T10:45:11.054 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:11.054 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:45:11.054 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 156.94580193499999 2024-09-10T10:45:11.054 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:11.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41d00a2b80 msgr2=0x7f41d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41d00a2b80 0x7f41d00a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f41d80bc620 tx=0x7f41d8087260 comp rx=0 tx=0).stop 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 msgr2=0x7f41e819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e819cc60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f41d8004af0 tx=0x7f41d8031f70 comp rx=0 tx=0).stop 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 shutdown_connections 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41d00a2b80 0x7f41d00a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e8102ef0 0x7f41e819cc60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41e8102570 0x7f41e81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 --2- 172.21.15.143:0/1891885094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41e8100e20 0x7f41e8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 >> 172.21.15.143:0/1891885094 conn(0x7f41e80fc7d0 msgr2=0x7f41e806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 shutdown_connections 2024-09-10T10:45:11.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:11.043+0000 7f41f05ab640 1 -- 172.21.15.143:0/1891885094 wait complete. 2024-09-10T10:45:11.764 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T10:45:11.862 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T10:45:11.862 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T10:45:11.862 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T10:45:11.862 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T10:45:11.886 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T10:45:11.886 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-10T10:45:11.886 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-10T10:45:11.887 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:45:11.887 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:11.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:11.887 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-09-10T10:45:11.903 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T10:45:11.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:11.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:45:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:11.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-09-10T10:45:12.062 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T10:45:12.062 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T10:45:12.062 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T10:45:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:12.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:12 smithi143 ceph-mon[31057]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:45:12.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.813+0000 7f1935ce3640 1 -- 172.21.15.143:0/4210164215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 msgr2=0x7f1930074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:12.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.813+0000 7f1935ce3640 1 --2- 172.21.15.143:0/4210164215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f1930074970 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1920009900 tx=0x7f192002a480 comp rx=0 tx=0).stop 2024-09-10T10:45:12.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 -- 172.21.15.143:0/4210164215 shutdown_connections 2024-09-10T10:45:12.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 --2- 172.21.15.143:0/4210164215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1930112b90 0x7f1930114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 --2- 172.21.15.143:0/4210164215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1930074eb0 0x7f1930112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 --2- 172.21.15.143:0/4210164215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f1930074970 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 -- 172.21.15.143:0/4210164215 >> 172.21.15.143:0/4210164215 conn(0x7f19300fd8e0 msgr2=0x7f19300ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 -- 172.21.15.143:0/4210164215 shutdown_connections 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 -- 172.21.15.143:0/4210164215 wait complete. 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 Processor -- start 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.814+0000 7f1935ce3640 1 -- start start 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f19301a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1930074eb0 0x7f19301a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1930112b90 0x7f193019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1930118420 con 0x7f1930074590 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19301182a0 con 0x7f1930074eb0 2024-09-10T10:45:12.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19301185a0 con 0x7f1930112b90 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f19301a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f19301a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41474/0 (socket says 172.21.15.143:41474) 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 -- 172.21.15.143:0/2366119555 learned_addr learned my addr 172.21.15.143:0/2366119555 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192ffff640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1930112b90 0x7f193019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1927fff640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1930074eb0 0x7f19301a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 -- 172.21.15.143:0/2366119555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1930112b90 msgr2=0x7f193019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1930112b90 0x7f193019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 -- 172.21.15.143:0/2366119555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1930074eb0 msgr2=0x7f19301a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1930074eb0 0x7f19301a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 -- 172.21.15.143:0/2366119555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193019ef30 con 0x7f1930074590 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192f7fe640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f19301a4610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1920005940 tx=0x7f1920004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192003f030 con 0x7f1930074590 2024-09-10T10:45:12.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193019f1c0 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1920003aa0 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.815+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920035070 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.816+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301ab470 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.817+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930075cd0 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.818+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1920042030 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.818+0000 7f192d7fa640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.818+0000 7f1927fff640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.818+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f19200efe60 con 0x7f1930074590 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.819+0000 7f1927fff640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f191c0a2b80 0x7f191c0a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1918005fd0 tx=0x7f1918005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:12.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:12.821+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19200bca60 con 0x7f1930074590 2024-09-10T10:45:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:12 smithi178 ceph-mon[34813]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:45:12.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:12 smithi144 ceph-mon[34230]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:45:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.025+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1930104ff0 con 0x7f1930074590 2024-09-10T10:45:13.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.026+0000 7f192d7fa640 1 -- 172.21.15.143:0/2366119555 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f19200bc360 con 0x7f1930074590 2024-09-10T10:45:13.027 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f191c0a2b80 msgr2=0x7f191c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f191c0a2b80 0x7f191c0a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1918005fd0 tx=0x7f1918005850 comp rx=0 tx=0).stop 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 msgr2=0x7f19301a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f19301a4610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1920005940 tx=0x7f1920004270 comp rx=0 tx=0).stop 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 shutdown_connections 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f191c0a2b80 0x7f191c0a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1930112b90 0x7f193019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1930074eb0 0x7f19301a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 --2- 172.21.15.143:0/2366119555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1930074590 0x7f19301a4610 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 >> 172.21.15.143:0/2366119555 conn(0x7f19300fd8e0 msgr2=0x7f19300ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 shutdown_connections 2024-09-10T10:45:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13.028+0000 7f1935ce3640 1 -- 172.21.15.143:0/2366119555 wait complete. 2024-09-10T10:45:13.732 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:13 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2366119555' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:45:13.762 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:13.762 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:13.779 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42177sec preferred_lft 42177sec 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link noprefixroute 2024-09-10T10:45:13.780 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:13.781 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:45:13.781 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:45:13.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:13.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:13.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:13.918 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:45:13.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:13 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2366119555' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:45:13.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:13 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2366119555' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:45:13.942 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-09-10T10:45:13.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:13.944 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-09-10T10:45:14.002 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-09-10T10:45:14.002 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:45:14.002 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:45:14.002 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:45:14.002 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:45:14.002 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:14.002 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:45:14.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:45:14.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:45:14.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:45:14.003 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:45:14.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:14.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:14.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:14.128 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-09-10T10:45:14.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:14.143 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-09-10T10:45:14.199 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:14.199 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:45:14.199 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:14.199 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T10:45:14.200 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:45:14.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:14.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:14.317 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T10:45:14.317 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:45:14.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:14.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:14.431 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:14.431 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-09-10T10:45:14.432 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-09-10T10:45:14.432 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:45:14.432 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-09-10T10:45:14.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:14.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:14.509 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-10T10:45:14.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:14.509 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:14.525 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:14.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:14.525 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:14.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-09-10T10:45:14.642 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:14 smithi143 ceph-mon[31057]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:45:14.646 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:45:14.702 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-09-10T10:45:14.702 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-09-10T10:45:14.702 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-09-10T10:45:14.702 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-09-10T10:45:14.702 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-09-10T10:45:14.702 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/selinux 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:45:14.703 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/9e89ccedca6f47585dae63a59261f6f61733668e8115ab8b06300dd6f90553cf/merged 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d902d0e612869f50dcaf70af0a29ce6a962ce9b6ad9a487a1ea15be5a8ccc46d/merged 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/0 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/3630a32f2bfa1ab51d64cf01622e1bb6f9fe829548d8ce08923612bf11c650d2/merged 2024-09-10T10:45:14.704 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/a6f1284f5096bb30a5dcbb209d96bd0c5c09bcd9a096633246e0cbe1284bf312/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d085932431a90f4493f59f8d45bdfc2b4934e5e2172a80d753307d4ace13960c/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/5bfc047991fa2d929c5acb5f4cdf099ee6e60da7bd43e0c36380c4097934c550/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/91133b17bd7ed747cbdd8fa21d95faf3ab947dd653bac10bdd0c015f538690d5/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/663c3868f6ed2a49572cd5f7a9d9c56e72e624e74b1aeae55e74d952c1f3f204/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d932de3595f6eecb2d9a4b27dbcb2e366196f358310da0e6d403cae1e3436cab/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/0e903cd12a8e8e5e0917bc6f3258f0a62de2f70a6fb9b13edea3e889bb460d2e/merged 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:14.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:14.705 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-09-10T10:45:14.758 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T10:45:14.758 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73 --client_fs=cephfs) 2024-09-10T10:45:14.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-09-10T10:45:14.826 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:45:14.883 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-09-10T10:45:14.883 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-09-10T10:45:14.883 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-09-10T10:45:14.883 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/selinux 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-09-10T10:45:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/9e89ccedca6f47585dae63a59261f6f61733668e8115ab8b06300dd6f90553cf/merged 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d902d0e612869f50dcaf70af0a29ce6a962ce9b6ad9a487a1ea15be5a8ccc46d/merged 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/0 2024-09-10T10:45:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/3630a32f2bfa1ab51d64cf01622e1bb6f9fe829548d8ce08923612bf11c650d2/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/a6f1284f5096bb30a5dcbb209d96bd0c5c09bcd9a096633246e0cbe1284bf312/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d085932431a90f4493f59f8d45bdfc2b4934e5e2172a80d753307d4ace13960c/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/5bfc047991fa2d929c5acb5f4cdf099ee6e60da7bd43e0c36380c4097934c550/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/91133b17bd7ed747cbdd8fa21d95faf3ab947dd653bac10bdd0c015f538690d5/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/663c3868f6ed2a49572cd5f7a9d9c56e72e624e74b1aeae55e74d952c1f3f204/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d932de3595f6eecb2d9a4b27dbcb2e366196f358310da0e6d403cae1e3436cab/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/0e903cd12a8e8e5e0917bc6f3258f0a62de2f70a6fb9b13edea3e889bb460d2e/merged 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:14.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:14.887 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-09-10T10:45:14.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-09-10T10:45:14.883+0000 7fa29952b480 -1 init, newargv = 0x55f9fdeffab0 newargc=15 2024-09-10T10:45:14.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[85758]: starting ceph client 2024-09-10T10:45:14.904 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[85758]: starting fuse 2024-09-10T10:45:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:14 smithi178 ceph-mon[34813]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:45:14.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:14 smithi144 ceph-mon[34230]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:45:16.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-09-10T10:45:16.033 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-09-10T10:45:16.091 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/selinux 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-09-10T10:45:16.092 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/9e89ccedca6f47585dae63a59261f6f61733668e8115ab8b06300dd6f90553cf/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d902d0e612869f50dcaf70af0a29ce6a962ce9b6ad9a487a1ea15be5a8ccc46d/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/0 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/3630a32f2bfa1ab51d64cf01622e1bb6f9fe829548d8ce08923612bf11c650d2/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/a6f1284f5096bb30a5dcbb209d96bd0c5c09bcd9a096633246e0cbe1284bf312/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d085932431a90f4493f59f8d45bdfc2b4934e5e2172a80d753307d4ace13960c/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/5bfc047991fa2d929c5acb5f4cdf099ee6e60da7bd43e0c36380c4097934c550/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/91133b17bd7ed747cbdd8fa21d95faf3ab947dd653bac10bdd0c015f538690d5/merged 2024-09-10T10:45:16.093 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/663c3868f6ed2a49572cd5f7a9d9c56e72e624e74b1aeae55e74d952c1f3f204/merged 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/d932de3595f6eecb2d9a4b27dbcb2e366196f358310da0e6d403cae1e3436cab/merged 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/containers/storage/overlay/0e903cd12a8e8e5e0917bc6f3258f0a62de2f70a6fb9b13edea3e889bb460d2e/merged 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:16.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:16.094 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-09-10T10:45:16.147 INFO:teuthology.orchestra.run.smithi143.stdout:120 2024-09-10T10:45:16.147 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T10:45:16.147 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-09-10T10:45:16.147 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-09-10T10:45:16.147 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-09-10T10:45:16.148 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-09-10T10:45:16.149 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-09-10T10:45:16.265 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.85758.asok 2024-09-10T10:45:16.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:16.276 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85758.asok 2024-09-10T10:45:16.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:16.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85758.asok status 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "85758", 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73" 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 15291, 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 15291 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 478985484 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 478985484 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.15291 192.168.0.1:0/478985484", 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/478985484", 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 5, 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 28, 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 89, 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:16.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:16.418 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-09-10T10:45:16.418 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:16.434 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-09-10T10:45:16.434 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:16.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:16.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:16.501 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T10:45:16.511 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:45:16.511 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T10:45:16.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-10T10:45:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 -- 172.21.15.143:0/3977812829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc102610 msgr2=0x7fafdc108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 --2- 172.21.15.143:0/3977812829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc102610 0x7fafdc108ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009900 tx=0x7fafd402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 -- 172.21.15.143:0/3977812829 shutdown_connections 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 --2- 172.21.15.143:0/3977812829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc102610 0x7fafdc108ea0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 --2- 172.21.15.143:0/3977812829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc101c50 0x7fafdc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 --2- 172.21.15.143:0/3977812829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafdc100a60 0x7fafdc100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 -- 172.21.15.143:0/3977812829 >> 172.21.15.143:0/3977812829 conn(0x7fafdc0fc1e0 msgr2=0x7fafdc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 -- 172.21.15.143:0/3977812829 shutdown_connections 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.642+0000 7fafe1d26640 1 -- 172.21.15.143:0/3977812829 wait complete. 2024-09-10T10:45:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 Processor -- start 2024-09-10T10:45:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 -- start start 2024-09-10T10:45:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 0x7fafdc197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc101c50 0x7fafdc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafdc102610 0x7fafdc19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10c570 con 0x7fafdc100a60 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10c3f0 con 0x7fafdc101c50 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafe1d26640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10c6f0 con 0x7fafdc102610 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 0x7fafdc197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 0x7fafdc197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42250/0 (socket says 172.21.15.143:42250) 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 -- 172.21.15.143:0/2145001922 learned_addr learned my addr 172.21.15.143:0/2145001922 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdbfff640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafdc102610 0x7fafdc19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdaffd640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc101c50 0x7fafdc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 -- 172.21.15.143:0/2145001922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafdc102610 msgr2=0x7fafdc19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafdc102610 0x7fafdc19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 -- 172.21.15.143:0/2145001922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc101c50 msgr2=0x7fafdc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.643+0000 7fafdb7fe640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc101c50 0x7fafdc198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafdb7fe640 1 -- 172.21.15.143:0/2145001922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc19f860 con 0x7fafdc100a60 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafdb7fe640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 0x7fafdc197c50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00ccb0 tx=0x7fafcc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc003b70 con 0x7fafdc100a60 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc19fb50 con 0x7fafdc100a60 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafcc00ae50 con 0x7fafdc100a60 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00f6f0 con 0x7fafdc100a60 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.644+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a00a0 con 0x7fafdc100a60 2024-09-10T10:45:16.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.645+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc1a0270 con 0x7fafdc100a60 2024-09-10T10:45:16.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.646+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fafcc004000 con 0x7fafdc100a60 2024-09-10T10:45:16.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.647+0000 7fafd8ff9640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fafc40a2c30 0x7fafc40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.647+0000 7fafdaffd640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fafc40a2c30 0x7fafc40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.647+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fafcc0cf6a0 con 0x7fafdc100a60 2024-09-10T10:45:16.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.647+0000 7fafdaffd640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fafc40a2c30 0x7fafc40a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafd0005fd0 tx=0x7fafd0009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.651+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafcc014070 con 0x7fafdc100a60 2024-09-10T10:45:16.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:16 smithi143 ceph-mon[31057]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T10:45:16.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:16.855+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fafdc1a0bb0 con 0x7fafdc100a60 2024-09-10T10:45:16.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:16 smithi178 ceph-mon[34813]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T10:45:16.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:16 smithi144 ceph-mon[34230]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T10:45:17.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.576+0000 7fafd8ff9640 1 -- 172.21.15.143:0/2145001922 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fafcc09c2a0 con 0x7fafdc100a60 2024-09-10T10:45:17.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fafc40a2c30 msgr2=0x7fafc40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fafc40a2c30 0x7fafc40a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafd0005fd0 tx=0x7fafd0009470 comp rx=0 tx=0).stop 2024-09-10T10:45:17.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 msgr2=0x7fafdc197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 0x7fafdc197c50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00ccb0 tx=0x7fafcc00a620 comp rx=0 tx=0).stop 2024-09-10T10:45:17.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 shutdown_connections 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fafc40a2c30 0x7fafc40a50f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafdc102610 0x7fafdc19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc101c50 0x7fafdc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 --2- 172.21.15.143:0/2145001922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafdc100a60 0x7fafdc197c50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 >> 172.21.15.143:0/2145001922 conn(0x7fafdc0fc1e0 msgr2=0x7fafdc0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 shutdown_connections 2024-09-10T10:45:17.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.579+0000 7fafe1d26640 1 -- 172.21.15.143:0/2145001922 wait complete. 2024-09-10T10:45:17.592 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-10T10:45:17.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 -- 172.21.15.143:0/1277054972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e4101d80 msgr2=0x7f77e4108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 --2- 172.21.15.143:0/1277054972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e4101d80 0x7f77e4108610 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009ca0 tx=0x7f77dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 -- 172.21.15.143:0/1277054972 shutdown_connections 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 --2- 172.21.15.143:0/1277054972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e4101d80 0x7f77e4108610 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 --2- 172.21.15.143:0/1277054972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 0x7f77e4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 --2- 172.21.15.143:0/1277054972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e41001d0 0x7f77e41005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 -- 172.21.15.143:0/1277054972 >> 172.21.15.143:0/1277054972 conn(0x7f77e40fb970 msgr2=0x7f77e40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 -- 172.21.15.143:0/1277054972 shutdown_connections 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 -- 172.21.15.143:0/1277054972 wait complete. 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 Processor -- start 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.694+0000 7f77ea518640 1 -- start start 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 0x7f77e4109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 0x7f77e419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e4101d80 0x7f77e419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410c590 con 0x7f77e41001d0 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410c410 con 0x7f77e41013c0 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410c710 con 0x7f77e4101d80 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 0x7f77e4109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 0x7f77e4109dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42262/0 (socket says 172.21.15.143:42262) 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 -- 172.21.15.143:0/145209615 learned_addr learned my addr 172.21.15.143:0/145209615 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e37fe640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 0x7f77e419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e8ab5640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e4101d80 0x7f77e419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 -- 172.21.15.143:0/145209615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e4101d80 msgr2=0x7f77e419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e4101d80 0x7f77e419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 -- 172.21.15.143:0/145209615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 msgr2=0x7f77e419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 0x7f77e419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 -- 172.21.15.143:0/145209615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e419f8e0 con 0x7f77e41001d0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e8ab5640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e4101d80 0x7f77e419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e37fe640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 0x7f77e419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e3fff640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 0x7f77e4109dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c970 tx=0x7f77d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4004220 con 0x7f77e41001d0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e419fb70 con 0x7f77e41001d0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77d40043c0 con 0x7f77e41001d0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e41a00c0 con 0x7f77e41001d0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.695+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d401d060 con 0x7f77e41001d0 2024-09-10T10:45:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.696+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e41074f0 con 0x7f77e41001d0 2024-09-10T10:45:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.698+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f77d40079b0 con 0x7f77e41001d0 2024-09-10T10:45:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.698+0000 7f77e17fa640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77cc0a2c30 0x7f77cc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.698+0000 7f77e37fe640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77cc0a2c30 0x7f77cc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.698+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f77d40cf590 con 0x7f77e41001d0 2024-09-10T10:45:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.699+0000 7f77e37fe640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77cc0a2c30 0x7f77cc0a50f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77e4109b40 tx=0x7f77d8005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.702+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77d4014030 con 0x7f77e41001d0 2024-09-10T10:45:17.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.893+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f77e4062790 con 0x7f77cc0a2c30 2024-09-10T10:45:17.899 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:17 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2145001922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T10:45:17.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.908+0000 7f77e17fa640 1 -- 172.21.15.143:0/145209615 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f77e4062790 con 0x7f77cc0a2c30 2024-09-10T10:45:17.910 INFO:teuthology.orchestra.run.smithi143.stdout:{} 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77cc0a2c30 msgr2=0x7f77cc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77cc0a2c30 0x7f77cc0a50f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77e4109b40 tx=0x7f77d8005ec0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 msgr2=0x7f77e4109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 0x7f77e4109dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c970 tx=0x7f77d400ce30 comp rx=0 tx=0).stop 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 shutdown_connections 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f77cc0a2c30 0x7f77cc0a50f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e4101d80 0x7f77e419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e41013c0 0x7f77e419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 --2- 172.21.15.143:0/145209615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77e41001d0 0x7f77e4109dd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 >> 172.21.15.143:0/145209615 conn(0x7f77e40fb970 msgr2=0x7f77e41046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:17.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 shutdown_connections 2024-09-10T10:45:17.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:17.911+0000 7f77ea518640 1 -- 172.21.15.143:0/145209615 wait complete. 2024-09-10T10:45:17.923 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T10:45:17.934 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T10:45:17.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:45:17.953 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:17 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2145001922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T10:45:17.953 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:17 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2145001922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T10:45:18.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.051+0000 7fd668ded640 1 -- 172.21.15.143:0/4055698530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 msgr2=0x7fd664102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.051+0000 7fd668ded640 1 --2- 172.21.15.143:0/4055698530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664102080 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009900 tx=0x7fd64c02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 -- 172.21.15.143:0/4055698530 shutdown_connections 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 --2- 172.21.15.143:0/4055698530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6641025c0 0x7fd664108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 --2- 172.21.15.143:0/4055698530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664102080 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 --2- 172.21.15.143:0/4055698530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd664100a10 0x7fd664100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 -- 172.21.15.143:0/4055698530 >> 172.21.15.143:0/4055698530 conn(0x7fd6640fc190 msgr2=0x7fd6640fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 -- 172.21.15.143:0/4055698530 shutdown_connections 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 -- 172.21.15.143:0/4055698530 wait complete. 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 Processor -- start 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.052+0000 7fd668ded640 1 -- start start 2024-09-10T10:45:18.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd664100a10 0x7fd664197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6641025c0 0x7fd66419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410c540 con 0x7fd664101c00 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410c3c0 con 0x7fd664100a10 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410c6c0 con 0x7fd6641025c0 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42296/0 (socket says 172.21.15.143:42296) 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 -- 172.21.15.143:0/2808940325 learned_addr learned my addr 172.21.15.143:0/2808940325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd662d9d640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6641025c0 0x7fd66419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd66259c640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd664100a10 0x7fd664197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 -- 172.21.15.143:0/2808940325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6641025c0 msgr2=0x7fd66419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6641025c0 0x7fd66419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 -- 172.21.15.143:0/2808940325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd664100a10 msgr2=0x7fd664197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd664100a10 0x7fd664197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 -- 172.21.15.143:0/2808940325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66419fa20 con 0x7fd664101c00 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd66259c640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd664100a10 0x7fd664197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd661d9b640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664198340 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd64c004770 tx=0x7fd64c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c03f030 con 0x7fd664101c00 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd64c003cb0 con 0x7fd664101c00 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66419fcb0 con 0x7fd664101c00 2024-09-10T10:45:18.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c035070 con 0x7fd664101c00 2024-09-10T10:45:18.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.053+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6641a01a0 con 0x7fd664101c00 2024-09-10T10:45:18.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.054+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6641a0370 con 0x7fd664101c00 2024-09-10T10:45:18.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.055+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd64c042030 con 0x7fd664101c00 2024-09-10T10:45:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.056+0000 7fd65b7fe640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6500a2b90 0x7fd6500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.056+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd64c0efd30 con 0x7fd664101c00 2024-09-10T10:45:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.056+0000 7fd66259c640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6500a2b90 0x7fd6500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.057+0000 7fd66259c640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6500a2b90 0x7fd6500a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6540058c0 tx=0x7fd654005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.060+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd64c0bc930 con 0x7fd664101c00 2024-09-10T10:45:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.226+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6641a0700 con 0x7fd664101c00 2024-09-10T10:45:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.228+0000 7fd65b7fe640 1 -- 172.21.15.143:0/2808940325 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7fd64c0bc230 con 0x7fd664101c00 2024-09-10T10:45:18.229 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:18.230 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6500a2b90 msgr2=0x7fd6500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6500a2b90 0x7fd6500a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6540058c0 tx=0x7fd654005850 comp rx=0 tx=0).stop 2024-09-10T10:45:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 msgr2=0x7fd664198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664198340 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd64c004770 tx=0x7fd64c0042f0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 shutdown_connections 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd6500a2b90 0x7fd6500a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6641025c0 0x7fd66419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd664101c00 0x7fd664198340 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 --2- 172.21.15.143:0/2808940325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd664100a10 0x7fd664197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 >> 172.21.15.143:0/2808940325 conn(0x7fd6640fc190 msgr2=0x7fd66406a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 shutdown_connections 2024-09-10T10:45:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.230+0000 7fd668ded640 1 -- 172.21.15.143:0/2808940325 wait complete. 2024-09-10T10:45:18.241 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:45:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:45:17.897+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:18.242 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:45:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:45:17.897+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:18.242 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:45:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:45:17.897+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:18.242 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:45:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:45:17.897+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:18.242 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:45:17 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:45:17.897+0000 7f53d691e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:18.243 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:38:11.637482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:42:49.489312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:42:52.085668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:42:58.507118+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T10:45:18.244 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:45:18.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 -- 172.21.15.143:0/1192775539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 msgr2=0x7ff65c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 --2- 172.21.15.143:0/1192775539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c100e10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff650009900 tx=0x7ff65002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 -- 172.21.15.143:0/1192775539 shutdown_connections 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 --2- 172.21.15.143:0/1192775539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff65c1025c0 0x7ff65c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 --2- 172.21.15.143:0/1192775539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 0x7ff65c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 --2- 172.21.15.143:0/1192775539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c100e10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 -- 172.21.15.143:0/1192775539 >> 172.21.15.143:0/1192775539 conn(0x7ff65c0fc1b0 msgr2=0x7ff65c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 -- 172.21.15.143:0/1192775539 shutdown_connections 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.345+0000 7ff663082640 1 -- 172.21.15.143:0/1192775539 wait complete. 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 Processor -- start 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 -- start start 2024-09-10T10:45:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 0x7ff65c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff65c1025c0 0x7ff65c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10d690 con 0x7ff65c100a10 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10d510 con 0x7ff65c1025c0 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff663082640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10d810 con 0x7ff65c101c00 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42312/0 (socket says 172.21.15.143:42312) 2024-09-10T10:45:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 -- 172.21.15.143:0/940960195 learned_addr learned my addr 172.21.15.143:0/940960195 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:18.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff66161f640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff65c1025c0 0x7ff65c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff65bfff640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 0x7ff65c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 -- 172.21.15.143:0/940960195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 msgr2=0x7ff65c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 0x7ff65c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 -- 172.21.15.143:0/940960195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff65c1025c0 msgr2=0x7ff65c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff65c1025c0 0x7ff65c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 -- 172.21.15.143:0/940960195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c19f9f0 con 0x7ff65c100a10 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff65bfff640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 0x7ff65c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.346+0000 7ff660e1e640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c197dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff6500314e0 tx=0x7ff650004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.347+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65003f030 con 0x7ff65c100a10 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.347+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff650003aa0 con 0x7ff65c100a10 2024-09-10T10:45:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.347+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650035070 con 0x7ff65c100a10 2024-09-10T10:45:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.347+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c19fc80 con 0x7ff65c100a10 2024-09-10T10:45:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.347+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1a01c0 con 0x7ff65c100a10 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.348+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c107f40 con 0x7ff65c100a10 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.349+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff650042030 con 0x7ff65c100a10 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.350+0000 7ff659ffb640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff64c0a2c10 0x7ff64c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.350+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff6500efea0 con 0x7ff65c100a10 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.350+0000 7ff65bfff640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff64c0a2c10 0x7ff64c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.350+0000 7ff65bfff640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff64c0a2c10 0x7ff64c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff65c1992e0 tx=0x7ff648005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.353+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6500bcaa0 con 0x7ff65c100a10 2024-09-10T10:45:18.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.519+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff65c02d970 con 0x7ff65c100a10 2024-09-10T10:45:18.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.519+0000 7ff659ffb640 1 -- 172.21.15.143:0/940960195 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ff6500bc3a0 con 0x7ff65c100a10 2024-09-10T10:45:18.521 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff64c0a2c10 msgr2=0x7ff64c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff64c0a2c10 0x7ff64c0a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff65c1992e0 tx=0x7ff648005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 msgr2=0x7ff65c197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c197dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff6500314e0 tx=0x7ff650004270 comp rx=0 tx=0).stop 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 shutdown_connections 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff64c0a2c10 0x7ff64c0a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff65c1025c0 0x7ff65c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff65c101c00 0x7ff65c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 --2- 172.21.15.143:0/940960195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff65c100a10 0x7ff65c197dd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.521+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 >> 172.21.15.143:0/940960195 conn(0x7ff65c0fc1b0 msgr2=0x7ff65c105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.522+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 shutdown_connections 2024-09-10T10:45:18.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.522+0000 7ff663082640 1 -- 172.21.15.143:0/940960195 wait complete. 2024-09-10T10:45:18.534 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T10:45:18.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.636+0000 7f5c615e4640 1 -- 172.21.15.143:0/3993759378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 msgr2=0x7f5c5c100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.636+0000 7f5c615e4640 1 --2- 172.21.15.143:0/3993759378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c100e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009900 tx=0x7f5c4c02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.636+0000 7f5c615e4640 1 -- 172.21.15.143:0/3993759378 shutdown_connections 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.636+0000 7f5c615e4640 1 --2- 172.21.15.143:0/3993759378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c1025b0 0x7f5c5c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.636+0000 7f5c615e4640 1 --2- 172.21.15.143:0/3993759378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c101bf0 0x7f5c5c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.636+0000 7f5c615e4640 1 --2- 172.21.15.143:0/3993759378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c100e00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- 172.21.15.143:0/3993759378 >> 172.21.15.143:0/3993759378 conn(0x7f5c5c0fc140 msgr2=0x7f5c5c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- 172.21.15.143:0/3993759378 shutdown_connections 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- 172.21.15.143:0/3993759378 wait complete. 2024-09-10T10:45:18.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 Processor -- start 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- start start 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101bf0 0x7f5c5c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c1025b0 0x7f5c5c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10c510 con 0x7f5c5c100a00 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10c390 con 0x7f5c5c101bf0 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.637+0000 7f5c615e4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10c690 con 0x7f5c5c1025b0 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42328/0 (socket says 172.21.15.143:42328) 2024-09-10T10:45:18.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 -- 172.21.15.143:0/679269552 learned_addr learned my addr 172.21.15.143:0/679269552 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5b7fe640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c1025b0 0x7f5c5c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 -- 172.21.15.143:0/679269552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c1025b0 msgr2=0x7f5c5c19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c1025b0 0x7f5c5c19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 -- 172.21.15.143:0/679269552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101bf0 msgr2=0x7f5c5c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101bf0 0x7f5c5c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 -- 172.21.15.143:0/679269552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c19f970 con 0x7f5c5c100a00 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5b7fe640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c1025b0 0x7f5c5c19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c5affd640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c197d50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009230 tx=0x7f5c4c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c03f030 con 0x7f5c5c100a00 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c19fc00 con 0x7f5c5c100a00 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c4c003cb0 con 0x7f5c5c100a00 2024-09-10T10:45:18.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c035070 con 0x7f5c5c100a00 2024-09-10T10:45:18.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.638+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a0140 con 0x7f5c5c100a00 2024-09-10T10:45:18.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.639+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c1a0310 con 0x7f5c5c100a00 2024-09-10T10:45:18.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.640+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5c4c042030 con 0x7f5c5c100a00 2024-09-10T10:45:18.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.641+0000 7f5c43fff640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c440a2b90 0x7f5c440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.641+0000 7f5c5a7fc640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c440a2b90 0x7f5c440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.641+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5c4c0efa90 con 0x7f5c5c100a00 2024-09-10T10:45:18.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.641+0000 7f5c5a7fc640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c440a2b90 0x7f5c440a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c199260 tx=0x7f5c50005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.645+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c4c0bc690 con 0x7f5c5c100a00 2024-09-10T10:45:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:18 smithi143 ceph-mon[31057]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-10T10:45:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2145001922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T10:45:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:18 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:45:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2808940325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:18.802 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:18 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/940960195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:45:18.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.813+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5c5c1a0600 con 0x7f5c5c100a00 2024-09-10T10:45:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.813+0000 7f5c43fff640 1 -- 172.21.15.143:0/679269552 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f5c4c0bbf90 con 0x7f5c5c100a00 2024-09-10T10:45:18.814 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c440a2b90 msgr2=0x7f5c440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c440a2b90 0x7f5c440a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c199260 tx=0x7f5c50005bf0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 msgr2=0x7f5c5c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c197d50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009230 tx=0x7f5c4c004340 comp rx=0 tx=0).stop 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 shutdown_connections 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5c440a2b90 0x7f5c440a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c5c1025b0 0x7f5c5c19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c5c101bf0 0x7f5c5c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 --2- 172.21.15.143:0/679269552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c5c100a00 0x7f5c5c197d50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 >> 172.21.15.143:0/679269552 conn(0x7f5c5c0fc140 msgr2=0x7f5c5c0693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 shutdown_connections 2024-09-10T10:45:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.815+0000 7f5c615e4640 1 -- 172.21.15.143:0/679269552 wait complete. 2024-09-10T10:45:18.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T10:45:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:18 smithi178 ceph-mon[34813]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-10T10:45:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:18 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2145001922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T10:45:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:18 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:45:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:18 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2808940325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:18.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:18 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/940960195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:45:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 -- 172.21.15.143:0/2367888982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c101c20 msgr2=0x7fe32c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 --2- 172.21.15.143:0/2367888982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c101c20 0x7fe32c1020a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe320009900 tx=0x7fe32002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 -- 172.21.15.143:0/2367888982 shutdown_connections 2024-09-10T10:45:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 --2- 172.21.15.143:0/2367888982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c1025e0 0x7fe32c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 --2- 172.21.15.143:0/2367888982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c101c20 0x7fe32c1020a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 --2- 172.21.15.143:0/2367888982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe32c100a30 0x7fe32c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 -- 172.21.15.143:0/2367888982 >> 172.21.15.143:0/2367888982 conn(0x7fe32c0fc1b0 msgr2=0x7fe32c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 -- 172.21.15.143:0/2367888982 shutdown_connections 2024-09-10T10:45:18.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 -- 172.21.15.143:0/2367888982 wait complete. 2024-09-10T10:45:18.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 Processor -- start 2024-09-10T10:45:18.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.929+0000 7fe33278d640 1 -- start start 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 0x7fe32c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe32c101c20 0x7fe32c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c1025e0 0x7fe32c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c10c560 con 0x7fe32c100a30 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c10c3e0 con 0x7fe32c1025e0 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c10c6e0 con 0x7fe32c101c20 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 0x7fe32c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 0x7fe32c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42348/0 (socket says 172.21.15.143:42348) 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 -- 172.21.15.143:0/3409569932 learned_addr learned my addr 172.21.15.143:0/3409569932 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:18.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32b7fe640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe32c101c20 0x7fe32c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe330d2a640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c1025e0 0x7fe32c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 -- 172.21.15.143:0/3409569932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe32c101c20 msgr2=0x7fe32c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe32c101c20 0x7fe32c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 -- 172.21.15.143:0/3409569932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c1025e0 msgr2=0x7fe32c19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c1025e0 0x7fe32c19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 -- 172.21.15.143:0/3409569932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c19fac0 con 0x7fe32c100a30 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe32bfff640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 0x7fe32c197ea0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00e9c0 tx=0x7fe31c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00cd80 con 0x7fe32c100a30 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31c0035f0 con 0x7fe32c100a30 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c19fdb0 con 0x7fe32c100a30 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.930+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c010690 con 0x7fe32c100a30 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.931+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1a0210 con 0x7fe32c100a30 2024-09-10T10:45:18.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.931+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32c1a04a0 con 0x7fe32c100a30 2024-09-10T10:45:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.933+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe31c0037a0 con 0x7fe32c100a30 2024-09-10T10:45:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.933+0000 7fe3297fa640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe3140a2c10 0x7fe3140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.933+0000 7fe32b7fe640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe3140a2c10 0x7fe3140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.933+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe31c0cf950 con 0x7fe32c100a30 2024-09-10T10:45:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.934+0000 7fe32b7fe640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe3140a2c10 0x7fe3140a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe3200314e0 tx=0x7fe320005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:18 smithi144 ceph-mon[34230]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-10T10:45:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:18 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2145001922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T10:45:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:18 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:45:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:18 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2808940325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:18 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/940960195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:45:18.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:18.936+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe31c09c550 con 0x7fe32c100a30 2024-09-10T10:45:19.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.104+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe32c1a0670 con 0x7fe32c100a30 2024-09-10T10:45:19.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.105+0000 7fe3297fa640 1 -- 172.21.15.143:0/3409569932 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fe31c09be50 con 0x7fe32c100a30 2024-09-10T10:45:19.106 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-09-10T10:45:19.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe3140a2c10 msgr2=0x7fe3140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe3140a2c10 0x7fe3140a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe3200314e0 tx=0x7fe320005b50 comp rx=0 tx=0).stop 2024-09-10T10:45:19.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 msgr2=0x7fe32c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 0x7fe32c197ea0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00e9c0 tx=0x7fe31c00ee80 comp rx=0 tx=0).stop 2024-09-10T10:45:19.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 shutdown_connections 2024-09-10T10:45:19.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe3140a2c10 0x7fe3140a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe32c1025e0 0x7fe32c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe32c101c20 0x7fe32c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 --2- 172.21.15.143:0/3409569932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32c100a30 0x7fe32c197ea0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 >> 172.21.15.143:0/3409569932 conn(0x7fe32c0fc1b0 msgr2=0x7fe32c06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 shutdown_connections 2024-09-10T10:45:19.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.107+0000 7fe33278d640 1 -- 172.21.15.143:0/3409569932 wait complete. 2024-09-10T10:45:19.119 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 -- 172.21.15.143:0/2070110108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 msgr2=0x7f70d0100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- 172.21.15.143:0/2070110108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0100e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009900 tx=0x7f70cc02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 -- 172.21.15.143:0/2070110108 shutdown_connections 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- 172.21.15.143:0/2070110108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d01025b0 0x7f70d0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- 172.21.15.143:0/2070110108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d0101bf0 0x7f70d0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- 172.21.15.143:0/2070110108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0100e00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 -- 172.21.15.143:0/2070110108 >> 172.21.15.143:0/2070110108 conn(0x7f70d00fc140 msgr2=0x7f70d00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 -- 172.21.15.143:0/2070110108 shutdown_connections 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 -- 172.21.15.143:0/2070110108 wait complete. 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 Processor -- start 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 -- start start 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d0101bf0 0x7f70d01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.223+0000 7f70d8578640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d01025b0 0x7f70d019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d8578640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f70d010d680 con 0x7f70d0100a00 2024-09-10T10:45:19.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d8578640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70d010d500 con 0x7f70d0101bf0 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d8578640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70d010d800 con 0x7f70d01025b0 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42362/0 (socket says 172.21.15.143:42362) 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 -- 172.21.15.143:0/1767113804 learned_addr learned my addr 172.21.15.143:0/1767113804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d5b13640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d0101bf0 0x7f70d01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6b15640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d01025b0 0x7f70d019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 -- 172.21.15.143:0/1767113804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d01025b0 msgr2=0x7f70d019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d01025b0 0x7f70d019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 -- 172.21.15.143:0/1767113804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d0101bf0 msgr2=0x7f70d01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d0101bf0 0x7f70d01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 -- 172.21.15.143:0/1767113804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d019f8e0 con 0x7f70d0100a00 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d5b13640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d0101bf0 0x7f70d01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6b15640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d01025b0 0x7f70d019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d6314640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0197ca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009230 tx=0x7f70cc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc03f030 con 0x7f70d0100a00 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70cc003cb0 con 0x7f70d0100a00 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d019fb70 con 0x7f70d0100a00 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc035070 con 0x7f70d0100a00 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.224+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d01a0060 con 0x7f70d0100a00 2024-09-10T10:45:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.225+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d01a0230 con 0x7f70d0100a00 2024-09-10T10:45:19.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.226+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f70cc042030 con 0x7f70d0100a00 2024-09-10T10:45:19.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.227+0000 7f70c77fe640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70bc0a2b90 0x7f70bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.227+0000 7f70d5b13640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70bc0a2b90 0x7f70bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.227+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f70cc0efed0 con 0x7f70d0100a00 2024-09-10T10:45:19.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.227+0000 7f70d5b13640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70bc0a2b90 0x7f70bc0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f70d01991a0 tx=0x7f70c0005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.230+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70cc0bcad0 con 0x7f70d0100a00 2024-09-10T10:45:19.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.397+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f70d01077e0 con 0x7f70d0100a00 2024-09-10T10:45:19.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.398+0000 7f70c77fe640 1 -- 172.21.15.143:0/1767113804 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f70cc0bc3d0 con 0x7f70d0100a00 2024-09-10T10:45:19.399 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-09-10T10:45:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70bc0a2b90 msgr2=0x7f70bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70bc0a2b90 0x7f70bc0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f70d01991a0 tx=0x7f70c0005d90 comp rx=0 tx=0).stop 2024-09-10T10:45:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 msgr2=0x7f70d0197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0197ca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009230 tx=0x7f70cc004340 comp rx=0 tx=0).stop 2024-09-10T10:45:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 shutdown_connections 2024-09-10T10:45:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f70bc0a2b90 0x7f70bc0a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d01025b0 0x7f70d019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70d0101bf0 0x7f70d01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 --2- 172.21.15.143:0/1767113804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70d0100a00 0x7f70d0197ca0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 >> 172.21.15.143:0/1767113804 conn(0x7f70d00fc140 msgr2=0x7f70d00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 shutdown_connections 2024-09-10T10:45:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:19.400+0000 7f70d8578640 1 -- 172.21.15.143:0/1767113804 wait complete. 2024-09-10T10:45:19.412 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T10:45:19.412 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:19.786 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:19 smithi143 ceph-mon[31057]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:19.787 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:19 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/679269552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.787 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:19 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3409569932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.787 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:19 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1767113804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:19 smithi178 ceph-mon[34813]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:19 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/679269552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:19 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3409569932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:19 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1767113804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:19 smithi144 ceph-mon[34230]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:19 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/679269552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:19 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3409569932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:45:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:19 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1767113804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:45:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 -- 172.21.15.143:0/3724723165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0101420 msgr2=0x7fe2b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/3724723165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0101420 0x7fe2b010d950 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009cd0 tx=0x7fe2a802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 -- 172.21.15.143:0/3724723165 shutdown_connections 2024-09-10T10:45:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/3724723165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0101420 0x7fe2b010d950 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/3724723165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0100a80 0x7fe2b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/3724723165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2b0106a80 0x7fe2b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 -- 172.21.15.143:0/3724723165 >> 172.21.15.143:0/3724723165 conn(0x7fe2b00fc7a0 msgr2=0x7fe2b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 -- 172.21.15.143:0/3724723165 shutdown_connections 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.194+0000 7fe2b4e86640 1 -- 172.21.15.143:0/3724723165 wait complete. 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 Processor -- start 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 -- start start 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2b0100a80 0x7fe2b0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0101420 0x7fe2b01974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 0x7fe2b0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0110d30 con 0x7fe2b0106a80 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0110bb0 con 0x7fe2b0101420 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.195+0000 7fe2b4e86640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0110eb0 con 0x7fe2b0100a80 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 0x7fe2b0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 0x7fe2b0197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42388/0 (socket says 172.21.15.143:42388) 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 -- 172.21.15.143:0/4102365809 learned_addr learned my addr 172.21.15.143:0/4102365809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2add9b640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0101420 0x7fe2b01974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2ae59c640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2b0100a80 0x7fe2b0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 -- 172.21.15.143:0/4102365809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2b0100a80 msgr2=0x7fe2b0196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2b0100a80 0x7fe2b0196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 -- 172.21.15.143:0/4102365809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0101420 msgr2=0x7fe2b01974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0101420 0x7fe2b01974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 -- 172.21.15.143:0/4102365809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01a6820 con 0x7fe2b0106a80 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2aed9d640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 0x7fe2b0197a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005cc0 tx=0x7fe2a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8004450 con 0x7fe2b0106a80 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a8031c90 con 0x7fe2b0106a80 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b01a6ab0 con 0x7fe2b0106a80 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a803b400 con 0x7fe2b0106a80 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.196+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a7040 con 0x7fe2b0106a80 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.197+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b0100ee0 con 0x7fe2b0106a80 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.199+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fe2a8004680 con 0x7fe2b0106a80 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.199+0000 7fe2a77fe640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe29c0a2c90 0x7fe29c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.200+0000 7fe2ae59c640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe29c0a2c90 0x7fe29c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.200+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe2a802e030 con 0x7fe2b0106a80 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.200+0000 7fe2ae59c640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe29c0a2c90 0x7fe29c0a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005fd0 tx=0x7fe2a000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.202+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2a80bd590 con 0x7fe2b0106a80 2024-09-10T10:45:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.404+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b006ad80 con 0x7fe2b0106a80 2024-09-10T10:45:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.405+0000 7fe2a77fe640 1 -- 172.21.15.143:0/4102365809 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7fe2b006ad80 con 0x7fe2b0106a80 2024-09-10T10:45:20.406 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:20.406 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:45:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:45:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe29c0a2c90 msgr2=0x7fe29c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe29c0a2c90 0x7fe29c0a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005fd0 tx=0x7fe2a000b390 comp rx=0 tx=0).stop 2024-09-10T10:45:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 msgr2=0x7fe2b0197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 0x7fe2b0197a30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005cc0 tx=0x7fe2a8003c20 comp rx=0 tx=0).stop 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 shutdown_connections 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe29c0a2c90 0x7fe29c0a5150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2b0106a80 0x7fe2b0197a30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0101420 0x7fe2b01974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 --2- 172.21.15.143:0/4102365809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2b0100a80 0x7fe2b0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 >> 172.21.15.143:0/4102365809 conn(0x7fe2b00fc7a0 msgr2=0x7fe2b00fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 shutdown_connections 2024-09-10T10:45:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:20.407+0000 7fe2b4e86640 1 -- 172.21.15.143:0/4102365809 wait complete. 2024-09-10T10:45:20.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:20 smithi143 ceph-mon[31057]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T10:45:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:20 smithi143 ceph-mon[31057]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-10T10:45:20.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:20 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4102365809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:20 smithi178 ceph-mon[34813]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T10:45:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:20 smithi178 ceph-mon[34813]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-10T10:45:20.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:20 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4102365809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:20 smithi144 ceph-mon[34230]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T10:45:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:20 smithi144 ceph-mon[34230]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-10T10:45:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:20 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4102365809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:21.182 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T10:45:21.182 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T10:45:21.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- 172.21.15.143:0/862032592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8102410 msgr2=0x7f2de8108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- 172.21.15.143:0/862032592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8102410 0x7f2de8108ca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009ca0 tx=0x7f2de002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- 172.21.15.143:0/862032592 shutdown_connections 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- 172.21.15.143:0/862032592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8102410 0x7f2de8108ca0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- 172.21.15.143:0/862032592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 0x7f2de8101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- 172.21.15.143:0/862032592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8100860 0x7f2de8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- 172.21.15.143:0/862032592 >> 172.21.15.143:0/862032592 conn(0x7f2de80fc000 msgr2=0x7f2de80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- 172.21.15.143:0/862032592 shutdown_connections 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- 172.21.15.143:0/862032592 wait complete. 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 Processor -- start 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- start start 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 0x7f2de81047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 0x7f2de8102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8102410 0x7f2de8103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810c370 con 0x7f2de8100860 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.284+0000 7f2dee99d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810c1f0 con 0x7f2de8101a50 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2dee99d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810c4f0 con 0x7f2de8102410 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 0x7f2de81047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 0x7f2de81047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42412/0 (socket says 172.21.15.143:42412) 2024-09-10T10:45:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 -- 172.21.15.143:0/1856985818 learned_addr learned my addr 172.21.15.143:0/1856985818 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded19a640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 0x7f2de8102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2dee19c640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8102410 0x7f2de8103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 -- 172.21.15.143:0/1856985818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8102410 msgr2=0x7f2de8103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8102410 0x7f2de8103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 -- 172.21.15.143:0/1856985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 msgr2=0x7f2de8102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 0x7f2de8102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 -- 172.21.15.143:0/1856985818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de8103c30 con 0x7f2de8100860 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded19a640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 0x7f2de8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ded99b640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 0x7f2de81047a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d950 tx=0x7f2de400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4017020 con 0x7f2de8100860 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2de4009c30 con 0x7f2de8100860 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de81a1cb0 con 0x7f2de8100860 2024-09-10T10:45:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400f6b0 con 0x7f2de8100860 2024-09-10T10:45:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.285+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de81a21d0 con 0x7f2de8100860 2024-09-10T10:45:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.286+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de81a23a0 con 0x7f2de8100860 2024-09-10T10:45:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.287+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2de401a020 con 0x7f2de8100860 2024-09-10T10:45:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.288+0000 7f2ddeffd640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2dd40a2c20 0x7f2dd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.288+0000 7f2ded19a640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2dd40a2c20 0x7f2dd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.288+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2de40cf530 con 0x7f2de8100860 2024-09-10T10:45:21.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.289+0000 7f2ded19a640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2dd40a2c20 0x7f2dd40a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005fd0 tx=0x7f2dd8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:21.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.292+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2de4021020 con 0x7f2de8100860 2024-09-10T10:45:21.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.454+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2de8069a70 con 0x7f2dd40a2c20 2024-09-10T10:45:21.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.462+0000 7f2ddeffd640 1 -- 172.21.15.143:0/1856985818 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340562 (secure 0 0 0) 0x7f2de8069a70 con 0x7f2dd40a2c20 2024-09-10T10:45:21.468 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:21.470 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:45:21.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2dd40a2c20 msgr2=0x7f2dd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2dd40a2c20 0x7f2dd40a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005fd0 tx=0x7f2dd8005850 comp rx=0 tx=0).stop 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 msgr2=0x7f2de81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 0x7f2de81047a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d950 tx=0x7f2de400de10 comp rx=0 tx=0).stop 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 shutdown_connections 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2dd40a2c20 0x7f2dd40a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2de8102410 0x7f2de8103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2de8101a50 0x7f2de8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 --2- 172.21.15.143:0/1856985818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de8100860 0x7f2de81047a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 >> 172.21.15.143:0/1856985818 conn(0x7f2de80fc000 msgr2=0x7f2de8069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 shutdown_connections 2024-09-10T10:45:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.466+0000 7f2dee99d640 1 -- 172.21.15.143:0/1856985818 wait complete. 2024-09-10T10:45:21.478 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-10T10:45:19.468862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662892,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1202,"num_read_kb":725,"num_write":455,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375776,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566048,"statfs":{"total":1151940427776,"available":1151555633152,"internally_reserved":0,"allocated":14385152,"data_stored":7955750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2475,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":54,"num_write":15,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003409"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:14.902511+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:45:14.902511+0000","last_peered":"2024-09-10T10:45:14.902511+0000","last_clean":"2024-09-10T10:45:14.902511+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:45:14.902511+0000","last_undegraded":"2024-09-10T10:45:14.902511+0000","last_fullsized":"2024-09-10T10:45:14.902511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441227+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:59.441227+0000","last_peered":"2024-09-10T10:44:59.441227+0000","last_clean":"2024-09-10T10:44:59.441227+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:59.441227+0000","last_undegraded":"2024-09-10T10:44:59.441227+0000","last_fullsized":"2024-09-10T10:44:59.441227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500745+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500745+0000","last_peered":"2024-09-10T10:44:00.500745+0000","last_clean":"2024-09-10T10:44:00.500745+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500745+0000","last_undegraded":"2024-09-10T10:44:00.500745+0000","last_fullsized":"2024-09-10T10:44:00.500745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505312+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505312+0000","last_peered":"2024-09-10T10:44:00.505312+0000","last_clean":"2024-09-10T10:44:00.505312+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505312+0000","last_undegraded":"2024-09-10T10:44:00.505312+0000","last_fullsized":"2024-09-10T10:44:00.505312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440037+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:59.440037+0000","last_peered":"2024-09-10T10:44:59.440037+0000","last_clean":"2024-09-10T10:44:59.440037+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:59.440037+0000","last_undegraded":"2024-09-10T10:44:59.440037+0000","last_fullsized":"2024-09-10T10:44:59.440037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503391+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503391+0000","last_peered":"2024-09-10T10:44:00.503391+0000","last_clean":"2024-09-10T10:44:00.503391+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503391+0000","last_undegraded":"2024-09-10T10:44:00.503391+0000","last_fullsized":"2024-09-10T10:44:00.503391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506237+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:44:00.506237+0000","last_peered":"2024-09-10T10:44:00.506237+0000","last_clean":"2024-09-10T10:44:00.506237+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:44:00.506237+0000","last_undegraded":"2024-09-10T10:44:00.506237+0000","last_fullsized":"2024-09-10T10:44:00.506237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442902+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:59.442902+0000","last_peered":"2024-09-10T10:44:59.442902+0000","last_clean":"2024-09-10T10:44:59.442902+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:59.442902+0000","last_undegraded":"2024-09-10T10:44:59.442902+0000","last_fullsized":"2024-09-10T10:44:59.442902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505929+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505929+0000","last_peered":"2024-09-10T10:44:00.505929+0000","last_clean":"2024-09-10T10:44:00.505929+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505929+0000","last_undegraded":"2024-09-10T10:44:00.505929+0000","last_fullsized":"2024-09-10T10:44:00.505929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442811+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:59.442811+0000","last_peered":"2024-09-10T10:44:59.442811+0000","last_clean":"2024-09-10T10:44:59.442811+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:59.442811+0000","last_undegraded":"2024-09-10T10:44:59.442811+0000","last_fullsized":"2024-09-10T10:44:59.442811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442546+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:59.442546+0000","last_peered":"2024-09-10T10:44:59.442546+0000","last_clean":"2024-09-10T10:44:59.442546+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:59.442546+0000","last_undegraded":"2024-09-10T10:44:59.442546+0000","last_fullsized":"2024-09-10T10:44:59.442546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503983+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503983+0000","last_peered":"2024-09-10T10:44:00.503983+0000","last_clean":"2024-09-10T10:44:00.503983+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503983+0000","last_undegraded":"2024-09-10T10:44:00.503983+0000","last_fullsized":"2024-09-10T10:44:00.503983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435612+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:59.435612+0000","last_peered":"2024-09-10T10:44:59.435612+0000","last_clean":"2024-09-10T10:44:59.435612+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:59.435612+0000","last_undegraded":"2024-09-10T10:44:59.435612+0000","last_fullsized":"2024-09-10T10:44:59.435612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.106863+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:45:01.106863+0000","last_peered":"2024-09-10T10:45:01.106863+0000","last_clean":"2024-09-10T10:45:01.106863+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:45:01.106863+0000","last_undegraded":"2024-09-10T10:45:01.106863+0000","last_fullsized":"2024-09-10T10:45:01.106863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.524134+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.524134+0000","last_peered":"2024-09-10T10:44:00.524134+0000","last_clean":"2024-09-10T10:44:00.524134+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.524134+0000","last_undegraded":"2024-09-10T10:44:00.524134+0000","last_fullsized":"2024-09-10T10:44:00.524134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437632+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:59.437632+0000","last_peered":"2024-09-10T10:44:59.437632+0000","last_clean":"2024-09-10T10:44:59.437632+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:59.437632+0000","last_undegraded":"2024-09-10T10:44:59.437632+0000","last_fullsized":"2024-09-10T10:44:59.437632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504947+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504947+0000","last_peered":"2024-09-10T10:44:00.504947+0000","last_clean":"2024-09-10T10:44:00.504947+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504947+0000","last_undegraded":"2024-09-10T10:44:00.504947+0000","last_fullsized":"2024-09-10T10:44:00.504947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500343+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500343+0000","last_peered":"2024-09-10T10:44:00.500343+0000","last_clean":"2024-09-10T10:44:00.500343+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500343+0000","last_undegraded":"2024-09-10T10:44:00.500343+0000","last_fullsized":"2024-09-10T10:44:00.500343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.422791+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:59.422791+0000","last_peered":"2024-09-10T10:44:59.422791+0000","last_clean":"2024-09-10T10:44:59.422791+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:59.422791+0000","last_undegraded":"2024-09-10T10:44:59.422791+0000","last_fullsized":"2024-09-10T10:44:59.422791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510061+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510061+0000","last_peered":"2024-09-10T10:44:00.510061+0000","last_clean":"2024-09-10T10:44:00.510061+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510061+0000","last_undegraded":"2024-09-10T10:44:00.510061+0000","last_fullsized":"2024-09-10T10:44:00.510061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504098+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504098+0000","last_peered":"2024-09-10T10:44:00.504098+0000","last_clean":"2024-09-10T10:44:00.504098+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504098+0000","last_undegraded":"2024-09-10T10:44:00.504098+0000","last_fullsized":"2024-09-10T10:44:00.504098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441460+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:59.441460+0000","last_peered":"2024-09-10T10:44:59.441460+0000","last_clean":"2024-09-10T10:44:59.441460+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:59.441460+0000","last_undegraded":"2024-09-10T10:44:59.441460+0000","last_fullsized":"2024-09-10T10:44:59.441460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504342+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504342+0000","last_peered":"2024-09-10T10:44:00.504342+0000","last_clean":"2024-09-10T10:44:00.504342+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504342+0000","last_undegraded":"2024-09-10T10:44:00.504342+0000","last_fullsized":"2024-09-10T10:44:00.504342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500934+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500934+0000","last_peered":"2024-09-10T10:44:00.500934+0000","last_clean":"2024-09-10T10:44:00.500934+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500934+0000","last_undegraded":"2024-09-10T10:44:00.500934+0000","last_fullsized":"2024-09-10T10:44:00.500934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440153+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:59.440153+0000","last_peered":"2024-09-10T10:44:59.440153+0000","last_clean":"2024-09-10T10:44:59.440153+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:59.440153+0000","last_undegraded":"2024-09-10T10:44:59.440153+0000","last_fullsized":"2024-09-10T10:44:59.440153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.480284+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:45:01.480284+0000","last_peered":"2024-09-10T10:45:01.480284+0000","last_clean":"2024-09-10T10:45:01.480284+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:45:01.480284+0000","last_undegraded":"2024-09-10T10:45:01.480284+0000","last_fullsized":"2024-09-10T10:45:01.480284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504339+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504339+0000","last_peered":"2024-09-10T10:44:00.504339+0000","last_clean":"2024-09-10T10:44:00.504339+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504339+0000","last_undegraded":"2024-09-10T10:44:00.504339+0000","last_fullsized":"2024-09-10T10:44:00.504339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443706+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:59.443706+0000","last_peered":"2024-09-10T10:44:59.443706+0000","last_clean":"2024-09-10T10:44:59.443706+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:59.443706+0000","last_undegraded":"2024-09-10T10:44:59.443706+0000","last_fullsized":"2024-09-10T10:44:59.443706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506338+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506338+0000","last_peered":"2024-09-10T10:44:00.506338+0000","last_clean":"2024-09-10T10:44:00.506338+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506338+0000","last_undegraded":"2024-09-10T10:44:00.506338+0000","last_fullsized":"2024-09-10T10:44:00.506338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503217+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503217+0000","last_peered":"2024-09-10T10:44:00.503217+0000","last_clean":"2024-09-10T10:44:00.503217+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503217+0000","last_undegraded":"2024-09-10T10:44:00.503217+0000","last_fullsized":"2024-09-10T10:44:00.503217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.424958+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:59.424958+0000","last_peered":"2024-09-10T10:44:59.424958+0000","last_clean":"2024-09-10T10:44:59.424958+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:59.424958+0000","last_undegraded":"2024-09-10T10:44:59.424958+0000","last_fullsized":"2024-09-10T10:44:59.424958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501376+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501376+0000","last_peered":"2024-09-10T10:44:00.501376+0000","last_clean":"2024-09-10T10:44:00.501376+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501376+0000","last_undegraded":"2024-09-10T10:44:00.501376+0000","last_fullsized":"2024-09-10T10:44:00.501376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505353+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505353+0000","last_peered":"2024-09-10T10:44:00.505353+0000","last_clean":"2024-09-10T10:44:00.505353+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505353+0000","last_undegraded":"2024-09-10T10:44:00.505353+0000","last_fullsized":"2024-09-10T10:44:00.505353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438858+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:59.438858+0000","last_peered":"2024-09-10T10:44:59.438858+0000","last_clean":"2024-09-10T10:44:59.438858+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:59.438858+0000","last_undegraded":"2024-09-10T10:44:59.438858+0000","last_fullsized":"2024-09-10T10:44:59.438858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'6","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:14.704378+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:45:14.704378+0000","last_peered":"2024-09-10T10:45:14.704378+0000","last_clean":"2024-09-10T10:45:14.704378+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:45:14.704378+0000","last_undegraded":"2024-09-10T10:45:14.704378+0000","last_fullsized":"2024-09-10T10:45:14.704378+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502631+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502631+0000","last_peered":"2024-09-10T10:44:00.502631+0000","last_clean":"2024-09-10T10:44:00.502631+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502631+0000","last_undegraded":"2024-09-10T10:44:00.502631+0000","last_fullsized":"2024-09-10T10:44:00.502631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439106+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:59.439106+0000","last_peered":"2024-09-10T10:44:59.439106+0000","last_clean":"2024-09-10T10:44:59.439106+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:59.439106+0000","last_undegraded":"2024-09-10T10:44:59.439106+0000","last_fullsized":"2024-09-10T10:44:59.439106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:12.699992+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:45:12.699992+0000","last_peered":"2024-09-10T10:45:12.699992+0000","last_clean":"2024-09-10T10:45:12.699992+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:45:12.699992+0000","last_undegraded":"2024-09-10T10:45:12.699992+0000","last_fullsized":"2024-09-10T10:45:12.699992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502777+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502777+0000","last_peered":"2024-09-10T10:44:00.502777+0000","last_clean":"2024-09-10T10:44:00.502777+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502777+0000","last_undegraded":"2024-09-10T10:44:00.502777+0000","last_fullsized":"2024-09-10T10:44:00.502777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444099+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:59.444099+0000","last_peered":"2024-09-10T10:44:59.444099+0000","last_clean":"2024-09-10T10:44:59.444099+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:59.444099+0000","last_undegraded":"2024-09-10T10:44:59.444099+0000","last_fullsized":"2024-09-10T10:44:59.444099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506093+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506093+0000","last_peered":"2024-09-10T10:44:00.506093+0000","last_clean":"2024-09-10T10:44:00.506093+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506093+0000","last_undegraded":"2024-09-10T10:44:00.506093+0000","last_fullsized":"2024-09-10T10:44:00.506093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516092+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.516092+0000","last_peered":"2024-09-10T10:44:00.516092+0000","last_clean":"2024-09-10T10:44:00.516092+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.516092+0000","last_undegraded":"2024-09-10T10:44:00.516092+0000","last_fullsized":"2024-09-10T10:44:00.516092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443062+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:59.443062+0000","last_peered":"2024-09-10T10:44:59.443062+0000","last_clean":"2024-09-10T10:44:59.443062+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:59.443062+0000","last_undegraded":"2024-09-10T10:44:59.443062+0000","last_fullsized":"2024-09-10T10:44:59.443062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503953+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503953+0000","last_peered":"2024-09-10T10:44:00.503953+0000","last_clean":"2024-09-10T10:44:00.503953+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503953+0000","last_undegraded":"2024-09-10T10:44:00.503953+0000","last_fullsized":"2024-09-10T10:44:00.503953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507101+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507101+0000","last_peered":"2024-09-10T10:44:00.507101+0000","last_clean":"2024-09-10T10:44:00.507101+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507101+0000","last_undegraded":"2024-09-10T10:44:00.507101+0000","last_fullsized":"2024-09-10T10:44:00.507101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442450+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:59.442450+0000","last_peered":"2024-09-10T10:44:59.442450+0000","last_clean":"2024-09-10T10:44:59.442450+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:59.442450+0000","last_undegraded":"2024-09-10T10:44:59.442450+0000","last_fullsized":"2024-09-10T10:44:59.442450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.756852+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:45:15.756852+0000","last_peered":"2024-09-10T10:45:15.756852+0000","last_clean":"2024-09-10T10:45:15.756852+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:45:15.756852+0000","last_undegraded":"2024-09-10T10:45:15.756852+0000","last_fullsized":"2024-09-10T10:45:15.756852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509662+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509662+0000","last_peered":"2024-09-10T10:44:00.509662+0000","last_clean":"2024-09-10T10:44:00.509662+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509662+0000","last_undegraded":"2024-09-10T10:44:00.509662+0000","last_fullsized":"2024-09-10T10:44:00.509662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442843+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:59.442843+0000","last_peered":"2024-09-10T10:44:59.442843+0000","last_clean":"2024-09-10T10:44:59.442843+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:59.442843+0000","last_undegraded":"2024-09-10T10:44:59.442843+0000","last_fullsized":"2024-09-10T10:44:59.442843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516384+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516384+0000","last_peered":"2024-09-10T10:44:00.516384+0000","last_clean":"2024-09-10T10:44:00.516384+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516384+0000","last_undegraded":"2024-09-10T10:44:00.516384+0000","last_fullsized":"2024-09-10T10:44:00.516384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500645+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500645+0000","last_peered":"2024-09-10T10:44:00.500645+0000","last_clean":"2024-09-10T10:44:00.500645+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500645+0000","last_undegraded":"2024-09-10T10:44:00.500645+0000","last_fullsized":"2024-09-10T10:44:00.500645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440420+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:59.440420+0000","last_peered":"2024-09-10T10:44:59.440420+0000","last_clean":"2024-09-10T10:44:59.440420+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:59.440420+0000","last_undegraded":"2024-09-10T10:44:59.440420+0000","last_fullsized":"2024-09-10T10:44:59.440420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503016+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.503016+0000","last_peered":"2024-09-10T10:44:00.503016+0000","last_clean":"2024-09-10T10:44:00.503016+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.503016+0000","last_undegraded":"2024-09-10T10:44:00.503016+0000","last_fullsized":"2024-09-10T10:44:00.503016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515085+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515085+0000","last_peered":"2024-09-10T10:44:00.515085+0000","last_clean":"2024-09-10T10:44:00.515085+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515085+0000","last_undegraded":"2024-09-10T10:44:00.515085+0000","last_fullsized":"2024-09-10T10:44:00.515085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440308+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:59.440308+0000","last_peered":"2024-09-10T10:44:59.440308+0000","last_clean":"2024-09-10T10:44:59.440308+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:59.440308+0000","last_undegraded":"2024-09-10T10:44:59.440308+0000","last_fullsized":"2024-09-10T10:44:59.440308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:05.823762+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:45:05.823762+0000","last_peered":"2024-09-10T10:45:05.823762+0000","last_clean":"2024-09-10T10:45:05.823762+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:45:05.823762+0000","last_undegraded":"2024-09-10T10:45:05.823762+0000","last_fullsized":"2024-09-10T10:45:05.823762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515302+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515302+0000","last_peered":"2024-09-10T10:44:00.515302+0000","last_clean":"2024-09-10T10:44:00.515302+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515302+0000","last_undegraded":"2024-09-10T10:44:00.515302+0000","last_fullsized":"2024-09-10T10:44:00.515302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442616+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442616+0000","last_peered":"2024-09-10T10:44:59.442616+0000","last_clean":"2024-09-10T10:44:59.442616+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:59.442616+0000","last_undegraded":"2024-09-10T10:44:59.442616+0000","last_fullsized":"2024-09-10T10:44:59.442616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505428+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505428+0000","last_peered":"2024-09-10T10:44:00.505428+0000","last_clean":"2024-09-10T10:44:00.505428+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505428+0000","last_undegraded":"2024-09-10T10:44:00.505428+0000","last_fullsized":"2024-09-10T10:44:00.505428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506639+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.506639+0000","last_peered":"2024-09-10T10:44:00.506639+0000","last_clean":"2024-09-10T10:44:00.506639+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.506639+0000","last_undegraded":"2024-09-10T10:44:00.506639+0000","last_fullsized":"2024-09-10T10:44:00.506639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441344+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:59.441344+0000","last_peered":"2024-09-10T10:44:59.441344+0000","last_clean":"2024-09-10T10:44:59.441344+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:59.441344+0000","last_undegraded":"2024-09-10T10:44:59.441344+0000","last_fullsized":"2024-09-10T10:44:59.441344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:16.706499+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:45:16.706499+0000","last_peered":"2024-09-10T10:45:16.706499+0000","last_clean":"2024-09-10T10:45:16.706499+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:45:16.706499+0000","last_undegraded":"2024-09-10T10:45:16.706499+0000","last_fullsized":"2024-09-10T10:45:16.706499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500337+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500337+0000","last_peered":"2024-09-10T10:44:00.500337+0000","last_clean":"2024-09-10T10:44:00.500337+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500337+0000","last_undegraded":"2024-09-10T10:44:00.500337+0000","last_fullsized":"2024-09-10T10:44:00.500337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445872+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:59.445872+0000","last_peered":"2024-09-10T10:44:59.445872+0000","last_clean":"2024-09-10T10:44:59.445872+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:59.445872+0000","last_undegraded":"2024-09-10T10:44:59.445872+0000","last_fullsized":"2024-09-10T10:44:59.445872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515497+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515497+0000","last_peered":"2024-09-10T10:44:00.515497+0000","last_clean":"2024-09-10T10:44:00.515497+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515497+0000","last_undegraded":"2024-09-10T10:44:00.515497+0000","last_fullsized":"2024-09-10T10:44:00.515497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502859+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502859+0000","last_peered":"2024-09-10T10:44:00.502859+0000","last_clean":"2024-09-10T10:44:00.502859+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502859+0000","last_undegraded":"2024-09-10T10:44:00.502859+0000","last_fullsized":"2024-09-10T10:44:00.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435638+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:59.435638+0000","last_peered":"2024-09-10T10:44:59.435638+0000","last_clean":"2024-09-10T10:44:59.435638+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:59.435638+0000","last_undegraded":"2024-09-10T10:44:59.435638+0000","last_fullsized":"2024-09-10T10:44:59.435638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516076+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516076+0000","last_peered":"2024-09-10T10:44:00.516076+0000","last_clean":"2024-09-10T10:44:00.516076+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516076+0000","last_undegraded":"2024-09-10T10:44:00.516076+0000","last_fullsized":"2024-09-10T10:44:00.516076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432274+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:59.432274+0000","last_peered":"2024-09-10T10:44:59.432274+0000","last_clean":"2024-09-10T10:44:59.432274+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:59.432274+0000","last_undegraded":"2024-09-10T10:44:59.432274+0000","last_fullsized":"2024-09-10T10:44:59.432274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":311,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.623041+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:45:15.623041+0000","last_peered":"2024-09-10T10:45:15.623041+0000","last_clean":"2024-09-10T10:45:15.623041+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:45:15.623041+0000","last_undegraded":"2024-09-10T10:45:15.623041+0000","last_fullsized":"2024-09-10T10:45:15.623041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62820,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":167,"num_write":41,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500085+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500085+0000","last_peered":"2024-09-10T10:44:00.500085+0000","last_clean":"2024-09-10T10:44:00.500085+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500085+0000","last_undegraded":"2024-09-10T10:44:00.500085+0000","last_fullsized":"2024-09-10T10:44:00.500085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516289+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516289+0000","last_peered":"2024-09-10T10:44:00.516289+0000","last_clean":"2024-09-10T10:44:00.516289+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516289+0000","last_undegraded":"2024-09-10T10:44:00.516289+0000","last_fullsized":"2024-09-10T10:44:00.516289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503778+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503778+0000","last_peered":"2024-09-10T10:44:00.503778+0000","last_clean":"2024-09-10T10:44:00.503778+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503778+0000","last_undegraded":"2024-09-10T10:44:00.503778+0000","last_fullsized":"2024-09-10T10:44:00.503778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506656+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506656+0000","last_peered":"2024-09-10T10:44:00.506656+0000","last_clean":"2024-09-10T10:44:00.506656+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506656+0000","last_undegraded":"2024-09-10T10:44:00.506656+0000","last_fullsized":"2024-09-10T10:44:00.506656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499856+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499856+0000","last_peered":"2024-09-10T10:44:00.499856+0000","last_clean":"2024-09-10T10:44:00.499856+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499856+0000","last_undegraded":"2024-09-10T10:44:00.499856+0000","last_fullsized":"2024-09-10T10:44:00.499856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442222+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:59.442222+0000","last_peered":"2024-09-10T10:44:59.442222+0000","last_clean":"2024-09-10T10:44:59.442222+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:59.442222+0000","last_undegraded":"2024-09-10T10:44:59.442222+0000","last_fullsized":"2024-09-10T10:44:59.442222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.756025+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:45:15.756025+0000","last_peered":"2024-09-10T10:45:15.756025+0000","last_clean":"2024-09-10T10:45:15.756025+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:45:15.756025+0000","last_undegraded":"2024-09-10T10:45:15.756025+0000","last_fullsized":"2024-09-10T10:45:15.756025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504723+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504723+0000","last_peered":"2024-09-10T10:44:00.504723+0000","last_clean":"2024-09-10T10:44:00.504723+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504723+0000","last_undegraded":"2024-09-10T10:44:00.504723+0000","last_fullsized":"2024-09-10T10:44:00.504723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500929+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500929+0000","last_peered":"2024-09-10T10:44:00.500929+0000","last_clean":"2024-09-10T10:44:00.500929+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500929+0000","last_undegraded":"2024-09-10T10:44:00.500929+0000","last_fullsized":"2024-09-10T10:44:00.500929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432970+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:59.432970+0000","last_peered":"2024-09-10T10:44:59.432970+0000","last_clean":"2024-09-10T10:44:59.432970+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:59.432970+0000","last_undegraded":"2024-09-10T10:44:59.432970+0000","last_fullsized":"2024-09-10T10:44:59.432970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515598+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515598+0000","last_peered":"2024-09-10T10:44:00.515598+0000","last_clean":"2024-09-10T10:44:00.515598+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515598+0000","last_undegraded":"2024-09-10T10:44:00.515598+0000","last_fullsized":"2024-09-10T10:44:00.515598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509926+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509926+0000","last_peered":"2024-09-10T10:44:00.509926+0000","last_clean":"2024-09-10T10:44:00.509926+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509926+0000","last_undegraded":"2024-09-10T10:44:00.509926+0000","last_fullsized":"2024-09-10T10:44:00.509926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.434606+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:59.434606+0000","last_peered":"2024-09-10T10:44:59.434606+0000","last_clean":"2024-09-10T10:44:59.434606+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:59.434606+0000","last_undegraded":"2024-09-10T10:44:59.434606+0000","last_fullsized":"2024-09-10T10:44:59.434606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502463+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502463+0000","last_peered":"2024-09-10T10:44:00.502463+0000","last_clean":"2024-09-10T10:44:00.502463+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502463+0000","last_undegraded":"2024-09-10T10:44:00.502463+0000","last_fullsized":"2024-09-10T10:44:00.502463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505325+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505325+0000","last_peered":"2024-09-10T10:44:00.505325+0000","last_clean":"2024-09-10T10:44:00.505325+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505325+0000","last_undegraded":"2024-09-10T10:44:00.505325+0000","last_fullsized":"2024-09-10T10:44:00.505325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438031+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:59.438031+0000","last_peered":"2024-09-10T10:44:59.438031+0000","last_clean":"2024-09-10T10:44:59.438031+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:59.438031+0000","last_undegraded":"2024-09-10T10:44:59.438031+0000","last_fullsized":"2024-09-10T10:44:59.438031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.707305+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:45:15.707305+0000","last_peered":"2024-09-10T10:45:15.707305+0000","last_clean":"2024-09-10T10:45:15.707305+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:45:15.707305+0000","last_undegraded":"2024-09-10T10:45:15.707305+0000","last_fullsized":"2024-09-10T10:45:15.707305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439297+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:59.439297+0000","last_peered":"2024-09-10T10:44:59.439297+0000","last_clean":"2024-09-10T10:44:59.439297+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:59.439297+0000","last_undegraded":"2024-09-10T10:44:59.439297+0000","last_fullsized":"2024-09-10T10:44:59.439297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440974+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:59.440974+0000","last_peered":"2024-09-10T10:44:59.440974+0000","last_clean":"2024-09-10T10:44:59.440974+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:59.440974+0000","last_undegraded":"2024-09-10T10:44:59.440974+0000","last_fullsized":"2024-09-10T10:44:59.440974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441221+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:59.441221+0000","last_peered":"2024-09-10T10:44:59.441221+0000","last_clean":"2024-09-10T10:44:59.441221+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:59.441221+0000","last_undegraded":"2024-09-10T10:44:59.441221+0000","last_fullsized":"2024-09-10T10:44:59.441221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440703+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:59.440703+0000","last_peered":"2024-09-10T10:44:59.440703+0000","last_clean":"2024-09-10T10:44:59.440703+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:59.440703+0000","last_undegraded":"2024-09-10T10:44:59.440703+0000","last_fullsized":"2024-09-10T10:44:59.440703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444329+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:59.444329+0000","last_peered":"2024-09-10T10:44:59.444329+0000","last_clean":"2024-09-10T10:44:59.444329+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:59.444329+0000","last_undegraded":"2024-09-10T10:44:59.444329+0000","last_fullsized":"2024-09-10T10:44:59.444329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438450+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:59.438450+0000","last_peered":"2024-09-10T10:44:59.438450+0000","last_clean":"2024-09-10T10:44:59.438450+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:59.438450+0000","last_undegraded":"2024-09-10T10:44:59.438450+0000","last_fullsized":"2024-09-10T10:44:59.438450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441275+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:59.441275+0000","last_peered":"2024-09-10T10:44:59.441275+0000","last_clean":"2024-09-10T10:44:59.441275+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:59.441275+0000","last_undegraded":"2024-09-10T10:44:59.441275+0000","last_fullsized":"2024-09-10T10:44:59.441275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437164+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:59.437164+0000","last_peered":"2024-09-10T10:44:59.437164+0000","last_clean":"2024-09-10T10:44:59.437164+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:59.437164+0000","last_undegraded":"2024-09-10T10:44:59.437164+0000","last_fullsized":"2024-09-10T10:44:59.437164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441514+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:59.441514+0000","last_peered":"2024-09-10T10:44:59.441514+0000","last_clean":"2024-09-10T10:44:59.441514+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:59.441514+0000","last_undegraded":"2024-09-10T10:44:59.441514+0000","last_fullsized":"2024-09-10T10:44:59.441514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442360+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:59.442360+0000","last_peered":"2024-09-10T10:44:59.442360+0000","last_clean":"2024-09-10T10:44:59.442360+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:59.442360+0000","last_undegraded":"2024-09-10T10:44:59.442360+0000","last_fullsized":"2024-09-10T10:44:59.442360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442124+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:59.442124+0000","last_peered":"2024-09-10T10:44:59.442124+0000","last_clean":"2024-09-10T10:44:59.442124+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:59.442124+0000","last_undegraded":"2024-09-10T10:44:59.442124+0000","last_fullsized":"2024-09-10T10:44:59.442124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442058+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:59.442058+0000","last_peered":"2024-09-10T10:44:59.442058+0000","last_clean":"2024-09-10T10:44:59.442058+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:59.442058+0000","last_undegraded":"2024-09-10T10:44:59.442058+0000","last_fullsized":"2024-09-10T10:44:59.442058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438851+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:59.438851+0000","last_peered":"2024-09-10T10:44:59.438851+0000","last_clean":"2024-09-10T10:44:59.438851+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:59.438851+0000","last_undegraded":"2024-09-10T10:44:59.438851+0000","last_fullsized":"2024-09-10T10:44:59.438851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429557+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:59.429557+0000","last_peered":"2024-09-10T10:44:59.429557+0000","last_clean":"2024-09-10T10:44:59.429557+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:59.429557+0000","last_undegraded":"2024-09-10T10:44:59.429557+0000","last_fullsized":"2024-09-10T10:44:59.429557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440551+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:59.440551+0000","last_peered":"2024-09-10T10:44:59.440551+0000","last_clean":"2024-09-10T10:44:59.440551+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:59.440551+0000","last_undegraded":"2024-09-10T10:44:59.440551+0000","last_fullsized":"2024-09-10T10:44:59.440551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437183+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:59.437183+0000","last_peered":"2024-09-10T10:44:59.437183+0000","last_clean":"2024-09-10T10:44:59.437183+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:59.437183+0000","last_undegraded":"2024-09-10T10:44:59.437183+0000","last_fullsized":"2024-09-10T10:44:59.437183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439780+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:59.439780+0000","last_peered":"2024-09-10T10:44:59.439780+0000","last_clean":"2024-09-10T10:44:59.439780+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:59.439780+0000","last_undegraded":"2024-09-10T10:44:59.439780+0000","last_fullsized":"2024-09-10T10:44:59.439780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441856+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:59.441856+0000","last_peered":"2024-09-10T10:44:59.441856+0000","last_clean":"2024-09-10T10:44:59.441856+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:59.441856+0000","last_undegraded":"2024-09-10T10:44:59.441856+0000","last_fullsized":"2024-09-10T10:44:59.441856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437328+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:59.437328+0000","last_peered":"2024-09-10T10:44:59.437328+0000","last_clean":"2024-09-10T10:44:59.437328+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:59.437328+0000","last_undegraded":"2024-09-10T10:44:59.437328+0000","last_fullsized":"2024-09-10T10:44:59.437328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438618+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:59.438618+0000","last_peered":"2024-09-10T10:44:59.438618+0000","last_clean":"2024-09-10T10:44:59.438618+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:59.438618+0000","last_undegraded":"2024-09-10T10:44:59.438618+0000","last_fullsized":"2024-09-10T10:44:59.438618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443917+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:59.443917+0000","last_peered":"2024-09-10T10:44:59.443917+0000","last_clean":"2024-09-10T10:44:59.443917+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:59.443917+0000","last_undegraded":"2024-09-10T10:44:59.443917+0000","last_fullsized":"2024-09-10T10:44:59.443917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442554+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:59.442554+0000","last_peered":"2024-09-10T10:44:59.442554+0000","last_clean":"2024-09-10T10:44:59.442554+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:59.442554+0000","last_undegraded":"2024-09-10T10:44:59.442554+0000","last_fullsized":"2024-09-10T10:44:59.442554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432695+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:59.432695+0000","last_peered":"2024-09-10T10:44:59.432695+0000","last_clean":"2024-09-10T10:44:59.432695+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:59.432695+0000","last_undegraded":"2024-09-10T10:44:59.432695+0000","last_fullsized":"2024-09-10T10:44:59.432695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442641+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442641+0000","last_peered":"2024-09-10T10:44:59.442641+0000","last_clean":"2024-09-10T10:44:59.442641+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:59.442641+0000","last_undegraded":"2024-09-10T10:44:59.442641+0000","last_fullsized":"2024-09-10T10:44:59.442641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440814+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:59.440814+0000","last_peered":"2024-09-10T10:44:59.440814+0000","last_clean":"2024-09-10T10:44:59.440814+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:59.440814+0000","last_undegraded":"2024-09-10T10:44:59.440814+0000","last_fullsized":"2024-09-10T10:44:59.440814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444968+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:59.444968+0000","last_peered":"2024-09-10T10:44:59.444968+0000","last_clean":"2024-09-10T10:44:59.444968+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:59.444968+0000","last_undegraded":"2024-09-10T10:44:59.444968+0000","last_fullsized":"2024-09-10T10:44:59.444968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428461+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:59.428461+0000","last_peered":"2024-09-10T10:44:59.428461+0000","last_clean":"2024-09-10T10:44:59.428461+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:59.428461+0000","last_undegraded":"2024-09-10T10:44:59.428461+0000","last_fullsized":"2024-09-10T10:44:59.428461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428915+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:59.428915+0000","last_peered":"2024-09-10T10:44:59.428915+0000","last_clean":"2024-09-10T10:44:59.428915+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:59.428915+0000","last_undegraded":"2024-09-10T10:44:59.428915+0000","last_fullsized":"2024-09-10T10:44:59.428915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441024+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:59.441024+0000","last_peered":"2024-09-10T10:44:59.441024+0000","last_clean":"2024-09-10T10:44:59.441024+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:59.441024+0000","last_undegraded":"2024-09-10T10:44:59.441024+0000","last_fullsized":"2024-09-10T10:44:59.441024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441764+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:59.441764+0000","last_peered":"2024-09-10T10:44:59.441764+0000","last_clean":"2024-09-10T10:44:59.441764+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:59.441764+0000","last_undegraded":"2024-09-10T10:44:59.441764+0000","last_fullsized":"2024-09-10T10:44:59.441764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439409+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:59.439409+0000","last_peered":"2024-09-10T10:44:59.439409+0000","last_clean":"2024-09-10T10:44:59.439409+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:59.439409+0000","last_undegraded":"2024-09-10T10:44:59.439409+0000","last_fullsized":"2024-09-10T10:44:59.439409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439118+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:59.439118+0000","last_peered":"2024-09-10T10:44:59.439118+0000","last_clean":"2024-09-10T10:44:59.439118+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:59.439118+0000","last_undegraded":"2024-09-10T10:44:59.439118+0000","last_fullsized":"2024-09-10T10:44:59.439118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72290,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1076,"num_read_kb":616,"num_write":272,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":221082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":501134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1154798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939701,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1096985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":495471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":563394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1024000,"data_stored":498177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1085440,"data_stored":559816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:21.484 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:21.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 -- 172.21.15.143:0/4289812808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 msgr2=0x7f719c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 --2- 172.21.15.143:0/4289812808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c100e30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7190009900 tx=0x7f719002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:21.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 -- 172.21.15.143:0/4289812808 shutdown_connections 2024-09-10T10:45:21.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 --2- 172.21.15.143:0/4289812808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 0x7f719c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 --2- 172.21.15.143:0/4289812808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f719c101c20 0x7f719c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 --2- 172.21.15.143:0/4289812808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c100e30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 -- 172.21.15.143:0/4289812808 >> 172.21.15.143:0/4289812808 conn(0x7f719c0fc1b0 msgr2=0x7f719c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 -- 172.21.15.143:0/4289812808 shutdown_connections 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.619+0000 7f71a2c80640 1 -- 172.21.15.143:0/4289812808 wait complete. 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 Processor -- start 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 -- start start 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c06af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f719c101c20 0x7f719c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 0x7f719c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10c560 con 0x7f719c100a30 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10c3e0 con 0x7f719c101c20 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a2c80640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10c6e0 con 0x7f719c1025e0 2024-09-10T10:45:21.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c06af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c06af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42428/0 (socket says 172.21.15.143:42428) 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 -- 172.21.15.143:0/3604345684 learned_addr learned my addr 172.21.15.143:0/3604345684 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a121d640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 0x7f719c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f719bfff640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f719c101c20 0x7f719c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 -- 172.21.15.143:0/3604345684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 msgr2=0x7f719c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 0x7f719c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 -- 172.21.15.143:0/3604345684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f719c101c20 msgr2=0x7f719c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f719c101c20 0x7f719c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 -- 172.21.15.143:0/3604345684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c19f9f0 con 0x7f719c100a30 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a121d640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 0x7f719c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.620+0000 7f71a0a1c640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c06af30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f71900314e0 tx=0x7f71900042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.621+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719003f030 con 0x7f719c100a30 2024-09-10T10:45:21.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.621+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c19fc20 con 0x7f719c100a30 2024-09-10T10:45:21.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.621+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7190003cb0 con 0x7f719c100a30 2024-09-10T10:45:21.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.621+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190035070 con 0x7f719c100a30 2024-09-10T10:45:21.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.621+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1a0110 con 0x7f719c100a30 2024-09-10T10:45:21.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.622+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c1a02e0 con 0x7f719c100a30 2024-09-10T10:45:21.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.623+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f7190042030 con 0x7f719c100a30 2024-09-10T10:45:21.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.624+0000 7f7199ffb640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f718c0a2c20 0x7f718c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.624+0000 7f719bfff640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f718c0a2c20 0x7f718c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.624+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71900f0020 con 0x7f719c100a30 2024-09-10T10:45:21.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.624+0000 7f719bfff640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f718c0a2c20 0x7f718c0a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f719c06aca0 tx=0x7f718800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:21.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.627+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71900bcac0 con 0x7f719c100a30 2024-09-10T10:45:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.790+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f719c062630 con 0x7f718c0a2c20 2024-09-10T10:45:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.797+0000 7f7199ffb640 1 -- 172.21.15.143:0/3604345684 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340563 (secure 0 0 0) 0x7f719c062630 con 0x7f718c0a2c20 2024-09-10T10:45:21.802 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:21.804 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:45:21.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f718c0a2c20 msgr2=0x7f718c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f718c0a2c20 0x7f718c0a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f719c06aca0 tx=0x7f718800a4c0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 msgr2=0x7f719c06af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c06af30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f71900314e0 tx=0x7f71900042e0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 shutdown_connections 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f718c0a2c20 0x7f718c0a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f719c1025e0 0x7f719c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f719c101c20 0x7f719c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 --2- 172.21.15.143:0/3604345684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c100a30 0x7f719c06af30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 >> 172.21.15.143:0/3604345684 conn(0x7f719c0fc1b0 msgr2=0x7f719c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 shutdown_connections 2024-09-10T10:45:21.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.801+0000 7f71a2c80640 1 -- 172.21.15.143:0/3604345684 wait complete. 2024-09-10T10:45:21.814 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T10:45:21.469308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663285,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1219,"num_read_kb":732,"num_write":459,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375780,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566044,"statfs":{"total":1151940427776,"available":1151555629056,"internally_reserved":0,"allocated":14389248,"data_stored":7966136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2868,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":61,"num_write":19,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003164"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:14.902511+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:45:14.902511+0000","last_peered":"2024-09-10T10:45:14.902511+0000","last_clean":"2024-09-10T10:45:14.902511+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:45:14.902511+0000","last_undegraded":"2024-09-10T10:45:14.902511+0000","last_fullsized":"2024-09-10T10:45:14.902511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441227+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:59.441227+0000","last_peered":"2024-09-10T10:44:59.441227+0000","last_clean":"2024-09-10T10:44:59.441227+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:59.441227+0000","last_undegraded":"2024-09-10T10:44:59.441227+0000","last_fullsized":"2024-09-10T10:44:59.441227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500745+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500745+0000","last_peered":"2024-09-10T10:44:00.500745+0000","last_clean":"2024-09-10T10:44:00.500745+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500745+0000","last_undegraded":"2024-09-10T10:44:00.500745+0000","last_fullsized":"2024-09-10T10:44:00.500745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505312+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505312+0000","last_peered":"2024-09-10T10:44:00.505312+0000","last_clean":"2024-09-10T10:44:00.505312+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505312+0000","last_undegraded":"2024-09-10T10:44:00.505312+0000","last_fullsized":"2024-09-10T10:44:00.505312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440037+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:59.440037+0000","last_peered":"2024-09-10T10:44:59.440037+0000","last_clean":"2024-09-10T10:44:59.440037+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:59.440037+0000","last_undegraded":"2024-09-10T10:44:59.440037+0000","last_fullsized":"2024-09-10T10:44:59.440037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503391+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503391+0000","last_peered":"2024-09-10T10:44:00.503391+0000","last_clean":"2024-09-10T10:44:00.503391+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503391+0000","last_undegraded":"2024-09-10T10:44:00.503391+0000","last_fullsized":"2024-09-10T10:44:00.503391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.560860+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:45:18.560860+0000","last_peered":"2024-09-10T10:45:18.560860+0000","last_clean":"2024-09-10T10:45:18.560860+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:45:18.560860+0000","last_undegraded":"2024-09-10T10:45:18.560860+0000","last_fullsized":"2024-09-10T10:45:18.560860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442902+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:59.442902+0000","last_peered":"2024-09-10T10:44:59.442902+0000","last_clean":"2024-09-10T10:44:59.442902+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:59.442902+0000","last_undegraded":"2024-09-10T10:44:59.442902+0000","last_fullsized":"2024-09-10T10:44:59.442902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505985+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505985+0000","last_peered":"2024-09-10T10:44:00.505985+0000","last_clean":"2024-09-10T10:44:00.505985+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505985+0000","last_undegraded":"2024-09-10T10:44:00.505985+0000","last_fullsized":"2024-09-10T10:44:00.505985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442811+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:59.442811+0000","last_peered":"2024-09-10T10:44:59.442811+0000","last_clean":"2024-09-10T10:44:59.442811+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:59.442811+0000","last_undegraded":"2024-09-10T10:44:59.442811+0000","last_fullsized":"2024-09-10T10:44:59.442811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442546+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:59.442546+0000","last_peered":"2024-09-10T10:44:59.442546+0000","last_clean":"2024-09-10T10:44:59.442546+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:59.442546+0000","last_undegraded":"2024-09-10T10:44:59.442546+0000","last_fullsized":"2024-09-10T10:44:59.442546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503983+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503983+0000","last_peered":"2024-09-10T10:44:00.503983+0000","last_clean":"2024-09-10T10:44:00.503983+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503983+0000","last_undegraded":"2024-09-10T10:44:00.503983+0000","last_fullsized":"2024-09-10T10:44:00.503983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435612+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:59.435612+0000","last_peered":"2024-09-10T10:44:59.435612+0000","last_clean":"2024-09-10T10:44:59.435612+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:59.435612+0000","last_undegraded":"2024-09-10T10:44:59.435612+0000","last_fullsized":"2024-09-10T10:44:59.435612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.106863+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:45:01.106863+0000","last_peered":"2024-09-10T10:45:01.106863+0000","last_clean":"2024-09-10T10:45:01.106863+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:45:01.106863+0000","last_undegraded":"2024-09-10T10:45:01.106863+0000","last_fullsized":"2024-09-10T10:45:01.106863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.524134+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.524134+0000","last_peered":"2024-09-10T10:44:00.524134+0000","last_clean":"2024-09-10T10:44:00.524134+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.524134+0000","last_undegraded":"2024-09-10T10:44:00.524134+0000","last_fullsized":"2024-09-10T10:44:00.524134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437632+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:59.437632+0000","last_peered":"2024-09-10T10:44:59.437632+0000","last_clean":"2024-09-10T10:44:59.437632+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:59.437632+0000","last_undegraded":"2024-09-10T10:44:59.437632+0000","last_fullsized":"2024-09-10T10:44:59.437632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504947+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504947+0000","last_peered":"2024-09-10T10:44:00.504947+0000","last_clean":"2024-09-10T10:44:00.504947+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504947+0000","last_undegraded":"2024-09-10T10:44:00.504947+0000","last_fullsized":"2024-09-10T10:44:00.504947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500343+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500343+0000","last_peered":"2024-09-10T10:44:00.500343+0000","last_clean":"2024-09-10T10:44:00.500343+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500343+0000","last_undegraded":"2024-09-10T10:44:00.500343+0000","last_fullsized":"2024-09-10T10:44:00.500343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.422791+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:59.422791+0000","last_peered":"2024-09-10T10:44:59.422791+0000","last_clean":"2024-09-10T10:44:59.422791+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:59.422791+0000","last_undegraded":"2024-09-10T10:44:59.422791+0000","last_fullsized":"2024-09-10T10:44:59.422791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510061+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510061+0000","last_peered":"2024-09-10T10:44:00.510061+0000","last_clean":"2024-09-10T10:44:00.510061+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510061+0000","last_undegraded":"2024-09-10T10:44:00.510061+0000","last_fullsized":"2024-09-10T10:44:00.510061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504098+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504098+0000","last_peered":"2024-09-10T10:44:00.504098+0000","last_clean":"2024-09-10T10:44:00.504098+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504098+0000","last_undegraded":"2024-09-10T10:44:00.504098+0000","last_fullsized":"2024-09-10T10:44:00.504098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441460+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:59.441460+0000","last_peered":"2024-09-10T10:44:59.441460+0000","last_clean":"2024-09-10T10:44:59.441460+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:59.441460+0000","last_undegraded":"2024-09-10T10:44:59.441460+0000","last_fullsized":"2024-09-10T10:44:59.441460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504342+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504342+0000","last_peered":"2024-09-10T10:44:00.504342+0000","last_clean":"2024-09-10T10:44:00.504342+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504342+0000","last_undegraded":"2024-09-10T10:44:00.504342+0000","last_fullsized":"2024-09-10T10:44:00.504342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500934+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500934+0000","last_peered":"2024-09-10T10:44:00.500934+0000","last_clean":"2024-09-10T10:44:00.500934+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500934+0000","last_undegraded":"2024-09-10T10:44:00.500934+0000","last_fullsized":"2024-09-10T10:44:00.500934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440153+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:59.440153+0000","last_peered":"2024-09-10T10:44:59.440153+0000","last_clean":"2024-09-10T10:44:59.440153+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:59.440153+0000","last_undegraded":"2024-09-10T10:44:59.440153+0000","last_fullsized":"2024-09-10T10:44:59.440153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.480284+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:45:01.480284+0000","last_peered":"2024-09-10T10:45:01.480284+0000","last_clean":"2024-09-10T10:45:01.480284+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:45:01.480284+0000","last_undegraded":"2024-09-10T10:45:01.480284+0000","last_fullsized":"2024-09-10T10:45:01.480284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504339+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504339+0000","last_peered":"2024-09-10T10:44:00.504339+0000","last_clean":"2024-09-10T10:44:00.504339+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504339+0000","last_undegraded":"2024-09-10T10:44:00.504339+0000","last_fullsized":"2024-09-10T10:44:00.504339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443706+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:59.443706+0000","last_peered":"2024-09-10T10:44:59.443706+0000","last_clean":"2024-09-10T10:44:59.443706+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:59.443706+0000","last_undegraded":"2024-09-10T10:44:59.443706+0000","last_fullsized":"2024-09-10T10:44:59.443706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506338+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506338+0000","last_peered":"2024-09-10T10:44:00.506338+0000","last_clean":"2024-09-10T10:44:00.506338+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506338+0000","last_undegraded":"2024-09-10T10:44:00.506338+0000","last_fullsized":"2024-09-10T10:44:00.506338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503217+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503217+0000","last_peered":"2024-09-10T10:44:00.503217+0000","last_clean":"2024-09-10T10:44:00.503217+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503217+0000","last_undegraded":"2024-09-10T10:44:00.503217+0000","last_fullsized":"2024-09-10T10:44:00.503217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.424958+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:59.424958+0000","last_peered":"2024-09-10T10:44:59.424958+0000","last_clean":"2024-09-10T10:44:59.424958+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:59.424958+0000","last_undegraded":"2024-09-10T10:44:59.424958+0000","last_fullsized":"2024-09-10T10:44:59.424958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501376+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501376+0000","last_peered":"2024-09-10T10:44:00.501376+0000","last_clean":"2024-09-10T10:44:00.501376+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501376+0000","last_undegraded":"2024-09-10T10:44:00.501376+0000","last_fullsized":"2024-09-10T10:44:00.501376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505353+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505353+0000","last_peered":"2024-09-10T10:44:00.505353+0000","last_clean":"2024-09-10T10:44:00.505353+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505353+0000","last_undegraded":"2024-09-10T10:44:00.505353+0000","last_fullsized":"2024-09-10T10:44:00.505353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438858+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:59.438858+0000","last_peered":"2024-09-10T10:44:59.438858+0000","last_clean":"2024-09-10T10:44:59.438858+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:59.438858+0000","last_undegraded":"2024-09-10T10:44:59.438858+0000","last_fullsized":"2024-09-10T10:44:59.438858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.709932+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:45:18.709932+0000","last_peered":"2024-09-10T10:45:18.709932+0000","last_clean":"2024-09-10T10:45:18.709932+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:45:18.709932+0000","last_undegraded":"2024-09-10T10:45:18.709932+0000","last_fullsized":"2024-09-10T10:45:18.709932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502631+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502631+0000","last_peered":"2024-09-10T10:44:00.502631+0000","last_clean":"2024-09-10T10:44:00.502631+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502631+0000","last_undegraded":"2024-09-10T10:44:00.502631+0000","last_fullsized":"2024-09-10T10:44:00.502631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439106+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:59.439106+0000","last_peered":"2024-09-10T10:44:59.439106+0000","last_clean":"2024-09-10T10:44:59.439106+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:59.439106+0000","last_undegraded":"2024-09-10T10:44:59.439106+0000","last_fullsized":"2024-09-10T10:44:59.439106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:17.707309+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:45:17.707309+0000","last_peered":"2024-09-10T10:45:17.707309+0000","last_clean":"2024-09-10T10:45:17.707309+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:45:17.707309+0000","last_undegraded":"2024-09-10T10:45:17.707309+0000","last_fullsized":"2024-09-10T10:45:17.707309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502777+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502777+0000","last_peered":"2024-09-10T10:44:00.502777+0000","last_clean":"2024-09-10T10:44:00.502777+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502777+0000","last_undegraded":"2024-09-10T10:44:00.502777+0000","last_fullsized":"2024-09-10T10:44:00.502777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444099+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:59.444099+0000","last_peered":"2024-09-10T10:44:59.444099+0000","last_clean":"2024-09-10T10:44:59.444099+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:59.444099+0000","last_undegraded":"2024-09-10T10:44:59.444099+0000","last_fullsized":"2024-09-10T10:44:59.444099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506093+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506093+0000","last_peered":"2024-09-10T10:44:00.506093+0000","last_clean":"2024-09-10T10:44:00.506093+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506093+0000","last_undegraded":"2024-09-10T10:44:00.506093+0000","last_fullsized":"2024-09-10T10:44:00.506093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516092+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.516092+0000","last_peered":"2024-09-10T10:44:00.516092+0000","last_clean":"2024-09-10T10:44:00.516092+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.516092+0000","last_undegraded":"2024-09-10T10:44:00.516092+0000","last_fullsized":"2024-09-10T10:44:00.516092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443062+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:59.443062+0000","last_peered":"2024-09-10T10:44:59.443062+0000","last_clean":"2024-09-10T10:44:59.443062+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:59.443062+0000","last_undegraded":"2024-09-10T10:44:59.443062+0000","last_fullsized":"2024-09-10T10:44:59.443062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503953+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503953+0000","last_peered":"2024-09-10T10:44:00.503953+0000","last_clean":"2024-09-10T10:44:00.503953+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503953+0000","last_undegraded":"2024-09-10T10:44:00.503953+0000","last_fullsized":"2024-09-10T10:44:00.503953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507101+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507101+0000","last_peered":"2024-09-10T10:44:00.507101+0000","last_clean":"2024-09-10T10:44:00.507101+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507101+0000","last_undegraded":"2024-09-10T10:44:00.507101+0000","last_fullsized":"2024-09-10T10:44:00.507101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442450+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:59.442450+0000","last_peered":"2024-09-10T10:44:59.442450+0000","last_clean":"2024-09-10T10:44:59.442450+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:59.442450+0000","last_undegraded":"2024-09-10T10:44:59.442450+0000","last_fullsized":"2024-09-10T10:44:59.442450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:19.763596+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:45:19.763596+0000","last_peered":"2024-09-10T10:45:19.763596+0000","last_clean":"2024-09-10T10:45:19.763596+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:45:19.763596+0000","last_undegraded":"2024-09-10T10:45:19.763596+0000","last_fullsized":"2024-09-10T10:45:19.763596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509662+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509662+0000","last_peered":"2024-09-10T10:44:00.509662+0000","last_clean":"2024-09-10T10:44:00.509662+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509662+0000","last_undegraded":"2024-09-10T10:44:00.509662+0000","last_fullsized":"2024-09-10T10:44:00.509662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442843+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:59.442843+0000","last_peered":"2024-09-10T10:44:59.442843+0000","last_clean":"2024-09-10T10:44:59.442843+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:59.442843+0000","last_undegraded":"2024-09-10T10:44:59.442843+0000","last_fullsized":"2024-09-10T10:44:59.442843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516384+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516384+0000","last_peered":"2024-09-10T10:44:00.516384+0000","last_clean":"2024-09-10T10:44:00.516384+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516384+0000","last_undegraded":"2024-09-10T10:44:00.516384+0000","last_fullsized":"2024-09-10T10:44:00.516384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500645+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500645+0000","last_peered":"2024-09-10T10:44:00.500645+0000","last_clean":"2024-09-10T10:44:00.500645+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500645+0000","last_undegraded":"2024-09-10T10:44:00.500645+0000","last_fullsized":"2024-09-10T10:44:00.500645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440420+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:59.440420+0000","last_peered":"2024-09-10T10:44:59.440420+0000","last_clean":"2024-09-10T10:44:59.440420+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:59.440420+0000","last_undegraded":"2024-09-10T10:44:59.440420+0000","last_fullsized":"2024-09-10T10:44:59.440420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503016+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.503016+0000","last_peered":"2024-09-10T10:44:00.503016+0000","last_clean":"2024-09-10T10:44:00.503016+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.503016+0000","last_undegraded":"2024-09-10T10:44:00.503016+0000","last_fullsized":"2024-09-10T10:44:00.503016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515085+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515085+0000","last_peered":"2024-09-10T10:44:00.515085+0000","last_clean":"2024-09-10T10:44:00.515085+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515085+0000","last_undegraded":"2024-09-10T10:44:00.515085+0000","last_fullsized":"2024-09-10T10:44:00.515085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440308+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:59.440308+0000","last_peered":"2024-09-10T10:44:59.440308+0000","last_clean":"2024-09-10T10:44:59.440308+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:59.440308+0000","last_undegraded":"2024-09-10T10:44:59.440308+0000","last_fullsized":"2024-09-10T10:44:59.440308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:05.823762+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:45:05.823762+0000","last_peered":"2024-09-10T10:45:05.823762+0000","last_clean":"2024-09-10T10:45:05.823762+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:45:05.823762+0000","last_undegraded":"2024-09-10T10:45:05.823762+0000","last_fullsized":"2024-09-10T10:45:05.823762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515302+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515302+0000","last_peered":"2024-09-10T10:44:00.515302+0000","last_clean":"2024-09-10T10:44:00.515302+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515302+0000","last_undegraded":"2024-09-10T10:44:00.515302+0000","last_fullsized":"2024-09-10T10:44:00.515302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442616+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442616+0000","last_peered":"2024-09-10T10:44:59.442616+0000","last_clean":"2024-09-10T10:44:59.442616+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:59.442616+0000","last_undegraded":"2024-09-10T10:44:59.442616+0000","last_fullsized":"2024-09-10T10:44:59.442616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505428+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505428+0000","last_peered":"2024-09-10T10:44:00.505428+0000","last_clean":"2024-09-10T10:44:00.505428+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505428+0000","last_undegraded":"2024-09-10T10:44:00.505428+0000","last_fullsized":"2024-09-10T10:44:00.505428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506639+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.506639+0000","last_peered":"2024-09-10T10:44:00.506639+0000","last_clean":"2024-09-10T10:44:00.506639+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.506639+0000","last_undegraded":"2024-09-10T10:44:00.506639+0000","last_fullsized":"2024-09-10T10:44:00.506639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441344+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:59.441344+0000","last_peered":"2024-09-10T10:44:59.441344+0000","last_clean":"2024-09-10T10:44:59.441344+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:59.441344+0000","last_undegraded":"2024-09-10T10:44:59.441344+0000","last_fullsized":"2024-09-10T10:44:59.441344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:16.706499+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:45:16.706499+0000","last_peered":"2024-09-10T10:45:16.706499+0000","last_clean":"2024-09-10T10:45:16.706499+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:45:16.706499+0000","last_undegraded":"2024-09-10T10:45:16.706499+0000","last_fullsized":"2024-09-10T10:45:16.706499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500337+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500337+0000","last_peered":"2024-09-10T10:44:00.500337+0000","last_clean":"2024-09-10T10:44:00.500337+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500337+0000","last_undegraded":"2024-09-10T10:44:00.500337+0000","last_fullsized":"2024-09-10T10:44:00.500337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445872+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:59.445872+0000","last_peered":"2024-09-10T10:44:59.445872+0000","last_clean":"2024-09-10T10:44:59.445872+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:59.445872+0000","last_undegraded":"2024-09-10T10:44:59.445872+0000","last_fullsized":"2024-09-10T10:44:59.445872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515497+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515497+0000","last_peered":"2024-09-10T10:44:00.515497+0000","last_clean":"2024-09-10T10:44:00.515497+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515497+0000","last_undegraded":"2024-09-10T10:44:00.515497+0000","last_fullsized":"2024-09-10T10:44:00.515497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502859+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502859+0000","last_peered":"2024-09-10T10:44:00.502859+0000","last_clean":"2024-09-10T10:44:00.502859+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502859+0000","last_undegraded":"2024-09-10T10:44:00.502859+0000","last_fullsized":"2024-09-10T10:44:00.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435638+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:59.435638+0000","last_peered":"2024-09-10T10:44:59.435638+0000","last_clean":"2024-09-10T10:44:59.435638+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:59.435638+0000","last_undegraded":"2024-09-10T10:44:59.435638+0000","last_fullsized":"2024-09-10T10:44:59.435638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516076+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516076+0000","last_peered":"2024-09-10T10:44:00.516076+0000","last_clean":"2024-09-10T10:44:00.516076+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516076+0000","last_undegraded":"2024-09-10T10:44:00.516076+0000","last_fullsized":"2024-09-10T10:44:00.516076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":311,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.623041+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:45:15.623041+0000","last_peered":"2024-09-10T10:45:15.623041+0000","last_clean":"2024-09-10T10:45:15.623041+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:45:15.623041+0000","last_undegraded":"2024-09-10T10:45:15.623041+0000","last_fullsized":"2024-09-10T10:45:15.623041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62820,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":167,"num_write":41,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500085+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500085+0000","last_peered":"2024-09-10T10:44:00.500085+0000","last_clean":"2024-09-10T10:44:00.500085+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500085+0000","last_undegraded":"2024-09-10T10:44:00.500085+0000","last_fullsized":"2024-09-10T10:44:00.500085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516289+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516289+0000","last_peered":"2024-09-10T10:44:00.516289+0000","last_clean":"2024-09-10T10:44:00.516289+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516289+0000","last_undegraded":"2024-09-10T10:44:00.516289+0000","last_fullsized":"2024-09-10T10:44:00.516289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503778+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503778+0000","last_peered":"2024-09-10T10:44:00.503778+0000","last_clean":"2024-09-10T10:44:00.503778+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503778+0000","last_undegraded":"2024-09-10T10:44:00.503778+0000","last_fullsized":"2024-09-10T10:44:00.503778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506656+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506656+0000","last_peered":"2024-09-10T10:44:00.506656+0000","last_clean":"2024-09-10T10:44:00.506656+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506656+0000","last_undegraded":"2024-09-10T10:44:00.506656+0000","last_fullsized":"2024-09-10T10:44:00.506656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499856+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499856+0000","last_peered":"2024-09-10T10:44:00.499856+0000","last_clean":"2024-09-10T10:44:00.499856+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499856+0000","last_undegraded":"2024-09-10T10:44:00.499856+0000","last_fullsized":"2024-09-10T10:44:00.499856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442320+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:59.442320+0000","last_peered":"2024-09-10T10:44:59.442320+0000","last_clean":"2024-09-10T10:44:59.442320+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:59.442320+0000","last_undegraded":"2024-09-10T10:44:59.442320+0000","last_fullsized":"2024-09-10T10:44:59.442320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.756025+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:45:15.756025+0000","last_peered":"2024-09-10T10:45:15.756025+0000","last_clean":"2024-09-10T10:45:15.756025+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:45:15.756025+0000","last_undegraded":"2024-09-10T10:45:15.756025+0000","last_fullsized":"2024-09-10T10:45:15.756025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504723+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504723+0000","last_peered":"2024-09-10T10:44:00.504723+0000","last_clean":"2024-09-10T10:44:00.504723+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504723+0000","last_undegraded":"2024-09-10T10:44:00.504723+0000","last_fullsized":"2024-09-10T10:44:00.504723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500929+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500929+0000","last_peered":"2024-09-10T10:44:00.500929+0000","last_clean":"2024-09-10T10:44:00.500929+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500929+0000","last_undegraded":"2024-09-10T10:44:00.500929+0000","last_fullsized":"2024-09-10T10:44:00.500929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432970+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:59.432970+0000","last_peered":"2024-09-10T10:44:59.432970+0000","last_clean":"2024-09-10T10:44:59.432970+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:59.432970+0000","last_undegraded":"2024-09-10T10:44:59.432970+0000","last_fullsized":"2024-09-10T10:44:59.432970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515598+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515598+0000","last_peered":"2024-09-10T10:44:00.515598+0000","last_clean":"2024-09-10T10:44:00.515598+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515598+0000","last_undegraded":"2024-09-10T10:44:00.515598+0000","last_fullsized":"2024-09-10T10:44:00.515598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509926+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509926+0000","last_peered":"2024-09-10T10:44:00.509926+0000","last_clean":"2024-09-10T10:44:00.509926+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509926+0000","last_undegraded":"2024-09-10T10:44:00.509926+0000","last_fullsized":"2024-09-10T10:44:00.509926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.434606+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:59.434606+0000","last_peered":"2024-09-10T10:44:59.434606+0000","last_clean":"2024-09-10T10:44:59.434606+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:59.434606+0000","last_undegraded":"2024-09-10T10:44:59.434606+0000","last_fullsized":"2024-09-10T10:44:59.434606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502516+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502516+0000","last_peered":"2024-09-10T10:44:00.502516+0000","last_clean":"2024-09-10T10:44:00.502516+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502516+0000","last_undegraded":"2024-09-10T10:44:00.502516+0000","last_fullsized":"2024-09-10T10:44:00.502516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505325+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505325+0000","last_peered":"2024-09-10T10:44:00.505325+0000","last_clean":"2024-09-10T10:44:00.505325+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505325+0000","last_undegraded":"2024-09-10T10:44:00.505325+0000","last_fullsized":"2024-09-10T10:44:00.505325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438031+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:59.438031+0000","last_peered":"2024-09-10T10:44:59.438031+0000","last_clean":"2024-09-10T10:44:59.438031+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:59.438031+0000","last_undegraded":"2024-09-10T10:44:59.438031+0000","last_fullsized":"2024-09-10T10:44:59.438031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.707305+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:45:15.707305+0000","last_peered":"2024-09-10T10:45:15.707305+0000","last_clean":"2024-09-10T10:45:15.707305+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:45:15.707305+0000","last_undegraded":"2024-09-10T10:45:15.707305+0000","last_fullsized":"2024-09-10T10:45:15.707305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439297+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:59.439297+0000","last_peered":"2024-09-10T10:44:59.439297+0000","last_clean":"2024-09-10T10:44:59.439297+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:59.439297+0000","last_undegraded":"2024-09-10T10:44:59.439297+0000","last_fullsized":"2024-09-10T10:44:59.439297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440974+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:59.440974+0000","last_peered":"2024-09-10T10:44:59.440974+0000","last_clean":"2024-09-10T10:44:59.440974+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:59.440974+0000","last_undegraded":"2024-09-10T10:44:59.440974+0000","last_fullsized":"2024-09-10T10:44:59.440974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441221+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:59.441221+0000","last_peered":"2024-09-10T10:44:59.441221+0000","last_clean":"2024-09-10T10:44:59.441221+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:59.441221+0000","last_undegraded":"2024-09-10T10:44:59.441221+0000","last_fullsized":"2024-09-10T10:44:59.441221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440703+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:59.440703+0000","last_peered":"2024-09-10T10:44:59.440703+0000","last_clean":"2024-09-10T10:44:59.440703+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:59.440703+0000","last_undegraded":"2024-09-10T10:44:59.440703+0000","last_fullsized":"2024-09-10T10:44:59.440703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444329+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:59.444329+0000","last_peered":"2024-09-10T10:44:59.444329+0000","last_clean":"2024-09-10T10:44:59.444329+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:59.444329+0000","last_undegraded":"2024-09-10T10:44:59.444329+0000","last_fullsized":"2024-09-10T10:44:59.444329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438450+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:59.438450+0000","last_peered":"2024-09-10T10:44:59.438450+0000","last_clean":"2024-09-10T10:44:59.438450+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:59.438450+0000","last_undegraded":"2024-09-10T10:44:59.438450+0000","last_fullsized":"2024-09-10T10:44:59.438450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441275+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:59.441275+0000","last_peered":"2024-09-10T10:44:59.441275+0000","last_clean":"2024-09-10T10:44:59.441275+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:59.441275+0000","last_undegraded":"2024-09-10T10:44:59.441275+0000","last_fullsized":"2024-09-10T10:44:59.441275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437164+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:59.437164+0000","last_peered":"2024-09-10T10:44:59.437164+0000","last_clean":"2024-09-10T10:44:59.437164+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:59.437164+0000","last_undegraded":"2024-09-10T10:44:59.437164+0000","last_fullsized":"2024-09-10T10:44:59.437164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441514+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:59.441514+0000","last_peered":"2024-09-10T10:44:59.441514+0000","last_clean":"2024-09-10T10:44:59.441514+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:59.441514+0000","last_undegraded":"2024-09-10T10:44:59.441514+0000","last_fullsized":"2024-09-10T10:44:59.441514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442360+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:59.442360+0000","last_peered":"2024-09-10T10:44:59.442360+0000","last_clean":"2024-09-10T10:44:59.442360+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:59.442360+0000","last_undegraded":"2024-09-10T10:44:59.442360+0000","last_fullsized":"2024-09-10T10:44:59.442360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442124+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:59.442124+0000","last_peered":"2024-09-10T10:44:59.442124+0000","last_clean":"2024-09-10T10:44:59.442124+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:59.442124+0000","last_undegraded":"2024-09-10T10:44:59.442124+0000","last_fullsized":"2024-09-10T10:44:59.442124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442058+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:59.442058+0000","last_peered":"2024-09-10T10:44:59.442058+0000","last_clean":"2024-09-10T10:44:59.442058+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:59.442058+0000","last_undegraded":"2024-09-10T10:44:59.442058+0000","last_fullsized":"2024-09-10T10:44:59.442058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438851+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:59.438851+0000","last_peered":"2024-09-10T10:44:59.438851+0000","last_clean":"2024-09-10T10:44:59.438851+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:59.438851+0000","last_undegraded":"2024-09-10T10:44:59.438851+0000","last_fullsized":"2024-09-10T10:44:59.438851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429557+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:59.429557+0000","last_peered":"2024-09-10T10:44:59.429557+0000","last_clean":"2024-09-10T10:44:59.429557+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:59.429557+0000","last_undegraded":"2024-09-10T10:44:59.429557+0000","last_fullsized":"2024-09-10T10:44:59.429557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440551+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:59.440551+0000","last_peered":"2024-09-10T10:44:59.440551+0000","last_clean":"2024-09-10T10:44:59.440551+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:59.440551+0000","last_undegraded":"2024-09-10T10:44:59.440551+0000","last_fullsized":"2024-09-10T10:44:59.440551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437183+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:59.437183+0000","last_peered":"2024-09-10T10:44:59.437183+0000","last_clean":"2024-09-10T10:44:59.437183+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:59.437183+0000","last_undegraded":"2024-09-10T10:44:59.437183+0000","last_fullsized":"2024-09-10T10:44:59.437183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439780+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:59.439780+0000","last_peered":"2024-09-10T10:44:59.439780+0000","last_clean":"2024-09-10T10:44:59.439780+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:59.439780+0000","last_undegraded":"2024-09-10T10:44:59.439780+0000","last_fullsized":"2024-09-10T10:44:59.439780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441856+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:59.441856+0000","last_peered":"2024-09-10T10:44:59.441856+0000","last_clean":"2024-09-10T10:44:59.441856+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:59.441856+0000","last_undegraded":"2024-09-10T10:44:59.441856+0000","last_fullsized":"2024-09-10T10:44:59.441856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437328+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:59.437328+0000","last_peered":"2024-09-10T10:44:59.437328+0000","last_clean":"2024-09-10T10:44:59.437328+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:59.437328+0000","last_undegraded":"2024-09-10T10:44:59.437328+0000","last_fullsized":"2024-09-10T10:44:59.437328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438618+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:59.438618+0000","last_peered":"2024-09-10T10:44:59.438618+0000","last_clean":"2024-09-10T10:44:59.438618+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:59.438618+0000","last_undegraded":"2024-09-10T10:44:59.438618+0000","last_fullsized":"2024-09-10T10:44:59.438618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444056+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:59.444056+0000","last_peered":"2024-09-10T10:44:59.444056+0000","last_clean":"2024-09-10T10:44:59.444056+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:59.444056+0000","last_undegraded":"2024-09-10T10:44:59.444056+0000","last_fullsized":"2024-09-10T10:44:59.444056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442679+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:59.442679+0000","last_peered":"2024-09-10T10:44:59.442679+0000","last_clean":"2024-09-10T10:44:59.442679+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:59.442679+0000","last_undegraded":"2024-09-10T10:44:59.442679+0000","last_fullsized":"2024-09-10T10:44:59.442679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432695+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:59.432695+0000","last_peered":"2024-09-10T10:44:59.432695+0000","last_clean":"2024-09-10T10:44:59.432695+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:59.432695+0000","last_undegraded":"2024-09-10T10:44:59.432695+0000","last_fullsized":"2024-09-10T10:44:59.432695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442732+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442732+0000","last_peered":"2024-09-10T10:44:59.442732+0000","last_clean":"2024-09-10T10:44:59.442732+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:59.442732+0000","last_undegraded":"2024-09-10T10:44:59.442732+0000","last_fullsized":"2024-09-10T10:44:59.442732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440814+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:59.440814+0000","last_peered":"2024-09-10T10:44:59.440814+0000","last_clean":"2024-09-10T10:44:59.440814+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:59.440814+0000","last_undegraded":"2024-09-10T10:44:59.440814+0000","last_fullsized":"2024-09-10T10:44:59.440814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444968+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:59.444968+0000","last_peered":"2024-09-10T10:44:59.444968+0000","last_clean":"2024-09-10T10:44:59.444968+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:59.444968+0000","last_undegraded":"2024-09-10T10:44:59.444968+0000","last_fullsized":"2024-09-10T10:44:59.444968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428461+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:59.428461+0000","last_peered":"2024-09-10T10:44:59.428461+0000","last_clean":"2024-09-10T10:44:59.428461+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:59.428461+0000","last_undegraded":"2024-09-10T10:44:59.428461+0000","last_fullsized":"2024-09-10T10:44:59.428461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428915+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:59.428915+0000","last_peered":"2024-09-10T10:44:59.428915+0000","last_clean":"2024-09-10T10:44:59.428915+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:59.428915+0000","last_undegraded":"2024-09-10T10:44:59.428915+0000","last_fullsized":"2024-09-10T10:44:59.428915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441024+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:59.441024+0000","last_peered":"2024-09-10T10:44:59.441024+0000","last_clean":"2024-09-10T10:44:59.441024+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:59.441024+0000","last_undegraded":"2024-09-10T10:44:59.441024+0000","last_fullsized":"2024-09-10T10:44:59.441024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441764+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:59.441764+0000","last_peered":"2024-09-10T10:44:59.441764+0000","last_clean":"2024-09-10T10:44:59.441764+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:59.441764+0000","last_undegraded":"2024-09-10T10:44:59.441764+0000","last_fullsized":"2024-09-10T10:44:59.441764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439409+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:59.439409+0000","last_peered":"2024-09-10T10:44:59.439409+0000","last_clean":"2024-09-10T10:44:59.439409+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:59.439409+0000","last_undegraded":"2024-09-10T10:44:59.439409+0000","last_fullsized":"2024-09-10T10:44:59.439409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439118+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:59.439118+0000","last_peered":"2024-09-10T10:44:59.439118+0000","last_clean":"2024-09-10T10:44:59.439118+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:59.439118+0000","last_undegraded":"2024-09-10T10:44:59.439118+0000","last_fullsized":"2024-09-10T10:44:59.439118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72683,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1093,"num_read_kb":623,"num_write":276,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":229983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":501134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1154798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939701,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168635,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1098863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":496359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":563394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1024000,"data_stored":498177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":567436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:21.821 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T10:45:21.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:21.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- 172.21.15.143:0/3299413309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28101c00 msgr2=0x7fdb28102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- 172.21.15.143:0/3299413309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28101c00 0x7fdb28102080 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009900 tx=0x7fdb1402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:21.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- 172.21.15.143:0/3299413309 shutdown_connections 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- 172.21.15.143:0/3299413309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb281025c0 0x7fdb28108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- 172.21.15.143:0/3299413309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28101c00 0x7fdb28102080 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- 172.21.15.143:0/3299413309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb28100a10 0x7fdb28100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- 172.21.15.143:0/3299413309 >> 172.21.15.143:0/3299413309 conn(0x7fdb280fc170 msgr2=0x7fdb280fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- 172.21.15.143:0/3299413309 shutdown_connections 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- 172.21.15.143:0/3299413309 wait complete. 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 Processor -- start 2024-09-10T10:45:21.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- start start 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb28100a10 0x7fdb281049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb28101c00 0x7fdb28102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 0x7fdb28103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2810c540 con 0x7fdb281025c0 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2810c3c0 con 0x7fdb28100a10 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.954+0000 7fdb2f131640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2810c6c0 con 0x7fdb28101c00 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 0x7fdb28103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 0x7fdb28103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42448/0 (socket says 172.21.15.143:42448) 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 -- 172.21.15.143:0/4011271376 learned_addr learned my addr 172.21.15.143:0/4011271376 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb27fff640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb28101c00 0x7fdb28102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2cecd640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb28100a10 0x7fdb281049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 -- 172.21.15.143:0/4011271376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb28101c00 msgr2=0x7fdb28102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb28101c00 0x7fdb28102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 -- 172.21.15.143:0/4011271376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb28100a10 msgr2=0x7fdb281049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb28100a10 0x7fdb281049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 -- 172.21.15.143:0/4011271376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb28103db0 con 0x7fdb281025c0 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2d6ce640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 0x7fdb28103520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000eb10 tx=0x7fdb2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20004b20 con 0x7fdb281025c0 2024-09-10T10:45:21.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb20004cc0 con 0x7fdb281025c0 2024-09-10T10:45:21.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb281a1ea0 con 0x7fdb281025c0 2024-09-10T10:45:21.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20010690 con 0x7fdb281025c0 2024-09-10T10:45:21.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.955+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb281a2300 con 0x7fdb281025c0 2024-09-10T10:45:21.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.956+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb281a25f0 con 0x7fdb281025c0 2024-09-10T10:45:21.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.957+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fdb20005020 con 0x7fdb281025c0 2024-09-10T10:45:21.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.958+0000 7fdb25ffb640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdb180a2c20 0x7fdb180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:21.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.958+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdb200cfca0 con 0x7fdb281025c0 2024-09-10T10:45:21.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.958+0000 7fdb2cecd640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdb180a2c20 0x7fdb180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:21.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.959+0000 7fdb2cecd640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdb180a2c20 0x7fdb180a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c005fd0 tx=0x7fdb1c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:21.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:21.962+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb2009c740 con 0x7fdb281025c0 2024-09-10T10:45:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.125+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb28069b00 con 0x7fdb180a2c20 2024-09-10T10:45:22.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.133+0000 7fdb25ffb640 1 -- 172.21.15.143:0/4011271376 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340563 (secure 0 0 0) 0x7fdb28069b00 con 0x7fdb180a2c20 2024-09-10T10:45:22.138 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:22.140 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:45:22.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdb180a2c20 msgr2=0x7fdb180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdb180a2c20 0x7fdb180a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c005fd0 tx=0x7fdb1c005850 comp rx=0 tx=0).stop 2024-09-10T10:45:22.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 msgr2=0x7fdb28103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 0x7fdb28103520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000eb10 tx=0x7fdb2000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 shutdown_connections 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdb180a2c20 0x7fdb180a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281025c0 0x7fdb28103520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb28101c00 0x7fdb28102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 --2- 172.21.15.143:0/4011271376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb28100a10 0x7fdb281049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 >> 172.21.15.143:0/4011271376 conn(0x7fdb280fc170 msgr2=0x7fdb280693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 shutdown_connections 2024-09-10T10:45:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.137+0000 7fdb2f131640 1 -- 172.21.15.143:0/4011271376 wait complete. 2024-09-10T10:45:22.148 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T10:45:21.469308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663285,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1219,"num_read_kb":732,"num_write":459,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375780,"kb_used_data":14052,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566044,"statfs":{"total":1151940427776,"available":1151555629056,"internally_reserved":0,"allocated":14389248,"data_stored":7966136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2868,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":61,"num_write":19,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003164"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:14.902511+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:45:14.902511+0000","last_peered":"2024-09-10T10:45:14.902511+0000","last_clean":"2024-09-10T10:45:14.902511+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:45:14.902511+0000","last_undegraded":"2024-09-10T10:45:14.902511+0000","last_fullsized":"2024-09-10T10:45:14.902511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441227+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:59.441227+0000","last_peered":"2024-09-10T10:44:59.441227+0000","last_clean":"2024-09-10T10:44:59.441227+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:59.441227+0000","last_undegraded":"2024-09-10T10:44:59.441227+0000","last_fullsized":"2024-09-10T10:44:59.441227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500745+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500745+0000","last_peered":"2024-09-10T10:44:00.500745+0000","last_clean":"2024-09-10T10:44:00.500745+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500745+0000","last_undegraded":"2024-09-10T10:44:00.500745+0000","last_fullsized":"2024-09-10T10:44:00.500745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505312+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505312+0000","last_peered":"2024-09-10T10:44:00.505312+0000","last_clean":"2024-09-10T10:44:00.505312+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505312+0000","last_undegraded":"2024-09-10T10:44:00.505312+0000","last_fullsized":"2024-09-10T10:44:00.505312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440037+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:59.440037+0000","last_peered":"2024-09-10T10:44:59.440037+0000","last_clean":"2024-09-10T10:44:59.440037+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:59.440037+0000","last_undegraded":"2024-09-10T10:44:59.440037+0000","last_fullsized":"2024-09-10T10:44:59.440037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503391+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503391+0000","last_peered":"2024-09-10T10:44:00.503391+0000","last_clean":"2024-09-10T10:44:00.503391+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503391+0000","last_undegraded":"2024-09-10T10:44:00.503391+0000","last_fullsized":"2024-09-10T10:44:00.503391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.560860+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:45:18.560860+0000","last_peered":"2024-09-10T10:45:18.560860+0000","last_clean":"2024-09-10T10:45:18.560860+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:45:18.560860+0000","last_undegraded":"2024-09-10T10:45:18.560860+0000","last_fullsized":"2024-09-10T10:45:18.560860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442902+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:59.442902+0000","last_peered":"2024-09-10T10:44:59.442902+0000","last_clean":"2024-09-10T10:44:59.442902+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:59.442902+0000","last_undegraded":"2024-09-10T10:44:59.442902+0000","last_fullsized":"2024-09-10T10:44:59.442902+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505985+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505985+0000","last_peered":"2024-09-10T10:44:00.505985+0000","last_clean":"2024-09-10T10:44:00.505985+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505985+0000","last_undegraded":"2024-09-10T10:44:00.505985+0000","last_fullsized":"2024-09-10T10:44:00.505985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442811+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:59.442811+0000","last_peered":"2024-09-10T10:44:59.442811+0000","last_clean":"2024-09-10T10:44:59.442811+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:59.442811+0000","last_undegraded":"2024-09-10T10:44:59.442811+0000","last_fullsized":"2024-09-10T10:44:59.442811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442546+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:59.442546+0000","last_peered":"2024-09-10T10:44:59.442546+0000","last_clean":"2024-09-10T10:44:59.442546+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:59.442546+0000","last_undegraded":"2024-09-10T10:44:59.442546+0000","last_fullsized":"2024-09-10T10:44:59.442546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503983+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503983+0000","last_peered":"2024-09-10T10:44:00.503983+0000","last_clean":"2024-09-10T10:44:00.503983+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503983+0000","last_undegraded":"2024-09-10T10:44:00.503983+0000","last_fullsized":"2024-09-10T10:44:00.503983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435612+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:59.435612+0000","last_peered":"2024-09-10T10:44:59.435612+0000","last_clean":"2024-09-10T10:44:59.435612+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:59.435612+0000","last_undegraded":"2024-09-10T10:44:59.435612+0000","last_fullsized":"2024-09-10T10:44:59.435612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.106863+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:45:01.106863+0000","last_peered":"2024-09-10T10:45:01.106863+0000","last_clean":"2024-09-10T10:45:01.106863+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:45:01.106863+0000","last_undegraded":"2024-09-10T10:45:01.106863+0000","last_fullsized":"2024-09-10T10:45:01.106863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.524134+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.524134+0000","last_peered":"2024-09-10T10:44:00.524134+0000","last_clean":"2024-09-10T10:44:00.524134+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.524134+0000","last_undegraded":"2024-09-10T10:44:00.524134+0000","last_fullsized":"2024-09-10T10:44:00.524134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437632+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:59.437632+0000","last_peered":"2024-09-10T10:44:59.437632+0000","last_clean":"2024-09-10T10:44:59.437632+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:59.437632+0000","last_undegraded":"2024-09-10T10:44:59.437632+0000","last_fullsized":"2024-09-10T10:44:59.437632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504947+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504947+0000","last_peered":"2024-09-10T10:44:00.504947+0000","last_clean":"2024-09-10T10:44:00.504947+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504947+0000","last_undegraded":"2024-09-10T10:44:00.504947+0000","last_fullsized":"2024-09-10T10:44:00.504947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500343+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500343+0000","last_peered":"2024-09-10T10:44:00.500343+0000","last_clean":"2024-09-10T10:44:00.500343+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500343+0000","last_undegraded":"2024-09-10T10:44:00.500343+0000","last_fullsized":"2024-09-10T10:44:00.500343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.422791+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:59.422791+0000","last_peered":"2024-09-10T10:44:59.422791+0000","last_clean":"2024-09-10T10:44:59.422791+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:59.422791+0000","last_undegraded":"2024-09-10T10:44:59.422791+0000","last_fullsized":"2024-09-10T10:44:59.422791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510061+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510061+0000","last_peered":"2024-09-10T10:44:00.510061+0000","last_clean":"2024-09-10T10:44:00.510061+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510061+0000","last_undegraded":"2024-09-10T10:44:00.510061+0000","last_fullsized":"2024-09-10T10:44:00.510061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504098+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504098+0000","last_peered":"2024-09-10T10:44:00.504098+0000","last_clean":"2024-09-10T10:44:00.504098+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504098+0000","last_undegraded":"2024-09-10T10:44:00.504098+0000","last_fullsized":"2024-09-10T10:44:00.504098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441460+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:59.441460+0000","last_peered":"2024-09-10T10:44:59.441460+0000","last_clean":"2024-09-10T10:44:59.441460+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:59.441460+0000","last_undegraded":"2024-09-10T10:44:59.441460+0000","last_fullsized":"2024-09-10T10:44:59.441460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504342+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504342+0000","last_peered":"2024-09-10T10:44:00.504342+0000","last_clean":"2024-09-10T10:44:00.504342+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504342+0000","last_undegraded":"2024-09-10T10:44:00.504342+0000","last_fullsized":"2024-09-10T10:44:00.504342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500934+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500934+0000","last_peered":"2024-09-10T10:44:00.500934+0000","last_clean":"2024-09-10T10:44:00.500934+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500934+0000","last_undegraded":"2024-09-10T10:44:00.500934+0000","last_fullsized":"2024-09-10T10:44:00.500934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440153+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:59.440153+0000","last_peered":"2024-09-10T10:44:59.440153+0000","last_clean":"2024-09-10T10:44:59.440153+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:59.440153+0000","last_undegraded":"2024-09-10T10:44:59.440153+0000","last_fullsized":"2024-09-10T10:44:59.440153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.480284+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:45:01.480284+0000","last_peered":"2024-09-10T10:45:01.480284+0000","last_clean":"2024-09-10T10:45:01.480284+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:45:01.480284+0000","last_undegraded":"2024-09-10T10:45:01.480284+0000","last_fullsized":"2024-09-10T10:45:01.480284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504339+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504339+0000","last_peered":"2024-09-10T10:44:00.504339+0000","last_clean":"2024-09-10T10:44:00.504339+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504339+0000","last_undegraded":"2024-09-10T10:44:00.504339+0000","last_fullsized":"2024-09-10T10:44:00.504339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443706+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:59.443706+0000","last_peered":"2024-09-10T10:44:59.443706+0000","last_clean":"2024-09-10T10:44:59.443706+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:59.443706+0000","last_undegraded":"2024-09-10T10:44:59.443706+0000","last_fullsized":"2024-09-10T10:44:59.443706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506338+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506338+0000","last_peered":"2024-09-10T10:44:00.506338+0000","last_clean":"2024-09-10T10:44:00.506338+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506338+0000","last_undegraded":"2024-09-10T10:44:00.506338+0000","last_fullsized":"2024-09-10T10:44:00.506338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503217+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503217+0000","last_peered":"2024-09-10T10:44:00.503217+0000","last_clean":"2024-09-10T10:44:00.503217+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503217+0000","last_undegraded":"2024-09-10T10:44:00.503217+0000","last_fullsized":"2024-09-10T10:44:00.503217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.424958+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:59.424958+0000","last_peered":"2024-09-10T10:44:59.424958+0000","last_clean":"2024-09-10T10:44:59.424958+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:59.424958+0000","last_undegraded":"2024-09-10T10:44:59.424958+0000","last_fullsized":"2024-09-10T10:44:59.424958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501376+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501376+0000","last_peered":"2024-09-10T10:44:00.501376+0000","last_clean":"2024-09-10T10:44:00.501376+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501376+0000","last_undegraded":"2024-09-10T10:44:00.501376+0000","last_fullsized":"2024-09-10T10:44:00.501376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505353+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505353+0000","last_peered":"2024-09-10T10:44:00.505353+0000","last_clean":"2024-09-10T10:44:00.505353+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505353+0000","last_undegraded":"2024-09-10T10:44:00.505353+0000","last_fullsized":"2024-09-10T10:44:00.505353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438858+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:59.438858+0000","last_peered":"2024-09-10T10:44:59.438858+0000","last_clean":"2024-09-10T10:44:59.438858+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:59.438858+0000","last_undegraded":"2024-09-10T10:44:59.438858+0000","last_fullsized":"2024-09-10T10:44:59.438858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.709932+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:45:18.709932+0000","last_peered":"2024-09-10T10:45:18.709932+0000","last_clean":"2024-09-10T10:45:18.709932+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:45:18.709932+0000","last_undegraded":"2024-09-10T10:45:18.709932+0000","last_fullsized":"2024-09-10T10:45:18.709932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502631+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502631+0000","last_peered":"2024-09-10T10:44:00.502631+0000","last_clean":"2024-09-10T10:44:00.502631+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502631+0000","last_undegraded":"2024-09-10T10:44:00.502631+0000","last_fullsized":"2024-09-10T10:44:00.502631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439106+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:59.439106+0000","last_peered":"2024-09-10T10:44:59.439106+0000","last_clean":"2024-09-10T10:44:59.439106+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:59.439106+0000","last_undegraded":"2024-09-10T10:44:59.439106+0000","last_fullsized":"2024-09-10T10:44:59.439106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:17.707309+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:45:17.707309+0000","last_peered":"2024-09-10T10:45:17.707309+0000","last_clean":"2024-09-10T10:45:17.707309+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:45:17.707309+0000","last_undegraded":"2024-09-10T10:45:17.707309+0000","last_fullsized":"2024-09-10T10:45:17.707309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502777+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502777+0000","last_peered":"2024-09-10T10:44:00.502777+0000","last_clean":"2024-09-10T10:44:00.502777+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502777+0000","last_undegraded":"2024-09-10T10:44:00.502777+0000","last_fullsized":"2024-09-10T10:44:00.502777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444099+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:59.444099+0000","last_peered":"2024-09-10T10:44:59.444099+0000","last_clean":"2024-09-10T10:44:59.444099+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:59.444099+0000","last_undegraded":"2024-09-10T10:44:59.444099+0000","last_fullsized":"2024-09-10T10:44:59.444099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506093+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506093+0000","last_peered":"2024-09-10T10:44:00.506093+0000","last_clean":"2024-09-10T10:44:00.506093+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506093+0000","last_undegraded":"2024-09-10T10:44:00.506093+0000","last_fullsized":"2024-09-10T10:44:00.506093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516092+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.516092+0000","last_peered":"2024-09-10T10:44:00.516092+0000","last_clean":"2024-09-10T10:44:00.516092+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.516092+0000","last_undegraded":"2024-09-10T10:44:00.516092+0000","last_fullsized":"2024-09-10T10:44:00.516092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443062+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:59.443062+0000","last_peered":"2024-09-10T10:44:59.443062+0000","last_clean":"2024-09-10T10:44:59.443062+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:59.443062+0000","last_undegraded":"2024-09-10T10:44:59.443062+0000","last_fullsized":"2024-09-10T10:44:59.443062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503953+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503953+0000","last_peered":"2024-09-10T10:44:00.503953+0000","last_clean":"2024-09-10T10:44:00.503953+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503953+0000","last_undegraded":"2024-09-10T10:44:00.503953+0000","last_fullsized":"2024-09-10T10:44:00.503953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507101+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507101+0000","last_peered":"2024-09-10T10:44:00.507101+0000","last_clean":"2024-09-10T10:44:00.507101+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507101+0000","last_undegraded":"2024-09-10T10:44:00.507101+0000","last_fullsized":"2024-09-10T10:44:00.507101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442450+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:59.442450+0000","last_peered":"2024-09-10T10:44:59.442450+0000","last_clean":"2024-09-10T10:44:59.442450+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:59.442450+0000","last_undegraded":"2024-09-10T10:44:59.442450+0000","last_fullsized":"2024-09-10T10:44:59.442450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:19.763596+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:45:19.763596+0000","last_peered":"2024-09-10T10:45:19.763596+0000","last_clean":"2024-09-10T10:45:19.763596+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:45:19.763596+0000","last_undegraded":"2024-09-10T10:45:19.763596+0000","last_fullsized":"2024-09-10T10:45:19.763596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509662+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509662+0000","last_peered":"2024-09-10T10:44:00.509662+0000","last_clean":"2024-09-10T10:44:00.509662+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509662+0000","last_undegraded":"2024-09-10T10:44:00.509662+0000","last_fullsized":"2024-09-10T10:44:00.509662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442843+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:59.442843+0000","last_peered":"2024-09-10T10:44:59.442843+0000","last_clean":"2024-09-10T10:44:59.442843+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:59.442843+0000","last_undegraded":"2024-09-10T10:44:59.442843+0000","last_fullsized":"2024-09-10T10:44:59.442843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516384+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516384+0000","last_peered":"2024-09-10T10:44:00.516384+0000","last_clean":"2024-09-10T10:44:00.516384+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516384+0000","last_undegraded":"2024-09-10T10:44:00.516384+0000","last_fullsized":"2024-09-10T10:44:00.516384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500645+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500645+0000","last_peered":"2024-09-10T10:44:00.500645+0000","last_clean":"2024-09-10T10:44:00.500645+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500645+0000","last_undegraded":"2024-09-10T10:44:00.500645+0000","last_fullsized":"2024-09-10T10:44:00.500645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440420+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:59.440420+0000","last_peered":"2024-09-10T10:44:59.440420+0000","last_clean":"2024-09-10T10:44:59.440420+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:59.440420+0000","last_undegraded":"2024-09-10T10:44:59.440420+0000","last_fullsized":"2024-09-10T10:44:59.440420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503016+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.503016+0000","last_peered":"2024-09-10T10:44:00.503016+0000","last_clean":"2024-09-10T10:44:00.503016+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.503016+0000","last_undegraded":"2024-09-10T10:44:00.503016+0000","last_fullsized":"2024-09-10T10:44:00.503016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515085+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515085+0000","last_peered":"2024-09-10T10:44:00.515085+0000","last_clean":"2024-09-10T10:44:00.515085+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515085+0000","last_undegraded":"2024-09-10T10:44:00.515085+0000","last_fullsized":"2024-09-10T10:44:00.515085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440308+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:59.440308+0000","last_peered":"2024-09-10T10:44:59.440308+0000","last_clean":"2024-09-10T10:44:59.440308+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:59.440308+0000","last_undegraded":"2024-09-10T10:44:59.440308+0000","last_fullsized":"2024-09-10T10:44:59.440308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:05.823762+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:45:05.823762+0000","last_peered":"2024-09-10T10:45:05.823762+0000","last_clean":"2024-09-10T10:45:05.823762+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:45:05.823762+0000","last_undegraded":"2024-09-10T10:45:05.823762+0000","last_fullsized":"2024-09-10T10:45:05.823762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515302+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515302+0000","last_peered":"2024-09-10T10:44:00.515302+0000","last_clean":"2024-09-10T10:44:00.515302+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515302+0000","last_undegraded":"2024-09-10T10:44:00.515302+0000","last_fullsized":"2024-09-10T10:44:00.515302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442616+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442616+0000","last_peered":"2024-09-10T10:44:59.442616+0000","last_clean":"2024-09-10T10:44:59.442616+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:59.442616+0000","last_undegraded":"2024-09-10T10:44:59.442616+0000","last_fullsized":"2024-09-10T10:44:59.442616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505428+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505428+0000","last_peered":"2024-09-10T10:44:00.505428+0000","last_clean":"2024-09-10T10:44:00.505428+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505428+0000","last_undegraded":"2024-09-10T10:44:00.505428+0000","last_fullsized":"2024-09-10T10:44:00.505428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506639+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.506639+0000","last_peered":"2024-09-10T10:44:00.506639+0000","last_clean":"2024-09-10T10:44:00.506639+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.506639+0000","last_undegraded":"2024-09-10T10:44:00.506639+0000","last_fullsized":"2024-09-10T10:44:00.506639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441344+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:59.441344+0000","last_peered":"2024-09-10T10:44:59.441344+0000","last_clean":"2024-09-10T10:44:59.441344+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:59.441344+0000","last_undegraded":"2024-09-10T10:44:59.441344+0000","last_fullsized":"2024-09-10T10:44:59.441344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:16.706499+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:45:16.706499+0000","last_peered":"2024-09-10T10:45:16.706499+0000","last_clean":"2024-09-10T10:45:16.706499+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:45:16.706499+0000","last_undegraded":"2024-09-10T10:45:16.706499+0000","last_fullsized":"2024-09-10T10:45:16.706499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500337+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500337+0000","last_peered":"2024-09-10T10:44:00.500337+0000","last_clean":"2024-09-10T10:44:00.500337+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500337+0000","last_undegraded":"2024-09-10T10:44:00.500337+0000","last_fullsized":"2024-09-10T10:44:00.500337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445872+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:59.445872+0000","last_peered":"2024-09-10T10:44:59.445872+0000","last_clean":"2024-09-10T10:44:59.445872+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:59.445872+0000","last_undegraded":"2024-09-10T10:44:59.445872+0000","last_fullsized":"2024-09-10T10:44:59.445872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515497+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515497+0000","last_peered":"2024-09-10T10:44:00.515497+0000","last_clean":"2024-09-10T10:44:00.515497+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515497+0000","last_undegraded":"2024-09-10T10:44:00.515497+0000","last_fullsized":"2024-09-10T10:44:00.515497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502859+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502859+0000","last_peered":"2024-09-10T10:44:00.502859+0000","last_clean":"2024-09-10T10:44:00.502859+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502859+0000","last_undegraded":"2024-09-10T10:44:00.502859+0000","last_fullsized":"2024-09-10T10:44:00.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435638+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:59.435638+0000","last_peered":"2024-09-10T10:44:59.435638+0000","last_clean":"2024-09-10T10:44:59.435638+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:59.435638+0000","last_undegraded":"2024-09-10T10:44:59.435638+0000","last_fullsized":"2024-09-10T10:44:59.435638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516076+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516076+0000","last_peered":"2024-09-10T10:44:00.516076+0000","last_clean":"2024-09-10T10:44:00.516076+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516076+0000","last_undegraded":"2024-09-10T10:44:00.516076+0000","last_fullsized":"2024-09-10T10:44:00.516076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":311,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.623041+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:45:15.623041+0000","last_peered":"2024-09-10T10:45:15.623041+0000","last_clean":"2024-09-10T10:45:15.623041+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:45:15.623041+0000","last_undegraded":"2024-09-10T10:45:15.623041+0000","last_fullsized":"2024-09-10T10:45:15.623041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62820,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":167,"num_write":41,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500085+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500085+0000","last_peered":"2024-09-10T10:44:00.500085+0000","last_clean":"2024-09-10T10:44:00.500085+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500085+0000","last_undegraded":"2024-09-10T10:44:00.500085+0000","last_fullsized":"2024-09-10T10:44:00.500085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516289+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516289+0000","last_peered":"2024-09-10T10:44:00.516289+0000","last_clean":"2024-09-10T10:44:00.516289+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516289+0000","last_undegraded":"2024-09-10T10:44:00.516289+0000","last_fullsized":"2024-09-10T10:44:00.516289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503778+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503778+0000","last_peered":"2024-09-10T10:44:00.503778+0000","last_clean":"2024-09-10T10:44:00.503778+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503778+0000","last_undegraded":"2024-09-10T10:44:00.503778+0000","last_fullsized":"2024-09-10T10:44:00.503778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506656+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506656+0000","last_peered":"2024-09-10T10:44:00.506656+0000","last_clean":"2024-09-10T10:44:00.506656+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506656+0000","last_undegraded":"2024-09-10T10:44:00.506656+0000","last_fullsized":"2024-09-10T10:44:00.506656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499856+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499856+0000","last_peered":"2024-09-10T10:44:00.499856+0000","last_clean":"2024-09-10T10:44:00.499856+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499856+0000","last_undegraded":"2024-09-10T10:44:00.499856+0000","last_fullsized":"2024-09-10T10:44:00.499856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442320+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:59.442320+0000","last_peered":"2024-09-10T10:44:59.442320+0000","last_clean":"2024-09-10T10:44:59.442320+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:59.442320+0000","last_undegraded":"2024-09-10T10:44:59.442320+0000","last_fullsized":"2024-09-10T10:44:59.442320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.756025+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:45:15.756025+0000","last_peered":"2024-09-10T10:45:15.756025+0000","last_clean":"2024-09-10T10:45:15.756025+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:45:15.756025+0000","last_undegraded":"2024-09-10T10:45:15.756025+0000","last_fullsized":"2024-09-10T10:45:15.756025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":208,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504723+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504723+0000","last_peered":"2024-09-10T10:44:00.504723+0000","last_clean":"2024-09-10T10:44:00.504723+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504723+0000","last_undegraded":"2024-09-10T10:44:00.504723+0000","last_fullsized":"2024-09-10T10:44:00.504723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500929+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500929+0000","last_peered":"2024-09-10T10:44:00.500929+0000","last_clean":"2024-09-10T10:44:00.500929+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500929+0000","last_undegraded":"2024-09-10T10:44:00.500929+0000","last_fullsized":"2024-09-10T10:44:00.500929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432970+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:59.432970+0000","last_peered":"2024-09-10T10:44:59.432970+0000","last_clean":"2024-09-10T10:44:59.432970+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:59.432970+0000","last_undegraded":"2024-09-10T10:44:59.432970+0000","last_fullsized":"2024-09-10T10:44:59.432970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515598+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515598+0000","last_peered":"2024-09-10T10:44:00.515598+0000","last_clean":"2024-09-10T10:44:00.515598+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515598+0000","last_undegraded":"2024-09-10T10:44:00.515598+0000","last_fullsized":"2024-09-10T10:44:00.515598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509926+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509926+0000","last_peered":"2024-09-10T10:44:00.509926+0000","last_clean":"2024-09-10T10:44:00.509926+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509926+0000","last_undegraded":"2024-09-10T10:44:00.509926+0000","last_fullsized":"2024-09-10T10:44:00.509926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.434606+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:59.434606+0000","last_peered":"2024-09-10T10:44:59.434606+0000","last_clean":"2024-09-10T10:44:59.434606+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:59.434606+0000","last_undegraded":"2024-09-10T10:44:59.434606+0000","last_fullsized":"2024-09-10T10:44:59.434606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502516+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502516+0000","last_peered":"2024-09-10T10:44:00.502516+0000","last_clean":"2024-09-10T10:44:00.502516+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502516+0000","last_undegraded":"2024-09-10T10:44:00.502516+0000","last_fullsized":"2024-09-10T10:44:00.502516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505325+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505325+0000","last_peered":"2024-09-10T10:44:00.505325+0000","last_clean":"2024-09-10T10:44:00.505325+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505325+0000","last_undegraded":"2024-09-10T10:44:00.505325+0000","last_fullsized":"2024-09-10T10:44:00.505325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438031+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:59.438031+0000","last_peered":"2024-09-10T10:44:59.438031+0000","last_clean":"2024-09-10T10:44:59.438031+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:59.438031+0000","last_undegraded":"2024-09-10T10:44:59.438031+0000","last_fullsized":"2024-09-10T10:44:59.438031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.707305+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:45:15.707305+0000","last_peered":"2024-09-10T10:45:15.707305+0000","last_clean":"2024-09-10T10:45:15.707305+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:45:15.707305+0000","last_undegraded":"2024-09-10T10:45:15.707305+0000","last_fullsized":"2024-09-10T10:45:15.707305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439297+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:59.439297+0000","last_peered":"2024-09-10T10:44:59.439297+0000","last_clean":"2024-09-10T10:44:59.439297+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:59.439297+0000","last_undegraded":"2024-09-10T10:44:59.439297+0000","last_fullsized":"2024-09-10T10:44:59.439297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440974+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:59.440974+0000","last_peered":"2024-09-10T10:44:59.440974+0000","last_clean":"2024-09-10T10:44:59.440974+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:59.440974+0000","last_undegraded":"2024-09-10T10:44:59.440974+0000","last_fullsized":"2024-09-10T10:44:59.440974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441221+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:59.441221+0000","last_peered":"2024-09-10T10:44:59.441221+0000","last_clean":"2024-09-10T10:44:59.441221+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:59.441221+0000","last_undegraded":"2024-09-10T10:44:59.441221+0000","last_fullsized":"2024-09-10T10:44:59.441221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440703+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:59.440703+0000","last_peered":"2024-09-10T10:44:59.440703+0000","last_clean":"2024-09-10T10:44:59.440703+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:59.440703+0000","last_undegraded":"2024-09-10T10:44:59.440703+0000","last_fullsized":"2024-09-10T10:44:59.440703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444329+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:59.444329+0000","last_peered":"2024-09-10T10:44:59.444329+0000","last_clean":"2024-09-10T10:44:59.444329+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:59.444329+0000","last_undegraded":"2024-09-10T10:44:59.444329+0000","last_fullsized":"2024-09-10T10:44:59.444329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438450+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:59.438450+0000","last_peered":"2024-09-10T10:44:59.438450+0000","last_clean":"2024-09-10T10:44:59.438450+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:59.438450+0000","last_undegraded":"2024-09-10T10:44:59.438450+0000","last_fullsized":"2024-09-10T10:44:59.438450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441275+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:59.441275+0000","last_peered":"2024-09-10T10:44:59.441275+0000","last_clean":"2024-09-10T10:44:59.441275+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:59.441275+0000","last_undegraded":"2024-09-10T10:44:59.441275+0000","last_fullsized":"2024-09-10T10:44:59.441275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437164+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:59.437164+0000","last_peered":"2024-09-10T10:44:59.437164+0000","last_clean":"2024-09-10T10:44:59.437164+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:59.437164+0000","last_undegraded":"2024-09-10T10:44:59.437164+0000","last_fullsized":"2024-09-10T10:44:59.437164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441514+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:59.441514+0000","last_peered":"2024-09-10T10:44:59.441514+0000","last_clean":"2024-09-10T10:44:59.441514+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:59.441514+0000","last_undegraded":"2024-09-10T10:44:59.441514+0000","last_fullsized":"2024-09-10T10:44:59.441514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442360+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:59.442360+0000","last_peered":"2024-09-10T10:44:59.442360+0000","last_clean":"2024-09-10T10:44:59.442360+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:59.442360+0000","last_undegraded":"2024-09-10T10:44:59.442360+0000","last_fullsized":"2024-09-10T10:44:59.442360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442124+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:59.442124+0000","last_peered":"2024-09-10T10:44:59.442124+0000","last_clean":"2024-09-10T10:44:59.442124+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:59.442124+0000","last_undegraded":"2024-09-10T10:44:59.442124+0000","last_fullsized":"2024-09-10T10:44:59.442124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442058+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:59.442058+0000","last_peered":"2024-09-10T10:44:59.442058+0000","last_clean":"2024-09-10T10:44:59.442058+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:59.442058+0000","last_undegraded":"2024-09-10T10:44:59.442058+0000","last_fullsized":"2024-09-10T10:44:59.442058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438851+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:59.438851+0000","last_peered":"2024-09-10T10:44:59.438851+0000","last_clean":"2024-09-10T10:44:59.438851+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:59.438851+0000","last_undegraded":"2024-09-10T10:44:59.438851+0000","last_fullsized":"2024-09-10T10:44:59.438851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429557+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:59.429557+0000","last_peered":"2024-09-10T10:44:59.429557+0000","last_clean":"2024-09-10T10:44:59.429557+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:59.429557+0000","last_undegraded":"2024-09-10T10:44:59.429557+0000","last_fullsized":"2024-09-10T10:44:59.429557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440551+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:59.440551+0000","last_peered":"2024-09-10T10:44:59.440551+0000","last_clean":"2024-09-10T10:44:59.440551+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:59.440551+0000","last_undegraded":"2024-09-10T10:44:59.440551+0000","last_fullsized":"2024-09-10T10:44:59.440551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437183+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:59.437183+0000","last_peered":"2024-09-10T10:44:59.437183+0000","last_clean":"2024-09-10T10:44:59.437183+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:59.437183+0000","last_undegraded":"2024-09-10T10:44:59.437183+0000","last_fullsized":"2024-09-10T10:44:59.437183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439780+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:59.439780+0000","last_peered":"2024-09-10T10:44:59.439780+0000","last_clean":"2024-09-10T10:44:59.439780+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:59.439780+0000","last_undegraded":"2024-09-10T10:44:59.439780+0000","last_fullsized":"2024-09-10T10:44:59.439780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441856+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:59.441856+0000","last_peered":"2024-09-10T10:44:59.441856+0000","last_clean":"2024-09-10T10:44:59.441856+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:59.441856+0000","last_undegraded":"2024-09-10T10:44:59.441856+0000","last_fullsized":"2024-09-10T10:44:59.441856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437328+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:59.437328+0000","last_peered":"2024-09-10T10:44:59.437328+0000","last_clean":"2024-09-10T10:44:59.437328+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:59.437328+0000","last_undegraded":"2024-09-10T10:44:59.437328+0000","last_fullsized":"2024-09-10T10:44:59.437328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438618+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:59.438618+0000","last_peered":"2024-09-10T10:44:59.438618+0000","last_clean":"2024-09-10T10:44:59.438618+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:59.438618+0000","last_undegraded":"2024-09-10T10:44:59.438618+0000","last_fullsized":"2024-09-10T10:44:59.438618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444056+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:59.444056+0000","last_peered":"2024-09-10T10:44:59.444056+0000","last_clean":"2024-09-10T10:44:59.444056+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:59.444056+0000","last_undegraded":"2024-09-10T10:44:59.444056+0000","last_fullsized":"2024-09-10T10:44:59.444056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442679+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:59.442679+0000","last_peered":"2024-09-10T10:44:59.442679+0000","last_clean":"2024-09-10T10:44:59.442679+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:59.442679+0000","last_undegraded":"2024-09-10T10:44:59.442679+0000","last_fullsized":"2024-09-10T10:44:59.442679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432695+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:59.432695+0000","last_peered":"2024-09-10T10:44:59.432695+0000","last_clean":"2024-09-10T10:44:59.432695+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:59.432695+0000","last_undegraded":"2024-09-10T10:44:59.432695+0000","last_fullsized":"2024-09-10T10:44:59.432695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442732+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442732+0000","last_peered":"2024-09-10T10:44:59.442732+0000","last_clean":"2024-09-10T10:44:59.442732+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:59.442732+0000","last_undegraded":"2024-09-10T10:44:59.442732+0000","last_fullsized":"2024-09-10T10:44:59.442732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440814+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:59.440814+0000","last_peered":"2024-09-10T10:44:59.440814+0000","last_clean":"2024-09-10T10:44:59.440814+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:59.440814+0000","last_undegraded":"2024-09-10T10:44:59.440814+0000","last_fullsized":"2024-09-10T10:44:59.440814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444968+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:59.444968+0000","last_peered":"2024-09-10T10:44:59.444968+0000","last_clean":"2024-09-10T10:44:59.444968+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:59.444968+0000","last_undegraded":"2024-09-10T10:44:59.444968+0000","last_fullsized":"2024-09-10T10:44:59.444968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428461+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:59.428461+0000","last_peered":"2024-09-10T10:44:59.428461+0000","last_clean":"2024-09-10T10:44:59.428461+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:59.428461+0000","last_undegraded":"2024-09-10T10:44:59.428461+0000","last_fullsized":"2024-09-10T10:44:59.428461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428915+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:59.428915+0000","last_peered":"2024-09-10T10:44:59.428915+0000","last_clean":"2024-09-10T10:44:59.428915+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:59.428915+0000","last_undegraded":"2024-09-10T10:44:59.428915+0000","last_fullsized":"2024-09-10T10:44:59.428915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441024+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:59.441024+0000","last_peered":"2024-09-10T10:44:59.441024+0000","last_clean":"2024-09-10T10:44:59.441024+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:59.441024+0000","last_undegraded":"2024-09-10T10:44:59.441024+0000","last_fullsized":"2024-09-10T10:44:59.441024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441764+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:59.441764+0000","last_peered":"2024-09-10T10:44:59.441764+0000","last_clean":"2024-09-10T10:44:59.441764+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:59.441764+0000","last_undegraded":"2024-09-10T10:44:59.441764+0000","last_fullsized":"2024-09-10T10:44:59.441764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439409+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:59.439409+0000","last_peered":"2024-09-10T10:44:59.439409+0000","last_clean":"2024-09-10T10:44:59.439409+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:59.439409+0000","last_undegraded":"2024-09-10T10:44:59.439409+0000","last_fullsized":"2024-09-10T10:44:59.439409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439118+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:59.439118+0000","last_peered":"2024-09-10T10:44:59.439118+0000","last_clean":"2024-09-10T10:44:59.439118+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:59.439118+0000","last_undegraded":"2024-09-10T10:44:59.439118+0000","last_fullsized":"2024-09-10T10:44:59.439118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72683,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1093,"num_read_kb":623,"num_write":276,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":229983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547241,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":501134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1154798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939701,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168635,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1098863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":496359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":563394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1024000,"data_stored":498177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":567436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:22.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:22.433 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:22.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:22 smithi143 ceph-mon[31057]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:22.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:22 smithi143 ceph-mon[31057]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:45:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:22 smithi178 ceph-mon[34813]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:22 smithi178 ceph-mon[34813]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:45:22.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:22 smithi144 ceph-mon[34230]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:22 smithi144 ceph-mon[34230]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 -- 172.21.15.143:0/957772950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60106f00 msgr2=0x7f4a6010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 --2- 172.21.15.143:0/957772950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60106f00 0x7f4a6010ea80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009900 tx=0x7f4a5802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 -- 172.21.15.143:0/957772950 shutdown_connections 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 --2- 172.21.15.143:0/957772950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60106f00 0x7f4a6010ea80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 --2- 172.21.15.143:0/957772950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a60102780 0x7f4a601067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 --2- 172.21.15.143:0/957772950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60101dd0 0x7f4a601021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 -- 172.21.15.143:0/957772950 >> 172.21.15.143:0/957772950 conn(0x7f4a600fd890 msgr2=0x7f4a600ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 -- 172.21.15.143:0/957772950 shutdown_connections 2024-09-10T10:45:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 -- 172.21.15.143:0/957772950 wait complete. 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.939+0000 7f4a654e6640 1 Processor -- start 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 -- start start 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 0x7f4a60198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a60102780 0x7f4a60198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60106f00 0x7f4a6019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111e10 con 0x7f4a60101dd0 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111c90 con 0x7f4a60106f00 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a654e6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111f90 con 0x7f4a60102780 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 0x7f4a60198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 0x7f4a60198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42466/0 (socket says 172.21.15.143:42466) 2024-09-10T10:45:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 -- 172.21.15.143:0/4167848115 learned_addr learned my addr 172.21.15.143:0/4167848115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5f7fe640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60106f00 0x7f4a6019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 -- 172.21.15.143:0/4167848115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a60102780 msgr2=0x7f4a60198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a60102780 0x7f4a60198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 -- 172.21.15.143:0/4167848115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60106f00 msgr2=0x7f4a6019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60106f00 0x7f4a6019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5effd640 1 -- 172.21.15.143:0/4167848115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6019d2e0 con 0x7f4a60101dd0 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.940+0000 7f4a5f7fe640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60106f00 0x7f4a6019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.941+0000 7f4a5effd640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 0x7f4a60198330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000db80 tx=0x7f4a5000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.941+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50017070 con 0x7f4a60101dd0 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.941+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6019d5d0 con 0x7f4a60101dd0 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.941+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a50003b70 con 0x7f4a60101dd0 2024-09-10T10:45:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.941+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50010480 con 0x7f4a60101dd0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.941+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601a4eb0 con 0x7f4a60101dd0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.942+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a601021b0 con 0x7f4a60101dd0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.943+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f4a50004020 con 0x7f4a60101dd0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.943+0000 7f4a57fff640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4a4c0a2c10 0x7f4a4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.944+0000 7f4a5e7fc640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4a4c0a2c10 0x7f4a4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.944+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4a500cf6a0 con 0x7f4a60101dd0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.944+0000 7f4a5e7fc640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4a4c0a2c10 0x7f4a4c0a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a600ffea0 tx=0x7f4a48009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:22.948+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a500212a0 con 0x7f4a60101dd0 2024-09-10T10:45:23.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.146+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a60199690 con 0x7f4a60101dd0 2024-09-10T10:45:23.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.147+0000 7f4a57fff640 1 -- 172.21.15.143:0/4167848115 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f4a5009c140 con 0x7f4a60101dd0 2024-09-10T10:45:23.148 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:23.148 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:45:23.148 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:45:23.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4a4c0a2c10 msgr2=0x7f4a4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:23.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4a4c0a2c10 0x7f4a4c0a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a600ffea0 tx=0x7f4a48009590 comp rx=0 tx=0).stop 2024-09-10T10:45:23.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 msgr2=0x7f4a60198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:23.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 0x7f4a60198330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000db80 tx=0x7f4a5000b630 comp rx=0 tx=0).stop 2024-09-10T10:45:23.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 shutdown_connections 2024-09-10T10:45:23.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4a4c0a2c10 0x7f4a4c0a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:23.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60106f00 0x7f4a6019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:23.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a60102780 0x7f4a60198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:23.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 --2- 172.21.15.143:0/4167848115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a60101dd0 0x7f4a60198330 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:23.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 >> 172.21.15.143:0/4167848115 conn(0x7f4a600fd890 msgr2=0x7f4a600fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:23.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 shutdown_connections 2024-09-10T10:45:23.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:23.149+0000 7f4a654e6640 1 -- 172.21.15.143:0/4167848115 wait complete. 2024-09-10T10:45:23.865 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:23 smithi143 ceph-mon[31057]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:23.865 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:23 smithi143 ceph-mon[31057]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:23.865 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:23 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4167848115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:23.892 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:45:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:23 smithi178 ceph-mon[34813]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:23 smithi178 ceph-mon[34813]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:23.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:23 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4167848115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:23.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:23 smithi144 ceph-mon[34230]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:23.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:23 smithi144 ceph-mon[34230]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:23.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:23 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4167848115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:24.139 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:24.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 -- 172.21.15.143:0/2950249301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b64106f20 msgr2=0x7f6b6410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 --2- 172.21.15.143:0/2950249301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b64106f20 0x7f6b6410eaa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009ca0 tx=0x7f6b5402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 -- 172.21.15.143:0/2950249301 shutdown_connections 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 --2- 172.21.15.143:0/2950249301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b64106f20 0x7f6b6410eaa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 --2- 172.21.15.143:0/2950249301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b641027a0 0x7f6b641067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 --2- 172.21.15.143:0/2950249301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b64101df0 0x7f6b641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 -- 172.21.15.143:0/2950249301 >> 172.21.15.143:0/2950249301 conn(0x7f6b640fd8f0 msgr2=0x7f6b640ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 -- 172.21.15.143:0/2950249301 shutdown_connections 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.615+0000 7f6b695d9640 1 -- 172.21.15.143:0/2950249301 wait complete. 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 Processor -- start 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 -- start start 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b64101df0 0x7f6b64198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 0x7f6b64198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b64106f20 0x7f6b6419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64111e30 con 0x7f6b641027a0 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64111cb0 con 0x7f6b64106f20 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.616+0000 7f6b695d9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64111fb0 con 0x7f6b64101df0 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 0x7f6b64198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 0x7f6b64198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42484/0 (socket says 172.21.15.143:42484) 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 -- 172.21.15.143:0/748482471 learned_addr learned my addr 172.21.15.143:0/748482471 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b62ffd640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b64101df0 0x7f6b64198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b637fe640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b64106f20 0x7f6b6419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 -- 172.21.15.143:0/748482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b64101df0 msgr2=0x7f6b64198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b64101df0 0x7f6b64198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 -- 172.21.15.143:0/748482471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b64106f20 msgr2=0x7f6b6419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b64106f20 0x7f6b6419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 -- 172.21.15.143:0/748482471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6419d300 con 0x7f6b641027a0 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b627fc640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 0x7f6b64198890 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800e9f0 tx=0x7f6b5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800cd80 con 0x7f6b641027a0 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6419d5f0 con 0x7f6b641027a0 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b580054d0 con 0x7f6b641027a0 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58010490 con 0x7f6b641027a0 2024-09-10T10:45:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.617+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b641a4ed0 con 0x7f6b641027a0 2024-09-10T10:45:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.618+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b641021d0 con 0x7f6b641027a0 2024-09-10T10:45:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.619+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6b58005020 con 0x7f6b641027a0 2024-09-10T10:45:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.620+0000 7f6b4bfff640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b4c0a2c10 0x7f6b4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.620+0000 7f6b62ffd640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b4c0a2c10 0x7f6b4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.620+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6b580cf8d0 con 0x7f6b641027a0 2024-09-10T10:45:24.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.620+0000 7f6b62ffd640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b4c0a2c10 0x7f6b4c0a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009bd0 tx=0x7f6b5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.624+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b5809c370 con 0x7f6b641027a0 2024-09-10T10:45:24.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.791+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b6410a100 con 0x7f6b641027a0 2024-09-10T10:45:24.793 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:24 smithi143 ceph-mon[31057]: pgmap v361: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T10:45:24.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.793+0000 7f6b4bfff640 1 -- 172.21.15.143:0/748482471 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7f6b5809bc70 con 0x7f6b641027a0 2024-09-10T10:45:24.795 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:24.795 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.795+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b4c0a2c10 msgr2=0x7f6b4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.795+0000 7f6b695d9640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b4c0a2c10 0x7f6b4c0a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009bd0 tx=0x7f6b5c005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 msgr2=0x7f6b64198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 0x7f6b64198890 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800e9f0 tx=0x7f6b5800edf0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 shutdown_connections 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b4c0a2c10 0x7f6b4c0a50d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b64106f20 0x7f6b6419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b641027a0 0x7f6b64198890 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 --2- 172.21.15.143:0/748482471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b64101df0 0x7f6b64198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 >> 172.21.15.143:0/748482471 conn(0x7f6b640fd8f0 msgr2=0x7f6b640fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 shutdown_connections 2024-09-10T10:45:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:24.796+0000 7f6b695d9640 1 -- 172.21.15.143:0/748482471 wait complete. 2024-09-10T10:45:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:24 smithi178 ceph-mon[34813]: pgmap v361: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T10:45:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:24 smithi144 ceph-mon[34230]: pgmap v361: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T10:45:25.503 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T10:45:25.503 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T10:45:25.519 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:45:25.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:45:25.520 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T10:45:25.520 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:45:25.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:45:25.783 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:25.784 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:25.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:25 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/748482471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:25.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:25 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/748482471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:25.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:25 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/748482471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 -- 172.21.15.143:0/350668483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144101de0 msgr2=0x7fc1441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 --2- 172.21.15.143:0/350668483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144101de0 0x7fc1441021c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc134009900 tx=0x7fc13402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 -- 172.21.15.143:0/350668483 shutdown_connections 2024-09-10T10:45:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 --2- 172.21.15.143:0/350668483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc144106f10 0x7fc14410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 --2- 172.21.15.143:0/350668483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc144102790 0x7fc1441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 --2- 172.21.15.143:0/350668483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144101de0 0x7fc1441021c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 -- 172.21.15.143:0/350668483 >> 172.21.15.143:0/350668483 conn(0x7fc1440fd8c0 msgr2=0x7fc1440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 -- 172.21.15.143:0/350668483 shutdown_connections 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.319+0000 7fc149aa0640 1 -- 172.21.15.143:0/350668483 wait complete. 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 Processor -- start 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 -- start start 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc144101de0 0x7fc1441983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 0x7fc144198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc144106f10 0x7fc14419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc144111e40 con 0x7fc144102790 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc144111cc0 con 0x7fc144106f10 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc149aa0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc144111fc0 con 0x7fc144101de0 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.320+0000 7fc142ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 0x7fc144198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 0x7fc144198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42504/0 (socket says 172.21.15.143:42504) 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 -- 172.21.15.143:0/3274494476 learned_addr learned my addr 172.21.15.143:0/3274494476 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc1437fe640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc144101de0 0x7fc1441983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc143fff640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc144106f10 0x7fc14419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 -- 172.21.15.143:0/3274494476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc144101de0 msgr2=0x7fc1441983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc144101de0 0x7fc1441983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 -- 172.21.15.143:0/3274494476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc144106f10 msgr2=0x7fc14419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc144106f10 0x7fc14419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 -- 172.21.15.143:0/3274494476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14419d370 con 0x7fc144102790 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc142ffd640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 0x7fc144198900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc13800b4a0 tx=0x7fc13800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc140ff9640 1 -- 172.21.15.143:0/3274494476 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138014070 con 0x7fc144102790 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14419d660 con 0x7fc144102790 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc140ff9640 1 -- 172.21.15.143:0/3274494476 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc138010ac0 con 0x7fc144102790 2024-09-10T10:45:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc140ff9640 1 -- 172.21.15.143:0/3274494476 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138017c30 con 0x7fc144102790 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.321+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1441a4f40 con 0x7fc144102790 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.323+0000 7fc140ff9640 1 -- 172.21.15.143:0/3274494476 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc138017070 con 0x7fc144102790 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 Processor -- start 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 -- start start 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc140ff9640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc12c0a2c10 0x7fc12c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200dead360 0x56200debbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 0x56200debe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56200de12280 con 0x56200debc150 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56200ddb9b10 con 0x56200deaca40 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc14be79740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56200de0e240 con 0x56200dead360 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc140ff9640 1 -- 172.21.15.143:0/3274494476 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1380cea50 con 0x7fc144102790 2024-09-10T10:45:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc143fff640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc12c0a2c10 0x7fc12c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc143fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 0x56200debe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc1437fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc143fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 0x56200debe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42506/0 (socket says 172.21.15.143:42506) 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.324+0000 7fc143fff640 1 -- 172.21.15.143:0/3729571929 learned_addr learned my addr 172.21.15.143:0/3729571929 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc142ffd640 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200dead360 0x56200debbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 -- 172.21.15.143:0/3729571929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200dead360 msgr2=0x56200debbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200dead360 0x56200debbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 -- 172.21.15.143:0/3729571929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 msgr2=0x56200deace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 -- 172.21.15.143:0/3729571929 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200ddc61f0 con 0x56200debc150 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc1437fe640 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc12c0a2c10 0x7fc12c0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc13c012c80 tx=0x7fc13c009e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc143fff640 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 0x56200debe540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc13c010bd0 tx=0x7fc13c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996bc518ff052d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc129ffb640 1 -- 172.21.15.143:0/3729571929 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c024070 con 0x56200debc150 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc129ffb640 1 -- 172.21.15.143:0/3729571929 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc13c00ebf0 con 0x56200debc150 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.325+0000 7fc129ffb640 1 -- 172.21.15.143:0/3729571929 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c005580 con 0x56200debc150 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 -- 172.21.15.143:0/3729571929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 msgr2=0x56200debe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 0x56200debe540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc13c010bd0 tx=0x7fc13c01c000 comp rx=0 tx=0).stop 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 -- 172.21.15.143:0/3729571929 shutdown_connections 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200debc150 0x56200debe540 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200dead360 0x56200debbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 --2- 172.21.15.143:0/3729571929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 -- 172.21.15.143:0/3729571929 >> 172.21.15.143:0/3729571929 conn(0x56200dea90f0 msgr2=0x56200dea94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 -- 172.21.15.143:0/3729571929 shutdown_connections 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 -- 172.21.15.143:0/3729571929 wait complete. 2024-09-10T10:45:26.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 -- start start 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.326+0000 7fc14be79740 1 Processor -- start 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deb90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200deb9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56200debed20 con 0x56200dead360 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56200ddccf10 con 0x56200deaca40 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56200debeea0 con 0x56200debc150 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 10 client.0 _ll_register_callbacks cb 0x56200dd95a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200deb9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200deb9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42516/0 (socket says 172.21.15.143:42516) 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 -- 172.21.15.143:0/84128315 learned_addr learned my addr 172.21.15.143:0/84128315 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc143fff640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc1437fe640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deb90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 -- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 msgr2=0x56200deb9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 -- 172.21.15.143:0/84128315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 msgr2=0x56200deb90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deb90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 -- 172.21.15.143:0/84128315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200db836a0 con 0x56200dead360 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc142ffd640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200deb9610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc138009420 tx=0x7fc13809afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4eb269800e4ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc122ffd640 1 -- 172.21.15.143:0/84128315 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13801e070 con 0x56200dead360 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc122ffd640 1 -- 172.21.15.143:0/84128315 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1380cc2f0 con 0x56200dead360 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13c00ebf0 con 0x56200dead360 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc122ffd640 1 -- 172.21.15.143:0/84128315 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1380a3b70 con 0x56200dead360 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.327+0000 7fc14be79740 10 client.15387 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56200decaa60 con 0x56200dead360 2024-09-10T10:45:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc122ffd640 1 -- 172.21.15.143:0/84128315 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc13c00ebf0 con 0x56200dead360 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc122ffd640 1 -- 172.21.15.143:0/84128315 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc138084120 con 0x56200dead360 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 10 client.15387 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 20 client.15387 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200decbe70 0x56200dece2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 4 client.15387 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56200de1f7f0 con 0x56200decbe70 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.328+0000 7fc1437fe640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200decbe70 0x56200dece2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.329+0000 7fc1437fe640 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200decbe70 0x56200dece2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b18c567e22e9bb1 server_cookie=debf4a7046668afe in_seq=0 out_seq=0 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.329+0000 7fc122ffd640 10 client.15387 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.329+0000 7fc122ffd640 1 -- 172.21.15.143:0/84128315 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56200de1f7f0 con 0x56200decbe70 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.329+0000 7fc122ffd640 10 client.15387 handle_command_reply: tid=0 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.329+0000 7fc14be79740 1 client.15387 shutdown 2024-09-10T10:45:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 msgr2=0x56200deb9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200deb9610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc138009420 tx=0x7fc13809afb0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 shutdown_connections 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200decbe70 0x56200dece2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200deb9610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 --2- 172.21.15.143:0/84128315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200deb90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc122ffd640 0 client.15387 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 >> 172.21.15.143:0/84128315 conn(0x56200dea90f0 msgr2=0x56200deaee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 shutdown_connections 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 1 -- 172.21.15.143:0/84128315 wait complete. 2024-09-10T10:45:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.330+0000 7fc14be79740 20 client.15387 trim_cache size 0 max 16384 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 Processor -- start 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 -- start start 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200ded3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200ded40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ac870 con 0x56200dead360 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ae3b0 con 0x56200deaca40 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc14be79740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441acf40 con 0x56200debc150 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200ded40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200ded40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42530/0 (socket says 172.21.15.143:42530) 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 -- 172.21.15.143:0/4091291973 learned_addr learned my addr 172.21.15.143:0/4091291973 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:26.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 -- 172.21.15.143:0/4091291973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 msgr2=0x56200deb6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 -- 172.21.15.143:0/4091291973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 msgr2=0x56200ded3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc1437fe640 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200ded3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200ded3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc142ffd640 1 -- 172.21.15.143:0/4091291973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56200deaf5b0 con 0x56200dead360 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.347+0000 7fc1437fe640 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200ded3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc142ffd640 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200ded40b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc1380973c0 tx=0x7fc13800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec001764477a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc1297fa640 1 -- 172.21.15.143:0/4091291973 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13801e070 con 0x56200dead360 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc1297fa640 1 -- 172.21.15.143:0/4091291973 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc13809f2b0 con 0x56200dead360 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc1297fa640 1 -- 172.21.15.143:0/4091291973 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380a37b0 con 0x56200dead360 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 -- 172.21.15.143:0/4091291973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 msgr2=0x56200ded40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200ded40b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc1380973c0 tx=0x7fc13800bf90 comp rx=0 tx=0).stop 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 -- 172.21.15.143:0/4091291973 shutdown_connections 2024-09-10T10:45:26.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200deb6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200dead360 0x56200ded40b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 --2- 172.21.15.143:0/4091291973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200deaca40 0x56200ded3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.348+0000 7fc14be79740 1 -- 172.21.15.143:0/4091291973 >> 172.21.15.143:0/4091291973 conn(0x56200dea90f0 msgr2=0x56200debb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 -- 172.21.15.143:0/4091291973 shutdown_connections 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 -- 172.21.15.143:0/4091291973 wait complete. 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 -- start start 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 Processor -- start 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 0x56200dec4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200dead360 0x56200dec4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200dec1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441adce0 con 0x56200deaca40 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ad610 con 0x56200dead360 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441aa850 con 0x56200debc150 2024-09-10T10:45:26.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc14be79740 10 client.0 _ll_register_callbacks cb 0x7fc1380092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc1437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 0x56200dec4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc1437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 0x56200dec4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42534/0 (socket says 172.21.15.143:42534) 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.349+0000 7fc1437fe640 1 -- 172.21.15.143:0/2975025755 learned_addr learned my addr 172.21.15.143:0/2975025755 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc142ffd640 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200dead360 0x56200dec4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc1437fe640 1 -- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 msgr2=0x56200dec1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc1437fe640 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200dec1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc1437fe640 1 -- 172.21.15.143:0/2975025755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200dead360 msgr2=0x56200dec4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc1437fe640 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200dead360 0x56200dec4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc1437fe640 1 -- 172.21.15.143:0/2975025755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56200dec19c0 con 0x56200deaca40 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc1437fe640 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 0x56200dec4410 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc134009900 tx=0x7fc134004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda24aa85d209dea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc122ffd640 1 -- 172.21.15.143:0/2975025755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134035040 con 0x56200deaca40 2024-09-10T10:45:26.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13809f2b0 con 0x56200deaca40 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc122ffd640 1 -- 172.21.15.143:0/2975025755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc134005450 con 0x56200deaca40 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc122ffd640 1 -- 172.21.15.143:0/2975025755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc134052050 con 0x56200deaca40 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc14be79740 10 client.15393 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56200df37660 con 0x56200deaca40 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.350+0000 7fc122ffd640 1 -- 172.21.15.143:0/2975025755 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc13809f2b0 con 0x56200deaca40 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc122ffd640 1 -- 172.21.15.143:0/2975025755 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc13404d030 con 0x56200deaca40 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc14be79740 10 client.15393 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc14be79740 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc14be79740 20 client.15393 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc14be79740 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200df37d80 0x56200df3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc14be79740 4 client.15393 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56200de19d00 con 0x56200df37d80 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc142ffd640 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200df37d80 0x56200df3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:26.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc122ffd640 10 client.15393 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.351+0000 7fc142ffd640 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200df37d80 0x56200df3a1e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e425dfe954fa3e server_cookie=c887724646a4386a in_seq=0 out_seq=0 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.352+0000 7fc122ffd640 1 -- 172.21.15.143:0/2975025755 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56200de19d00 con 0x56200df37d80 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.352+0000 7fc122ffd640 10 client.15393 handle_command_reply: tid=0 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.352+0000 7fc14be79740 1 client.15393 shutdown 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.352+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 msgr2=0x56200dec4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.352+0000 7fc14be79740 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 0x56200dec4410 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc134009900 tx=0x7fc134004270 comp rx=0 tx=0).stop 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 shutdown_connections 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56200df37d80 0x56200df3a1e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56200debc150 0x56200dec1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc122ffd640 0 client.15393 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56200dead360 0x56200dec4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 --2- 172.21.15.143:0/2975025755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56200deaca40 0x56200dec4410 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 >> 172.21.15.143:0/2975025755 conn(0x56200dea90f0 msgr2=0x56200dea7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 shutdown_connections 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 1 -- 172.21.15.143:0/2975025755 wait complete. 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc14be79740 20 client.15393 trim_cache size 0 max 16384 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "42cb4ae6-66ac-49bb-a88e-e2b5f460185b", 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:45:26.366 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc12c0a2c10 msgr2=0x7fc12c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc12c0a2c10 0x7fc12c0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc13c012c80 tx=0x7fc13c009e80 comp rx=0 tx=0).stop 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 msgr2=0x7fc144198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 0x7fc144198900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc13800b4a0 tx=0x7fc13800b960 comp rx=0 tx=0).stop 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 shutdown_connections 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc12c0a2c10 0x7fc12c0a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc144106f10 0x7fc14419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc144102790 0x7fc144198900 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 --2- 172.21.15.143:0/3274494476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc144101de0 0x7fc1441983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.353+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 >> 172.21.15.143:0/3274494476 conn(0x7fc1440fd8c0 msgr2=0x7fc1440fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:26.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.354+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 shutdown_connections 2024-09-10T10:45:26.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:26.354+0000 7fc149aa0640 1 -- 172.21.15.143:0/3274494476 wait complete. 2024-09-10T10:45:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:26 smithi178 ceph-mon[34813]: pgmap v362: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-10T10:45:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:26 smithi144 ceph-mon[34230]: pgmap v362: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-10T10:45:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:26 smithi143 ceph-mon[31057]: pgmap v362: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-10T10:45:27.242 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 -- 172.21.15.143:0/775202699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4103780 msgr2=0x7f3da4107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 --2- 172.21.15.143:0/775202699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4103780 0x7f3da4107bc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009930 tx=0x7f3d9802a420 comp rx=0 tx=0).stop 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 -- 172.21.15.143:0/775202699 shutdown_connections 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 --2- 172.21.15.143:0/775202699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da4108300 0x7f3da410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 --2- 172.21.15.143:0/775202699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4103780 0x7f3da4107bc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 --2- 172.21.15.143:0/775202699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da4102dd0 0x7f3da41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 -- 172.21.15.143:0/775202699 >> 172.21.15.143:0/775202699 conn(0x7f3da40fc7d0 msgr2=0x7f3da40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 -- 172.21.15.143:0/775202699 shutdown_connections 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.420+0000 7f3da88df640 1 -- 172.21.15.143:0/775202699 wait complete. 2024-09-10T10:45:27.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 Processor -- start 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 -- start start 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da4102dd0 0x7f3da4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da4103780 0x7f3da41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 0x7f3da419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4111f00 con 0x7f3da4108300 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4111d80 con 0x7f3da4103780 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da88df640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3da4112080 con 0x7f3da4102dd0 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.421+0000 7f3da2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 0x7f3da419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 0x7f3da419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50158/0 (socket says 172.21.15.143:50158) 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 -- 172.21.15.143:0/2934166508 learned_addr learned my addr 172.21.15.143:0/2934166508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da1d9b640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da4103780 0x7f3da41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da259c640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da4102dd0 0x7f3da4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 -- 172.21.15.143:0/2934166508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da4102dd0 msgr2=0x7f3da4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da4102dd0 0x7f3da4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 -- 172.21.15.143:0/2934166508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da4103780 msgr2=0x7f3da41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da4103780 0x7f3da41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 -- 172.21.15.143:0/2934166508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da419d430 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da2d9d640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 0x7f3da419cd50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00cad0 tx=0x7f3d9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00ae20 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d9c005020 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da419d720 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00f6b0 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.422+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da41a5000 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.424+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da41031b0 con 0x7f3da4108300 2024-09-10T10:45:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.424+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3d9c0051d0 con 0x7f3da4108300 2024-09-10T10:45:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.425+0000 7f3d937fe640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3d8c0a2c10 0x7f3d8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.425+0000 7f3da259c640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3d8c0a2c10 0x7f3d8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.425+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3d9c0cf550 con 0x7f3da4108300 2024-09-10T10:45:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.426+0000 7f3da259c640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3d8c0a2c10 0x7f3d8c0a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d94005e10 tx=0x7f3d94005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.433+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d9c014030 con 0x7f3da4108300 2024-09-10T10:45:27.534 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:27.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.609+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3da4062630 con 0x7f3d8c0a2c10 2024-09-10T10:45:27.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.616+0000 7f3d937fe640 1 -- 172.21.15.143:0/2934166508 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340564 (secure 0 0 0) 0x7f3da4062630 con 0x7f3d8c0a2c10 2024-09-10T10:45:27.622 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:27.623 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:45:27.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3d8c0a2c10 msgr2=0x7f3d8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3d8c0a2c10 0x7f3d8c0a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d94005e10 tx=0x7f3d94005da0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 msgr2=0x7f3da419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 0x7f3da419cd50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00cad0 tx=0x7f3d9c00cf90 comp rx=0 tx=0).stop 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 shutdown_connections 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3d8c0a2c10 0x7f3d8c0a50d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3da4108300 0x7f3da419cd50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da4103780 0x7f3da41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 --2- 172.21.15.143:0/2934166508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3da4102dd0 0x7f3da4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 >> 172.21.15.143:0/2934166508 conn(0x7f3da40fc7d0 msgr2=0x7f3da4105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 shutdown_connections 2024-09-10T10:45:27.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:27.620+0000 7f3da88df640 1 -- 172.21.15.143:0/2934166508 wait complete. 2024-09-10T10:45:27.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:27 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:45:27.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:27 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:45:27.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:27 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:45:27.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:27 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:45:27.859 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:27 smithi143 ceph-mon[31057]: pgmap v363: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-10T10:45:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:27 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:45:27.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:27 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:45:27.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:27 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:45:27.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:27 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:45:27.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:27 smithi178 ceph-mon[34813]: pgmap v363: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-10T10:45:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:27 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:45:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:27 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:45:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:27 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:45:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:27 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:45:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:27 smithi144 ceph-mon[34230]: pgmap v363: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-09-10T10:45:28.604 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T10:45:27.471011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670512,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":787,"num_write":463,"num_write_kb":3112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375788,"kb_used_data":14060,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566036,"statfs":{"total":1151940427776,"available":1151555620864,"internally_reserved":0,"allocated":14397440,"data_stored":7980590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10095,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":86,"num_write":23,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003282"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.718019+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:45:25.718019+0000","last_peered":"2024-09-10T10:45:25.718019+0000","last_clean":"2024-09-10T10:45:25.718019+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:45:25.718019+0000","last_undegraded":"2024-09-10T10:45:25.718019+0000","last_fullsized":"2024-09-10T10:45:25.718019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441227+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:59.441227+0000","last_peered":"2024-09-10T10:44:59.441227+0000","last_clean":"2024-09-10T10:44:59.441227+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:59.441227+0000","last_undegraded":"2024-09-10T10:44:59.441227+0000","last_fullsized":"2024-09-10T10:44:59.441227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500815+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500815+0000","last_peered":"2024-09-10T10:44:00.500815+0000","last_clean":"2024-09-10T10:44:00.500815+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500815+0000","last_undegraded":"2024-09-10T10:44:00.500815+0000","last_fullsized":"2024-09-10T10:44:00.500815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505312+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505312+0000","last_peered":"2024-09-10T10:44:00.505312+0000","last_clean":"2024-09-10T10:44:00.505312+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505312+0000","last_undegraded":"2024-09-10T10:44:00.505312+0000","last_fullsized":"2024-09-10T10:44:00.505312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440037+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:59.440037+0000","last_peered":"2024-09-10T10:44:59.440037+0000","last_clean":"2024-09-10T10:44:59.440037+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:59.440037+0000","last_undegraded":"2024-09-10T10:44:59.440037+0000","last_fullsized":"2024-09-10T10:44:59.440037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503391+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503391+0000","last_peered":"2024-09-10T10:44:00.503391+0000","last_clean":"2024-09-10T10:44:00.503391+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503391+0000","last_undegraded":"2024-09-10T10:44:00.503391+0000","last_fullsized":"2024-09-10T10:44:00.503391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.560860+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:45:18.560860+0000","last_peered":"2024-09-10T10:45:18.560860+0000","last_clean":"2024-09-10T10:45:18.560860+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:45:18.560860+0000","last_undegraded":"2024-09-10T10:45:18.560860+0000","last_fullsized":"2024-09-10T10:45:18.560860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442992+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:59.442992+0000","last_peered":"2024-09-10T10:44:59.442992+0000","last_clean":"2024-09-10T10:44:59.442992+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:59.442992+0000","last_undegraded":"2024-09-10T10:44:59.442992+0000","last_fullsized":"2024-09-10T10:44:59.442992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505985+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505985+0000","last_peered":"2024-09-10T10:44:00.505985+0000","last_clean":"2024-09-10T10:44:00.505985+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505985+0000","last_undegraded":"2024-09-10T10:44:00.505985+0000","last_fullsized":"2024-09-10T10:44:00.505985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442911+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:59.442911+0000","last_peered":"2024-09-10T10:44:59.442911+0000","last_clean":"2024-09-10T10:44:59.442911+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:59.442911+0000","last_undegraded":"2024-09-10T10:44:59.442911+0000","last_fullsized":"2024-09-10T10:44:59.442911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442546+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:59.442546+0000","last_peered":"2024-09-10T10:44:59.442546+0000","last_clean":"2024-09-10T10:44:59.442546+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:59.442546+0000","last_undegraded":"2024-09-10T10:44:59.442546+0000","last_fullsized":"2024-09-10T10:44:59.442546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503983+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503983+0000","last_peered":"2024-09-10T10:44:00.503983+0000","last_clean":"2024-09-10T10:44:00.503983+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503983+0000","last_undegraded":"2024-09-10T10:44:00.503983+0000","last_fullsized":"2024-09-10T10:44:00.503983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435612+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:59.435612+0000","last_peered":"2024-09-10T10:44:59.435612+0000","last_clean":"2024-09-10T10:44:59.435612+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:59.435612+0000","last_undegraded":"2024-09-10T10:44:59.435612+0000","last_fullsized":"2024-09-10T10:44:59.435612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.106863+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:45:01.106863+0000","last_peered":"2024-09-10T10:45:01.106863+0000","last_clean":"2024-09-10T10:45:01.106863+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:45:01.106863+0000","last_undegraded":"2024-09-10T10:45:01.106863+0000","last_fullsized":"2024-09-10T10:45:01.106863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.524134+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.524134+0000","last_peered":"2024-09-10T10:44:00.524134+0000","last_clean":"2024-09-10T10:44:00.524134+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.524134+0000","last_undegraded":"2024-09-10T10:44:00.524134+0000","last_fullsized":"2024-09-10T10:44:00.524134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437632+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:59.437632+0000","last_peered":"2024-09-10T10:44:59.437632+0000","last_clean":"2024-09-10T10:44:59.437632+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:59.437632+0000","last_undegraded":"2024-09-10T10:44:59.437632+0000","last_fullsized":"2024-09-10T10:44:59.437632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504947+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504947+0000","last_peered":"2024-09-10T10:44:00.504947+0000","last_clean":"2024-09-10T10:44:00.504947+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504947+0000","last_undegraded":"2024-09-10T10:44:00.504947+0000","last_fullsized":"2024-09-10T10:44:00.504947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500343+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500343+0000","last_peered":"2024-09-10T10:44:00.500343+0000","last_clean":"2024-09-10T10:44:00.500343+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500343+0000","last_undegraded":"2024-09-10T10:44:00.500343+0000","last_fullsized":"2024-09-10T10:44:00.500343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.422791+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:59.422791+0000","last_peered":"2024-09-10T10:44:59.422791+0000","last_clean":"2024-09-10T10:44:59.422791+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:59.422791+0000","last_undegraded":"2024-09-10T10:44:59.422791+0000","last_fullsized":"2024-09-10T10:44:59.422791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510061+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510061+0000","last_peered":"2024-09-10T10:44:00.510061+0000","last_clean":"2024-09-10T10:44:00.510061+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510061+0000","last_undegraded":"2024-09-10T10:44:00.510061+0000","last_fullsized":"2024-09-10T10:44:00.510061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504098+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504098+0000","last_peered":"2024-09-10T10:44:00.504098+0000","last_clean":"2024-09-10T10:44:00.504098+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504098+0000","last_undegraded":"2024-09-10T10:44:00.504098+0000","last_fullsized":"2024-09-10T10:44:00.504098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441460+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:59.441460+0000","last_peered":"2024-09-10T10:44:59.441460+0000","last_clean":"2024-09-10T10:44:59.441460+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:59.441460+0000","last_undegraded":"2024-09-10T10:44:59.441460+0000","last_fullsized":"2024-09-10T10:44:59.441460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504342+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504342+0000","last_peered":"2024-09-10T10:44:00.504342+0000","last_clean":"2024-09-10T10:44:00.504342+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504342+0000","last_undegraded":"2024-09-10T10:44:00.504342+0000","last_fullsized":"2024-09-10T10:44:00.504342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500934+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500934+0000","last_peered":"2024-09-10T10:44:00.500934+0000","last_clean":"2024-09-10T10:44:00.500934+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500934+0000","last_undegraded":"2024-09-10T10:44:00.500934+0000","last_fullsized":"2024-09-10T10:44:00.500934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440153+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:59.440153+0000","last_peered":"2024-09-10T10:44:59.440153+0000","last_clean":"2024-09-10T10:44:59.440153+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:59.440153+0000","last_undegraded":"2024-09-10T10:44:59.440153+0000","last_fullsized":"2024-09-10T10:44:59.440153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'18","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.270359+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:45:15.270359+0000","last_peered":"2024-09-10T10:45:15.270359+0000","last_clean":"2024-09-10T10:45:15.270359+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:45:15.270359+0000","last_undegraded":"2024-09-10T10:45:15.270359+0000","last_fullsized":"2024-09-10T10:45:15.270359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504339+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504339+0000","last_peered":"2024-09-10T10:44:00.504339+0000","last_clean":"2024-09-10T10:44:00.504339+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504339+0000","last_undegraded":"2024-09-10T10:44:00.504339+0000","last_fullsized":"2024-09-10T10:44:00.504339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443706+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:59.443706+0000","last_peered":"2024-09-10T10:44:59.443706+0000","last_clean":"2024-09-10T10:44:59.443706+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:59.443706+0000","last_undegraded":"2024-09-10T10:44:59.443706+0000","last_fullsized":"2024-09-10T10:44:59.443706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506338+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506338+0000","last_peered":"2024-09-10T10:44:00.506338+0000","last_clean":"2024-09-10T10:44:00.506338+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506338+0000","last_undegraded":"2024-09-10T10:44:00.506338+0000","last_fullsized":"2024-09-10T10:44:00.506338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503217+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503217+0000","last_peered":"2024-09-10T10:44:00.503217+0000","last_clean":"2024-09-10T10:44:00.503217+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503217+0000","last_undegraded":"2024-09-10T10:44:00.503217+0000","last_fullsized":"2024-09-10T10:44:00.503217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.424958+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:59.424958+0000","last_peered":"2024-09-10T10:44:59.424958+0000","last_clean":"2024-09-10T10:44:59.424958+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:59.424958+0000","last_undegraded":"2024-09-10T10:44:59.424958+0000","last_fullsized":"2024-09-10T10:44:59.424958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501376+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501376+0000","last_peered":"2024-09-10T10:44:00.501376+0000","last_clean":"2024-09-10T10:44:00.501376+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501376+0000","last_undegraded":"2024-09-10T10:44:00.501376+0000","last_fullsized":"2024-09-10T10:44:00.501376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505353+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505353+0000","last_peered":"2024-09-10T10:44:00.505353+0000","last_clean":"2024-09-10T10:44:00.505353+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505353+0000","last_undegraded":"2024-09-10T10:44:00.505353+0000","last_fullsized":"2024-09-10T10:44:00.505353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438858+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:59.438858+0000","last_peered":"2024-09-10T10:44:59.438858+0000","last_clean":"2024-09-10T10:44:59.438858+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:59.438858+0000","last_undegraded":"2024-09-10T10:44:59.438858+0000","last_fullsized":"2024-09-10T10:44:59.438858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:23.716080+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:45:23.716080+0000","last_peered":"2024-09-10T10:45:23.716080+0000","last_clean":"2024-09-10T10:45:23.716080+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:45:23.716080+0000","last_undegraded":"2024-09-10T10:45:23.716080+0000","last_fullsized":"2024-09-10T10:45:23.716080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502631+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502631+0000","last_peered":"2024-09-10T10:44:00.502631+0000","last_clean":"2024-09-10T10:44:00.502631+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502631+0000","last_undegraded":"2024-09-10T10:44:00.502631+0000","last_fullsized":"2024-09-10T10:44:00.502631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439106+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:59.439106+0000","last_peered":"2024-09-10T10:44:59.439106+0000","last_clean":"2024-09-10T10:44:59.439106+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:59.439106+0000","last_undegraded":"2024-09-10T10:44:59.439106+0000","last_fullsized":"2024-09-10T10:44:59.439106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:22.719923+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:45:22.719923+0000","last_peered":"2024-09-10T10:45:22.719923+0000","last_clean":"2024-09-10T10:45:22.719923+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:45:22.719923+0000","last_undegraded":"2024-09-10T10:45:22.719923+0000","last_fullsized":"2024-09-10T10:45:22.719923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502777+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502777+0000","last_peered":"2024-09-10T10:44:00.502777+0000","last_clean":"2024-09-10T10:44:00.502777+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502777+0000","last_undegraded":"2024-09-10T10:44:00.502777+0000","last_fullsized":"2024-09-10T10:44:00.502777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444099+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:59.444099+0000","last_peered":"2024-09-10T10:44:59.444099+0000","last_clean":"2024-09-10T10:44:59.444099+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:59.444099+0000","last_undegraded":"2024-09-10T10:44:59.444099+0000","last_fullsized":"2024-09-10T10:44:59.444099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506093+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506093+0000","last_peered":"2024-09-10T10:44:00.506093+0000","last_clean":"2024-09-10T10:44:00.506093+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506093+0000","last_undegraded":"2024-09-10T10:44:00.506093+0000","last_fullsized":"2024-09-10T10:44:00.506093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516092+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.516092+0000","last_peered":"2024-09-10T10:44:00.516092+0000","last_clean":"2024-09-10T10:44:00.516092+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.516092+0000","last_undegraded":"2024-09-10T10:44:00.516092+0000","last_fullsized":"2024-09-10T10:44:00.516092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443062+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:59.443062+0000","last_peered":"2024-09-10T10:44:59.443062+0000","last_clean":"2024-09-10T10:44:59.443062+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:59.443062+0000","last_undegraded":"2024-09-10T10:44:59.443062+0000","last_fullsized":"2024-09-10T10:44:59.443062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503953+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503953+0000","last_peered":"2024-09-10T10:44:00.503953+0000","last_clean":"2024-09-10T10:44:00.503953+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503953+0000","last_undegraded":"2024-09-10T10:44:00.503953+0000","last_fullsized":"2024-09-10T10:44:00.503953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507101+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507101+0000","last_peered":"2024-09-10T10:44:00.507101+0000","last_clean":"2024-09-10T10:44:00.507101+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507101+0000","last_undegraded":"2024-09-10T10:44:00.507101+0000","last_fullsized":"2024-09-10T10:44:00.507101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442450+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:59.442450+0000","last_peered":"2024-09-10T10:44:59.442450+0000","last_clean":"2024-09-10T10:44:59.442450+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:59.442450+0000","last_undegraded":"2024-09-10T10:44:59.442450+0000","last_fullsized":"2024-09-10T10:44:59.442450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:24.770712+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:45:24.770712+0000","last_peered":"2024-09-10T10:45:24.770712+0000","last_clean":"2024-09-10T10:45:24.770712+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:45:24.770712+0000","last_undegraded":"2024-09-10T10:45:24.770712+0000","last_fullsized":"2024-09-10T10:45:24.770712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509662+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509662+0000","last_peered":"2024-09-10T10:44:00.509662+0000","last_clean":"2024-09-10T10:44:00.509662+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509662+0000","last_undegraded":"2024-09-10T10:44:00.509662+0000","last_fullsized":"2024-09-10T10:44:00.509662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442843+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:59.442843+0000","last_peered":"2024-09-10T10:44:59.442843+0000","last_clean":"2024-09-10T10:44:59.442843+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:59.442843+0000","last_undegraded":"2024-09-10T10:44:59.442843+0000","last_fullsized":"2024-09-10T10:44:59.442843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516384+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516384+0000","last_peered":"2024-09-10T10:44:00.516384+0000","last_clean":"2024-09-10T10:44:00.516384+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516384+0000","last_undegraded":"2024-09-10T10:44:00.516384+0000","last_fullsized":"2024-09-10T10:44:00.516384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500645+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500645+0000","last_peered":"2024-09-10T10:44:00.500645+0000","last_clean":"2024-09-10T10:44:00.500645+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500645+0000","last_undegraded":"2024-09-10T10:44:00.500645+0000","last_fullsized":"2024-09-10T10:44:00.500645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440420+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:59.440420+0000","last_peered":"2024-09-10T10:44:59.440420+0000","last_clean":"2024-09-10T10:44:59.440420+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:59.440420+0000","last_undegraded":"2024-09-10T10:44:59.440420+0000","last_fullsized":"2024-09-10T10:44:59.440420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503016+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.503016+0000","last_peered":"2024-09-10T10:44:00.503016+0000","last_clean":"2024-09-10T10:44:00.503016+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.503016+0000","last_undegraded":"2024-09-10T10:44:00.503016+0000","last_fullsized":"2024-09-10T10:44:00.503016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515085+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515085+0000","last_peered":"2024-09-10T10:44:00.515085+0000","last_clean":"2024-09-10T10:44:00.515085+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515085+0000","last_undegraded":"2024-09-10T10:44:00.515085+0000","last_fullsized":"2024-09-10T10:44:00.515085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440308+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:59.440308+0000","last_peered":"2024-09-10T10:44:59.440308+0000","last_clean":"2024-09-10T10:44:59.440308+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:59.440308+0000","last_undegraded":"2024-09-10T10:44:59.440308+0000","last_fullsized":"2024-09-10T10:44:59.440308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:05.823762+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:45:05.823762+0000","last_peered":"2024-09-10T10:45:05.823762+0000","last_clean":"2024-09-10T10:45:05.823762+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:45:05.823762+0000","last_undegraded":"2024-09-10T10:45:05.823762+0000","last_fullsized":"2024-09-10T10:45:05.823762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515302+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515302+0000","last_peered":"2024-09-10T10:44:00.515302+0000","last_clean":"2024-09-10T10:44:00.515302+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515302+0000","last_undegraded":"2024-09-10T10:44:00.515302+0000","last_fullsized":"2024-09-10T10:44:00.515302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442616+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442616+0000","last_peered":"2024-09-10T10:44:59.442616+0000","last_clean":"2024-09-10T10:44:59.442616+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:59.442616+0000","last_undegraded":"2024-09-10T10:44:59.442616+0000","last_fullsized":"2024-09-10T10:44:59.442616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505428+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505428+0000","last_peered":"2024-09-10T10:44:00.505428+0000","last_clean":"2024-09-10T10:44:00.505428+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505428+0000","last_undegraded":"2024-09-10T10:44:00.505428+0000","last_fullsized":"2024-09-10T10:44:00.505428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507155+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.507155+0000","last_peered":"2024-09-10T10:44:00.507155+0000","last_clean":"2024-09-10T10:44:00.507155+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.507155+0000","last_undegraded":"2024-09-10T10:44:00.507155+0000","last_fullsized":"2024-09-10T10:44:00.507155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441344+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:59.441344+0000","last_peered":"2024-09-10T10:44:59.441344+0000","last_clean":"2024-09-10T10:44:59.441344+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:59.441344+0000","last_undegraded":"2024-09-10T10:44:59.441344+0000","last_fullsized":"2024-09-10T10:44:59.441344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:21.713408+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:45:21.713408+0000","last_peered":"2024-09-10T10:45:21.713408+0000","last_clean":"2024-09-10T10:45:21.713408+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:45:21.713408+0000","last_undegraded":"2024-09-10T10:45:21.713408+0000","last_fullsized":"2024-09-10T10:45:21.713408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500386+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500386+0000","last_peered":"2024-09-10T10:44:00.500386+0000","last_clean":"2024-09-10T10:44:00.500386+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500386+0000","last_undegraded":"2024-09-10T10:44:00.500386+0000","last_fullsized":"2024-09-10T10:44:00.500386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445872+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:59.445872+0000","last_peered":"2024-09-10T10:44:59.445872+0000","last_clean":"2024-09-10T10:44:59.445872+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:59.445872+0000","last_undegraded":"2024-09-10T10:44:59.445872+0000","last_fullsized":"2024-09-10T10:44:59.445872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515497+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515497+0000","last_peered":"2024-09-10T10:44:00.515497+0000","last_clean":"2024-09-10T10:44:00.515497+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515497+0000","last_undegraded":"2024-09-10T10:44:00.515497+0000","last_fullsized":"2024-09-10T10:44:00.515497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502910+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502910+0000","last_peered":"2024-09-10T10:44:00.502910+0000","last_clean":"2024-09-10T10:44:00.502910+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502910+0000","last_undegraded":"2024-09-10T10:44:00.502910+0000","last_fullsized":"2024-09-10T10:44:00.502910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435638+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:59.435638+0000","last_peered":"2024-09-10T10:44:59.435638+0000","last_clean":"2024-09-10T10:44:59.435638+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:59.435638+0000","last_undegraded":"2024-09-10T10:44:59.435638+0000","last_fullsized":"2024-09-10T10:44:59.435638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516482+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516482+0000","last_peered":"2024-09-10T10:44:00.516482+0000","last_clean":"2024-09-10T10:44:00.516482+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516482+0000","last_undegraded":"2024-09-10T10:44:00.516482+0000","last_fullsized":"2024-09-10T10:44:00.516482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":336,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.639756+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:45:25.639756+0000","last_peered":"2024-09-10T10:45:25.639756+0000","last_clean":"2024-09-10T10:45:25.639756+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:45:25.639756+0000","last_undegraded":"2024-09-10T10:45:25.639756+0000","last_fullsized":"2024-09-10T10:45:25.639756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70047,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":186,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500133+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500133+0000","last_peered":"2024-09-10T10:44:00.500133+0000","last_clean":"2024-09-10T10:44:00.500133+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500133+0000","last_undegraded":"2024-09-10T10:44:00.500133+0000","last_fullsized":"2024-09-10T10:44:00.500133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516289+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516289+0000","last_peered":"2024-09-10T10:44:00.516289+0000","last_clean":"2024-09-10T10:44:00.516289+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516289+0000","last_undegraded":"2024-09-10T10:44:00.516289+0000","last_fullsized":"2024-09-10T10:44:00.516289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503829+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503829+0000","last_peered":"2024-09-10T10:44:00.503829+0000","last_clean":"2024-09-10T10:44:00.503829+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503829+0000","last_undegraded":"2024-09-10T10:44:00.503829+0000","last_fullsized":"2024-09-10T10:44:00.503829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506656+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506656+0000","last_peered":"2024-09-10T10:44:00.506656+0000","last_clean":"2024-09-10T10:44:00.506656+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506656+0000","last_undegraded":"2024-09-10T10:44:00.506656+0000","last_fullsized":"2024-09-10T10:44:00.506656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499917+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499917+0000","last_peered":"2024-09-10T10:44:00.499917+0000","last_clean":"2024-09-10T10:44:00.499917+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499917+0000","last_undegraded":"2024-09-10T10:44:00.499917+0000","last_fullsized":"2024-09-10T10:44:00.499917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442320+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:59.442320+0000","last_peered":"2024-09-10T10:44:59.442320+0000","last_clean":"2024-09-10T10:44:59.442320+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:59.442320+0000","last_undegraded":"2024-09-10T10:44:59.442320+0000","last_fullsized":"2024-09-10T10:44:59.442320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":270,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.721017+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:45:25.721017+0000","last_peered":"2024-09-10T10:45:25.721017+0000","last_clean":"2024-09-10T10:45:25.721017+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:45:25.721017+0000","last_undegraded":"2024-09-10T10:45:25.721017+0000","last_fullsized":"2024-09-10T10:45:25.721017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504776+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504776+0000","last_peered":"2024-09-10T10:44:00.504776+0000","last_clean":"2024-09-10T10:44:00.504776+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504776+0000","last_undegraded":"2024-09-10T10:44:00.504776+0000","last_fullsized":"2024-09-10T10:44:00.504776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500929+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500929+0000","last_peered":"2024-09-10T10:44:00.500929+0000","last_clean":"2024-09-10T10:44:00.500929+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500929+0000","last_undegraded":"2024-09-10T10:44:00.500929+0000","last_fullsized":"2024-09-10T10:44:00.500929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.433074+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:59.433074+0000","last_peered":"2024-09-10T10:44:59.433074+0000","last_clean":"2024-09-10T10:44:59.433074+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:59.433074+0000","last_undegraded":"2024-09-10T10:44:59.433074+0000","last_fullsized":"2024-09-10T10:44:59.433074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515598+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515598+0000","last_peered":"2024-09-10T10:44:00.515598+0000","last_clean":"2024-09-10T10:44:00.515598+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515598+0000","last_undegraded":"2024-09-10T10:44:00.515598+0000","last_fullsized":"2024-09-10T10:44:00.515598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509926+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509926+0000","last_peered":"2024-09-10T10:44:00.509926+0000","last_clean":"2024-09-10T10:44:00.509926+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509926+0000","last_undegraded":"2024-09-10T10:44:00.509926+0000","last_fullsized":"2024-09-10T10:44:00.509926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.434606+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:59.434606+0000","last_peered":"2024-09-10T10:44:59.434606+0000","last_clean":"2024-09-10T10:44:59.434606+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:59.434606+0000","last_undegraded":"2024-09-10T10:44:59.434606+0000","last_fullsized":"2024-09-10T10:44:59.434606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502516+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502516+0000","last_peered":"2024-09-10T10:44:00.502516+0000","last_clean":"2024-09-10T10:44:00.502516+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502516+0000","last_undegraded":"2024-09-10T10:44:00.502516+0000","last_fullsized":"2024-09-10T10:44:00.502516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505395+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505395+0000","last_peered":"2024-09-10T10:44:00.505395+0000","last_clean":"2024-09-10T10:44:00.505395+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505395+0000","last_undegraded":"2024-09-10T10:44:00.505395+0000","last_fullsized":"2024-09-10T10:44:00.505395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438633+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:59.438633+0000","last_peered":"2024-09-10T10:44:59.438633+0000","last_clean":"2024-09-10T10:44:59.438633+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:59.438633+0000","last_undegraded":"2024-09-10T10:44:59.438633+0000","last_fullsized":"2024-09-10T10:44:59.438633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.721739+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:45:25.721739+0000","last_peered":"2024-09-10T10:45:25.721739+0000","last_clean":"2024-09-10T10:45:25.721739+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:45:25.721739+0000","last_undegraded":"2024-09-10T10:45:25.721739+0000","last_fullsized":"2024-09-10T10:45:25.721739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439433+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:59.439433+0000","last_peered":"2024-09-10T10:44:59.439433+0000","last_clean":"2024-09-10T10:44:59.439433+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:59.439433+0000","last_undegraded":"2024-09-10T10:44:59.439433+0000","last_fullsized":"2024-09-10T10:44:59.439433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440974+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:59.440974+0000","last_peered":"2024-09-10T10:44:59.440974+0000","last_clean":"2024-09-10T10:44:59.440974+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:59.440974+0000","last_undegraded":"2024-09-10T10:44:59.440974+0000","last_fullsized":"2024-09-10T10:44:59.440974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441221+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:59.441221+0000","last_peered":"2024-09-10T10:44:59.441221+0000","last_clean":"2024-09-10T10:44:59.441221+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:59.441221+0000","last_undegraded":"2024-09-10T10:44:59.441221+0000","last_fullsized":"2024-09-10T10:44:59.441221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440703+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:59.440703+0000","last_peered":"2024-09-10T10:44:59.440703+0000","last_clean":"2024-09-10T10:44:59.440703+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:59.440703+0000","last_undegraded":"2024-09-10T10:44:59.440703+0000","last_fullsized":"2024-09-10T10:44:59.440703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444329+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:59.444329+0000","last_peered":"2024-09-10T10:44:59.444329+0000","last_clean":"2024-09-10T10:44:59.444329+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:59.444329+0000","last_undegraded":"2024-09-10T10:44:59.444329+0000","last_fullsized":"2024-09-10T10:44:59.444329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438450+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:59.438450+0000","last_peered":"2024-09-10T10:44:59.438450+0000","last_clean":"2024-09-10T10:44:59.438450+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:59.438450+0000","last_undegraded":"2024-09-10T10:44:59.438450+0000","last_fullsized":"2024-09-10T10:44:59.438450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441275+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:59.441275+0000","last_peered":"2024-09-10T10:44:59.441275+0000","last_clean":"2024-09-10T10:44:59.441275+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:59.441275+0000","last_undegraded":"2024-09-10T10:44:59.441275+0000","last_fullsized":"2024-09-10T10:44:59.441275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437164+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:59.437164+0000","last_peered":"2024-09-10T10:44:59.437164+0000","last_clean":"2024-09-10T10:44:59.437164+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:59.437164+0000","last_undegraded":"2024-09-10T10:44:59.437164+0000","last_fullsized":"2024-09-10T10:44:59.437164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441514+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:59.441514+0000","last_peered":"2024-09-10T10:44:59.441514+0000","last_clean":"2024-09-10T10:44:59.441514+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:59.441514+0000","last_undegraded":"2024-09-10T10:44:59.441514+0000","last_fullsized":"2024-09-10T10:44:59.441514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442360+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:59.442360+0000","last_peered":"2024-09-10T10:44:59.442360+0000","last_clean":"2024-09-10T10:44:59.442360+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:59.442360+0000","last_undegraded":"2024-09-10T10:44:59.442360+0000","last_fullsized":"2024-09-10T10:44:59.442360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442124+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:59.442124+0000","last_peered":"2024-09-10T10:44:59.442124+0000","last_clean":"2024-09-10T10:44:59.442124+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:59.442124+0000","last_undegraded":"2024-09-10T10:44:59.442124+0000","last_fullsized":"2024-09-10T10:44:59.442124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442058+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:59.442058+0000","last_peered":"2024-09-10T10:44:59.442058+0000","last_clean":"2024-09-10T10:44:59.442058+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:59.442058+0000","last_undegraded":"2024-09-10T10:44:59.442058+0000","last_fullsized":"2024-09-10T10:44:59.442058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438851+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:59.438851+0000","last_peered":"2024-09-10T10:44:59.438851+0000","last_clean":"2024-09-10T10:44:59.438851+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:59.438851+0000","last_undegraded":"2024-09-10T10:44:59.438851+0000","last_fullsized":"2024-09-10T10:44:59.438851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429557+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:59.429557+0000","last_peered":"2024-09-10T10:44:59.429557+0000","last_clean":"2024-09-10T10:44:59.429557+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:59.429557+0000","last_undegraded":"2024-09-10T10:44:59.429557+0000","last_fullsized":"2024-09-10T10:44:59.429557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440551+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:59.440551+0000","last_peered":"2024-09-10T10:44:59.440551+0000","last_clean":"2024-09-10T10:44:59.440551+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:59.440551+0000","last_undegraded":"2024-09-10T10:44:59.440551+0000","last_fullsized":"2024-09-10T10:44:59.440551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437183+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:59.437183+0000","last_peered":"2024-09-10T10:44:59.437183+0000","last_clean":"2024-09-10T10:44:59.437183+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:59.437183+0000","last_undegraded":"2024-09-10T10:44:59.437183+0000","last_fullsized":"2024-09-10T10:44:59.437183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439780+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:59.439780+0000","last_peered":"2024-09-10T10:44:59.439780+0000","last_clean":"2024-09-10T10:44:59.439780+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:59.439780+0000","last_undegraded":"2024-09-10T10:44:59.439780+0000","last_fullsized":"2024-09-10T10:44:59.439780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441856+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:59.441856+0000","last_peered":"2024-09-10T10:44:59.441856+0000","last_clean":"2024-09-10T10:44:59.441856+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:59.441856+0000","last_undegraded":"2024-09-10T10:44:59.441856+0000","last_fullsized":"2024-09-10T10:44:59.441856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437328+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:59.437328+0000","last_peered":"2024-09-10T10:44:59.437328+0000","last_clean":"2024-09-10T10:44:59.437328+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:59.437328+0000","last_undegraded":"2024-09-10T10:44:59.437328+0000","last_fullsized":"2024-09-10T10:44:59.437328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438618+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:59.438618+0000","last_peered":"2024-09-10T10:44:59.438618+0000","last_clean":"2024-09-10T10:44:59.438618+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:59.438618+0000","last_undegraded":"2024-09-10T10:44:59.438618+0000","last_fullsized":"2024-09-10T10:44:59.438618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444056+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:59.444056+0000","last_peered":"2024-09-10T10:44:59.444056+0000","last_clean":"2024-09-10T10:44:59.444056+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:59.444056+0000","last_undegraded":"2024-09-10T10:44:59.444056+0000","last_fullsized":"2024-09-10T10:44:59.444056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442679+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:59.442679+0000","last_peered":"2024-09-10T10:44:59.442679+0000","last_clean":"2024-09-10T10:44:59.442679+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:59.442679+0000","last_undegraded":"2024-09-10T10:44:59.442679+0000","last_fullsized":"2024-09-10T10:44:59.442679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.433231+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:59.433231+0000","last_peered":"2024-09-10T10:44:59.433231+0000","last_clean":"2024-09-10T10:44:59.433231+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:59.433231+0000","last_undegraded":"2024-09-10T10:44:59.433231+0000","last_fullsized":"2024-09-10T10:44:59.433231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442732+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442732+0000","last_peered":"2024-09-10T10:44:59.442732+0000","last_clean":"2024-09-10T10:44:59.442732+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:59.442732+0000","last_undegraded":"2024-09-10T10:44:59.442732+0000","last_fullsized":"2024-09-10T10:44:59.442732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440910+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:59.440910+0000","last_peered":"2024-09-10T10:44:59.440910+0000","last_clean":"2024-09-10T10:44:59.440910+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:59.440910+0000","last_undegraded":"2024-09-10T10:44:59.440910+0000","last_fullsized":"2024-09-10T10:44:59.440910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445105+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:59.445105+0000","last_peered":"2024-09-10T10:44:59.445105+0000","last_clean":"2024-09-10T10:44:59.445105+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:59.445105+0000","last_undegraded":"2024-09-10T10:44:59.445105+0000","last_fullsized":"2024-09-10T10:44:59.445105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428701+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:59.428701+0000","last_peered":"2024-09-10T10:44:59.428701+0000","last_clean":"2024-09-10T10:44:59.428701+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:59.428701+0000","last_undegraded":"2024-09-10T10:44:59.428701+0000","last_fullsized":"2024-09-10T10:44:59.428701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429050+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:59.429050+0000","last_peered":"2024-09-10T10:44:59.429050+0000","last_clean":"2024-09-10T10:44:59.429050+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:59.429050+0000","last_undegraded":"2024-09-10T10:44:59.429050+0000","last_fullsized":"2024-09-10T10:44:59.429050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441024+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:59.441024+0000","last_peered":"2024-09-10T10:44:59.441024+0000","last_clean":"2024-09-10T10:44:59.441024+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:59.441024+0000","last_undegraded":"2024-09-10T10:44:59.441024+0000","last_fullsized":"2024-09-10T10:44:59.441024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441764+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:59.441764+0000","last_peered":"2024-09-10T10:44:59.441764+0000","last_clean":"2024-09-10T10:44:59.441764+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:59.441764+0000","last_undegraded":"2024-09-10T10:44:59.441764+0000","last_fullsized":"2024-09-10T10:44:59.441764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439409+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:59.439409+0000","last_peered":"2024-09-10T10:44:59.439409+0000","last_clean":"2024-09-10T10:44:59.439409+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:59.439409+0000","last_undegraded":"2024-09-10T10:44:59.439409+0000","last_fullsized":"2024-09-10T10:44:59.439409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439118+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:59.439118+0000","last_peered":"2024-09-10T10:44:59.439118+0000","last_clean":"2024-09-10T10:44:59.439118+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:59.439118+0000","last_undegraded":"2024-09-10T10:44:59.439118+0000","last_fullsized":"2024-09-10T10:44:59.439118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79910,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":678,"num_write":280,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":239730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547242,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":501134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776178,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1162025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168636,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1098863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":496359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790028,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":570621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215189,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1024000,"data_stored":498177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":567436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:28.609 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:45:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 -- 172.21.15.143:0/56445769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4106f00 msgr2=0x7f95e410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 --2- 172.21.15.143:0/56445769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4106f00 0x7f95e410ea80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009ca0 tx=0x7f95dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 -- 172.21.15.143:0/56445769 shutdown_connections 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 --2- 172.21.15.143:0/56445769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4106f00 0x7f95e410ea80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 --2- 172.21.15.143:0/56445769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4102780 0x7f95e41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 --2- 172.21.15.143:0/56445769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e4101dd0 0x7f95e41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 -- 172.21.15.143:0/56445769 >> 172.21.15.143:0/56445769 conn(0x7f95e40fd8b0 msgr2=0x7f95e40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 -- 172.21.15.143:0/56445769 shutdown_connections 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.792+0000 7f95e9855640 1 -- 172.21.15.143:0/56445769 wait complete. 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.793+0000 7f95e9855640 1 Processor -- start 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.793+0000 7f95e9855640 1 -- start start 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e9855640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4101dd0 0x7f95e4073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e9855640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 0x7f95e406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e9855640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e4106f00 0x7f95e406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e9855640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111e10 con 0x7f95e4102780 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e9855640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111c90 con 0x7f95e4101dd0 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e9855640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111f90 con 0x7f95e4106f00 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 0x7f95e406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 0x7f95e406ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50178/0 (socket says 172.21.15.143:50178) 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 -- 172.21.15.143:0/1966532812 learned_addr learned my addr 172.21.15.143:0/1966532812 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4101dd0 0x7f95e4073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e37fe640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e4106f00 0x7f95e406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 -- 172.21.15.143:0/1966532812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e4106f00 msgr2=0x7f95e406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e4106f00 0x7f95e406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 -- 172.21.15.143:0/1966532812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4101dd0 msgr2=0x7f95e4073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.794+0000 7f95e27fc640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4101dd0 0x7f95e4073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e27fc640 1 -- 172.21.15.143:0/1966532812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e406d590 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4101dd0 0x7f95e4073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e27fc640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 0x7f95e406ca90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f95d800d950 tx=0x7f95d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e8853640 1 -- 172.21.15.143:0/1966532812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8017070 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e8853640 1 -- 172.21.15.143:0/1966532812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d8009c30 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e406d880 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e8853640 1 -- 172.21.15.143:0/1966532812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800f620 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.795+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e40718a0 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.797+0000 7f95e8853640 1 -- 172.21.15.143:0/1966532812 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f95d80109a0 con 0x7f95e4102780 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.797+0000 7f95ebc2e740 1 Processor -- start 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 -- start start 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90ea4a0 0x564af90ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e8853640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95cc0a2c90 0x7f95cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af90f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564af90502e0 con 0x564af90eadc0 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564af8ff7ce0 con 0x564af90f5680 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564af904c2a0 con 0x564af90ea4a0 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af90f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af90f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50184/0 (socket says 172.21.15.143:50184) 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 -- 172.21.15.143:0/510140307 learned_addr learned my addr 172.21.15.143:0/510140307 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95cc0a2c90 0x7f95cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e37fe640 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90ea4a0 0x564af90ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e37fe640 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e8853640 1 -- 172.21.15.143:0/1966532812 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95d80cf0b0 con 0x7f95e4102780 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 -- 172.21.15.143:0/510140307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90ea4a0 msgr2=0x564af90ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90ea4a0 0x564af90ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 -- 172.21.15.143:0/510140307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 msgr2=0x564af90f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e27fc640 1 -- 172.21.15.143:0/510140307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af8fdc440 con 0x564af90eadc0 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.798+0000 7f95e37fe640 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90ea4a0 0x564af90ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95cc0a2c90 0x7f95cc0a5150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fc0 tx=0x7f95d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95e27fc640 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af90f5140 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f95d807f700 tx=0x7f95d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0ebc7890b6e309 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95c9ffb640 1 -- 172.21.15.143:0/510140307 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d80adf90 con 0x564af90eadc0 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95c9ffb640 1 -- 172.21.15.143:0/510140307 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d809bb50 con 0x564af90eadc0 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95c9ffb640 1 -- 172.21.15.143:0/510140307 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d80a40c0 con 0x564af90eadc0 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95ebc2e740 1 -- 172.21.15.143:0/510140307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 msgr2=0x564af90f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.799+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af90f5140 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f95d807f700 tx=0x7f95d8005880 comp rx=0 tx=0).stop 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 -- 172.21.15.143:0/510140307 shutdown_connections 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af90f5140 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/510140307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90ea4a0 0x564af90ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 -- 172.21.15.143:0/510140307 >> 172.21.15.143:0/510140307 conn(0x564af90e6a30 msgr2=0x564af90e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 -- 172.21.15.143:0/510140307 shutdown_connections 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 -- 172.21.15.143:0/510140307 wait complete. 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 -- start start 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 Processor -- start 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 0x564af90f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90f2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.800+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564af90f81f0 con 0x564af90ea4a0 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564af900b050 con 0x564af90f5680 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564af90f8370 con 0x564af90eadc0 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95ebc2e740 10 client.0 _ll_register_callbacks cb 0x564af904b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 0x564af90f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 0x564af90f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50196/0 (socket says 172.21.15.143:50196) 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 -- 172.21.15.143:0/2603907624 learned_addr learned my addr 172.21.15.143:0/2603907624 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e37fe640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e27fc640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 -- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 msgr2=0x564af90f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 -- 172.21.15.143:0/2603907624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 msgr2=0x564af90f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 -- 172.21.15.143:0/2603907624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af8dc13c0 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e27fc640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 0x564af90f4fc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f95d4003830 tx=0x7f95d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ccffc22a22ed89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95c2ffd640 1 -- 172.21.15.143:0/2603907624 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d400f980 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d809bb50 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95c2ffd640 1 -- 172.21.15.143:0/2603907624 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95d400fb20 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95c2ffd640 1 -- 172.21.15.143:0/2603907624 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95d400fd20 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.801+0000 7f95ebc2e740 10 client.15411 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564af9103c50 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95c2ffd640 1 -- 172.21.15.143:0/2603907624 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d809bb50 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95c2ffd640 1 -- 172.21.15.143:0/2603907624 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f95d401a070 con 0x564af90ea4a0 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 10 client.15411 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:45:28.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 20 client.15411 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af9105490 0x564af9107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 4 client.15411 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564af905d850 con 0x564af9105490 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.802+0000 7f95e27fc640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af9105490 0x564af9107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95e27fc640 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af9105490 0x564af9107910 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e44da31bbc37a5 server_cookie=26bc9bbab6361e71 in_seq=0 out_seq=0 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95c2ffd640 10 client.15411 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95c2ffd640 1 -- 172.21.15.143:0/2603907624 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564af905d850 con 0x564af9105490 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95c2ffd640 10 client.15411 handle_command_reply: tid=0 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95ebc2e740 1 client.15411 shutdown 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 msgr2=0x564af90f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.803+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 0x564af90f4fc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f95d4003830 tx=0x7f95d4004730 comp rx=0 tx=0).stop 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 shutdown_connections 2024-09-10T10:45:28.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90f5680 0x564af90f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af9105490 0x564af9107910 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/2603907624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90ea4a0 0x564af90f4fc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 >> 172.21.15.143:0/2603907624 conn(0x564af90e6a30 msgr2=0x564af90ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95c2ffd640 0 client.15411 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 shutdown_connections 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 1 -- 172.21.15.143:0/2603907624 wait complete. 2024-09-10T10:45:28.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.804+0000 7f95ebc2e740 20 client.15411 trim_cache size 0 max 16384 2024-09-10T10:45:28.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 Processor -- start 2024-09-10T10:45:28.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 -- start start 2024-09-10T10:45:28.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90e6a30 0x564af90f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af910d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af910d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b73f0 con 0x564af90eadc0 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b8f30 con 0x564af90ea4a0 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b7ac0 con 0x564af90e6a30 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95e37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af910d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95e37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af910d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50202/0 (socket says 172.21.15.143:50202) 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95e37fe640 1 -- 172.21.15.143:0/3623691381 learned_addr learned my addr 172.21.15.143:0/3623691381 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95e27fc640 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af910d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.821+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90e6a30 0x564af90f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e37fe640 1 -- 172.21.15.143:0/3623691381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90e6a30 msgr2=0x564af90f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e37fe640 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90e6a30 0x564af90f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e37fe640 1 -- 172.21.15.143:0/3623691381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 msgr2=0x564af910d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e37fe640 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af910d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e37fe640 1 -- 172.21.15.143:0/3623691381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af90f4be0 con 0x564af90eadc0 2024-09-10T10:45:28.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e27fc640 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af910d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:28.830 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:28 smithi143 ceph-mon[31057]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95e37fe640 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af910d760 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f95dc035000 tx=0x7f95dc031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824f23dbaa3c0128 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95c97fa640 1 -- 172.21.15.143:0/3623691381 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc049070 con 0x564af90eadc0 2024-09-10T10:45:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95c97fa640 1 -- 172.21.15.143:0/3623691381 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95dc036040 con 0x564af90eadc0 2024-09-10T10:45:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.822+0000 7f95c97fa640 1 -- 172.21.15.143:0/3623691381 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc004390 con 0x564af90eadc0 2024-09-10T10:45:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 -- 172.21.15.143:0/3623691381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 msgr2=0x564af910d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af910d760 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f95dc035000 tx=0x7f95dc031820 comp rx=0 tx=0).stop 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 -- 172.21.15.143:0/3623691381 shutdown_connections 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90eadc0 0x564af910d760 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af910d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/3623691381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90e6a30 0x564af90f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 -- 172.21.15.143:0/3623691381 >> 172.21.15.143:0/3623691381 conn(0x564af90f2910 msgr2=0x564af90f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 -- 172.21.15.143:0/3623691381 shutdown_connections 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 -- 172.21.15.143:0/3623691381 wait complete. 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.823+0000 7f95ebc2e740 1 -- start start 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 Processor -- start 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 0x564af90fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af90fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b8860 con 0x564af90e6a30 2024-09-10T10:45:28.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b8190 con 0x564af90ea4a0 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41b53d0 con 0x564af90eadc0 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 0x564af90fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95ebc2e740 10 client.0 _ll_register_callbacks cb 0x7f95d4004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af90fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af90fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:33028/0 (socket says 172.21.15.143:33028) 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 0x564af90fb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50214/0 (socket says 172.21.15.143:50214) 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 -- 172.21.15.143:0/400790568 learned_addr learned my addr 172.21.15.143:0/400790568 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e37fe640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 -- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 msgr2=0x564af90fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 -- 172.21.15.143:0/400790568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 msgr2=0x564af90fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 0x564af90fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e27fc640 1 -- 172.21.15.143:0/400790568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af916f250 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.824+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 0x564af90fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95e27fc640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af90fb8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f95d809bd00 tx=0x7f95d8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2943545e61a36c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95c2ffd640 1 -- 172.21.15.143:0/400790568 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d809bb50 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc036040 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95c2ffd640 1 -- 172.21.15.143:0/400790568 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95d8097ac0 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95c2ffd640 1 -- 172.21.15.143:0/400790568 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95d80a40c0 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95ebc2e740 10 client.24992 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564af9170540 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.825+0000 7f95c2ffd640 1 -- 172.21.15.143:0/400790568 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95dc036040 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95c2ffd640 1 -- 172.21.15.143:0/400790568 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f95d80add20 con 0x564af90ea4a0 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95ebc2e740 10 client.24992 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:28.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95ebc2e740 10 client.24992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95ebc2e740 20 client.24992 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af90f5680 0x564af91728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95ebc2e740 4 client.24992 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564af9057d60 con 0x564af90f5680 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af90f5680 0x564af91728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95c2ffd640 10 client.24992 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.826+0000 7f95e2ffd640 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af90f5680 0x564af91728e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47aad52fa27a3e57 server_cookie=8ecf86ad79ea14af in_seq=0 out_seq=0 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95c2ffd640 1 -- 172.21.15.143:0/400790568 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564af9057d60 con 0x564af90f5680 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95c2ffd640 10 client.24992 handle_command_reply: tid=0 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 client.24992 shutdown 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 msgr2=0x564af90fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af90fb8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f95d809bd00 tx=0x7f95d8005bd0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 shutdown_connections 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564af90f5680 0x564af91728e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564af90eadc0 0x564af90fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95c2ffd640 0 client.24992 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564af90ea4a0 0x564af90fb8c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 --2- 172.21.15.143:0/400790568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564af90e6a30 0x564af90fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 >> 172.21.15.143:0/400790568 conn(0x564af90f2910 msgr2=0x564af90f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 shutdown_connections 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 1 -- 172.21.15.143:0/400790568 wait complete. 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.827+0000 7f95ebc2e740 20 client.24992 trim_cache size 0 max 16384 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:28.838 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: "42cb4ae6-66ac-49bb-a88e-e2b5f460185b": { 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "42cb4ae6-66ac-49bb-a88e-e2b5f460185b", 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:45:28.839 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95cc0a2c90 msgr2=0x7f95cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95cc0a2c90 0x7f95cc0a5150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fc0 tx=0x7f95d4005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 msgr2=0x7f95e406ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 0x7f95e406ca90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f95d800d950 tx=0x7f95d800de10 comp rx=0 tx=0).stop 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 shutdown_connections 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95cc0a2c90 0x7f95cc0a5150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e4106f00 0x7f95e406cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95e4102780 0x7f95e406ca90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 --2- 172.21.15.143:0/1966532812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e4101dd0 0x7f95e4073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 >> 172.21.15.143:0/1966532812 conn(0x7f95e40fd8b0 msgr2=0x7f95e40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 shutdown_connections 2024-09-10T10:45:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:28.828+0000 7f95e9855640 1 -- 172.21.15.143:0/1966532812 wait complete. 2024-09-10T10:45:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:28 smithi178 ceph-mon[34813]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:28 smithi144 ceph-mon[34230]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:29.652 INFO:tasks.cephfs.filesystem:scrub status for tag:42cb4ae6-66ac-49bb-a88e-e2b5f460185b - {'path': '/', 'tag': '42cb4ae6-66ac-49bb-a88e-e2b5f460185b', 'options': 'recursive,force'} 2024-09-10T10:45:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:29.913 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:29 smithi143 ceph-mon[31057]: pgmap v364: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T10:45:29.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:29 smithi178 ceph-mon[34813]: pgmap v364: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T10:45:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:29 smithi144 ceph-mon[34230]: pgmap v364: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T10:45:30.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 -- 172.21.15.143:0/193405787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 msgr2=0x7fef041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 --2- 172.21.15.143:0/193405787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef041029b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feef4009900 tx=0x7feef402a480 comp rx=0 tx=0).stop 2024-09-10T10:45:30.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 -- 172.21.15.143:0/193405787 shutdown_connections 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 --2- 172.21.15.143:0/193405787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 0x7fef0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 --2- 172.21.15.143:0/193405787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef041029b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 --2- 172.21.15.143:0/193405787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef04100e20 0x7fef04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.190+0000 7fef0bec2640 1 -- 172.21.15.143:0/193405787 >> 172.21.15.143:0/193405787 conn(0x7fef040fc7d0 msgr2=0x7fef040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.191+0000 7fef0bec2640 1 -- 172.21.15.143:0/193405787 shutdown_connections 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.191+0000 7fef0bec2640 1 -- 172.21.15.143:0/193405787 wait complete. 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.191+0000 7fef0bec2640 1 Processor -- start 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.191+0000 7fef0bec2640 1 -- start start 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0bec2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef04100e20 0x7fef04198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0bec2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef04198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0bec2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 0x7fef0419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0bec2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fef04110e50 con 0x7fef04102570 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0bec2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef04110cd0 con 0x7fef04102ef0 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0bec2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef04110fd0 con 0x7fef04100e20 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef04198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef04198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50238/0 (socket says 172.21.15.143:50238) 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 -- 172.21.15.143:0/2673431705 learned_addr learned my addr 172.21.15.143:0/2673431705 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef09c5e640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef04100e20 0x7fef04198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0a45f640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 0x7fef0419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 -- 172.21.15.143:0/2673431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef04100e20 msgr2=0x7fef04198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef04100e20 0x7fef04198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 -- 172.21.15.143:0/2673431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 msgr2=0x7fef0419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 0x7fef0419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 -- 172.21.15.143:0/2673431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0419d4c0 con 0x7fef04102570 2024-09-10T10:45:30.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0a45f640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 0x7fef0419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7fef0945d640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef04198a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feef4006010 tx=0x7feef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.192+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef403f030 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.193+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0419d750 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.193+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef4003cb0 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.193+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4035070 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.193+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a5090 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.194+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04101200 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.195+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7feef4042030 con 0x7fef04102570 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.195+0000 7feefaffd640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feef00a2c10 0x7feef00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.196+0000 7fef09c5e640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feef00a2c10 0x7feef00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.196+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feef40efda0 con 0x7fef04102570 2024-09-10T10:45:30.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.196+0000 7fef09c5e640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feef00a2c10 0x7feef00a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef00009990 tx=0x7fef00009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:30.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.201+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feef40bc840 con 0x7fef04102570 2024-09-10T10:45:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.372+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef04062630 con 0x7feef00a2c10 2024-09-10T10:45:30.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.380+0000 7feefaffd640 1 -- 172.21.15.143:0/2673431705 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340562 (secure 0 0 0) 0x7fef04062630 con 0x7feef00a2c10 2024-09-10T10:45:30.387 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:30.388 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feef00a2c10 msgr2=0x7feef00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feef00a2c10 0x7feef00a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef00009990 tx=0x7fef00009110 comp rx=0 tx=0).stop 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 msgr2=0x7fef04198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef04198a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feef4006010 tx=0x7feef40042f0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 shutdown_connections 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feef00a2c10 0x7feef00a50d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04102ef0 0x7fef0419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04102570 0x7fef04198a50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 --2- 172.21.15.143:0/2673431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef04100e20 0x7fef04198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 >> 172.21.15.143:0/2673431705 conn(0x7fef040fc7d0 msgr2=0x7fef0406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 shutdown_connections 2024-09-10T10:45:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:30.384+0000 7fef0bec2640 1 -- 172.21.15.143:0/2673431705 wait complete. 2024-09-10T10:45:30.860 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:30 smithi143 ceph-mon[31057]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:30 smithi178 ceph-mon[34813]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:30.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:30 smithi144 ceph-mon[34230]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:31.213 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-10T10:45:29.471541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670512,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1308,"num_read_kb":787,"num_write":463,"num_write_kb":3112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375788,"kb_used_data":14060,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566036,"statfs":{"total":1151940427776,"available":1151555620864,"internally_reserved":0,"allocated":14397440,"data_stored":7980590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8115,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":63,"num_write":9,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003391"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.718019+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:45:25.718019+0000","last_peered":"2024-09-10T10:45:25.718019+0000","last_clean":"2024-09-10T10:45:25.718019+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:45:25.718019+0000","last_undegraded":"2024-09-10T10:45:25.718019+0000","last_fullsized":"2024-09-10T10:45:25.718019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441227+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:59.441227+0000","last_peered":"2024-09-10T10:44:59.441227+0000","last_clean":"2024-09-10T10:44:59.441227+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:59.441227+0000","last_undegraded":"2024-09-10T10:44:59.441227+0000","last_fullsized":"2024-09-10T10:44:59.441227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500815+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500815+0000","last_peered":"2024-09-10T10:44:00.500815+0000","last_clean":"2024-09-10T10:44:00.500815+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500815+0000","last_undegraded":"2024-09-10T10:44:00.500815+0000","last_fullsized":"2024-09-10T10:44:00.500815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505312+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505312+0000","last_peered":"2024-09-10T10:44:00.505312+0000","last_clean":"2024-09-10T10:44:00.505312+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505312+0000","last_undegraded":"2024-09-10T10:44:00.505312+0000","last_fullsized":"2024-09-10T10:44:00.505312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440037+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:59.440037+0000","last_peered":"2024-09-10T10:44:59.440037+0000","last_clean":"2024-09-10T10:44:59.440037+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:59.440037+0000","last_undegraded":"2024-09-10T10:44:59.440037+0000","last_fullsized":"2024-09-10T10:44:59.440037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503448+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503448+0000","last_peered":"2024-09-10T10:44:00.503448+0000","last_clean":"2024-09-10T10:44:00.503448+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503448+0000","last_undegraded":"2024-09-10T10:44:00.503448+0000","last_fullsized":"2024-09-10T10:44:00.503448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.560860+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:45:18.560860+0000","last_peered":"2024-09-10T10:45:18.560860+0000","last_clean":"2024-09-10T10:45:18.560860+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:45:18.560860+0000","last_undegraded":"2024-09-10T10:45:18.560860+0000","last_fullsized":"2024-09-10T10:45:18.560860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442992+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:59.442992+0000","last_peered":"2024-09-10T10:44:59.442992+0000","last_clean":"2024-09-10T10:44:59.442992+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:59.442992+0000","last_undegraded":"2024-09-10T10:44:59.442992+0000","last_fullsized":"2024-09-10T10:44:59.442992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505985+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505985+0000","last_peered":"2024-09-10T10:44:00.505985+0000","last_clean":"2024-09-10T10:44:00.505985+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505985+0000","last_undegraded":"2024-09-10T10:44:00.505985+0000","last_fullsized":"2024-09-10T10:44:00.505985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442911+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:59.442911+0000","last_peered":"2024-09-10T10:44:59.442911+0000","last_clean":"2024-09-10T10:44:59.442911+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:59.442911+0000","last_undegraded":"2024-09-10T10:44:59.442911+0000","last_fullsized":"2024-09-10T10:44:59.442911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442546+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:59.442546+0000","last_peered":"2024-09-10T10:44:59.442546+0000","last_clean":"2024-09-10T10:44:59.442546+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:59.442546+0000","last_undegraded":"2024-09-10T10:44:59.442546+0000","last_fullsized":"2024-09-10T10:44:59.442546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503983+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503983+0000","last_peered":"2024-09-10T10:44:00.503983+0000","last_clean":"2024-09-10T10:44:00.503983+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503983+0000","last_undegraded":"2024-09-10T10:44:00.503983+0000","last_fullsized":"2024-09-10T10:44:00.503983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435612+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:59.435612+0000","last_peered":"2024-09-10T10:44:59.435612+0000","last_clean":"2024-09-10T10:44:59.435612+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:59.435612+0000","last_undegraded":"2024-09-10T10:44:59.435612+0000","last_fullsized":"2024-09-10T10:44:59.435612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.106863+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:45:01.106863+0000","last_peered":"2024-09-10T10:45:01.106863+0000","last_clean":"2024-09-10T10:45:01.106863+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:45:01.106863+0000","last_undegraded":"2024-09-10T10:45:01.106863+0000","last_fullsized":"2024-09-10T10:45:01.106863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.524134+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.524134+0000","last_peered":"2024-09-10T10:44:00.524134+0000","last_clean":"2024-09-10T10:44:00.524134+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.524134+0000","last_undegraded":"2024-09-10T10:44:00.524134+0000","last_fullsized":"2024-09-10T10:44:00.524134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437632+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:59.437632+0000","last_peered":"2024-09-10T10:44:59.437632+0000","last_clean":"2024-09-10T10:44:59.437632+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:59.437632+0000","last_undegraded":"2024-09-10T10:44:59.437632+0000","last_fullsized":"2024-09-10T10:44:59.437632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504947+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504947+0000","last_peered":"2024-09-10T10:44:00.504947+0000","last_clean":"2024-09-10T10:44:00.504947+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504947+0000","last_undegraded":"2024-09-10T10:44:00.504947+0000","last_fullsized":"2024-09-10T10:44:00.504947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500343+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500343+0000","last_peered":"2024-09-10T10:44:00.500343+0000","last_clean":"2024-09-10T10:44:00.500343+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500343+0000","last_undegraded":"2024-09-10T10:44:00.500343+0000","last_fullsized":"2024-09-10T10:44:00.500343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.422791+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:59.422791+0000","last_peered":"2024-09-10T10:44:59.422791+0000","last_clean":"2024-09-10T10:44:59.422791+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:59.422791+0000","last_undegraded":"2024-09-10T10:44:59.422791+0000","last_fullsized":"2024-09-10T10:44:59.422791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510061+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510061+0000","last_peered":"2024-09-10T10:44:00.510061+0000","last_clean":"2024-09-10T10:44:00.510061+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510061+0000","last_undegraded":"2024-09-10T10:44:00.510061+0000","last_fullsized":"2024-09-10T10:44:00.510061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504098+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504098+0000","last_peered":"2024-09-10T10:44:00.504098+0000","last_clean":"2024-09-10T10:44:00.504098+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504098+0000","last_undegraded":"2024-09-10T10:44:00.504098+0000","last_fullsized":"2024-09-10T10:44:00.504098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441460+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:59.441460+0000","last_peered":"2024-09-10T10:44:59.441460+0000","last_clean":"2024-09-10T10:44:59.441460+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:59.441460+0000","last_undegraded":"2024-09-10T10:44:59.441460+0000","last_fullsized":"2024-09-10T10:44:59.441460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504342+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504342+0000","last_peered":"2024-09-10T10:44:00.504342+0000","last_clean":"2024-09-10T10:44:00.504342+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504342+0000","last_undegraded":"2024-09-10T10:44:00.504342+0000","last_fullsized":"2024-09-10T10:44:00.504342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500934+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500934+0000","last_peered":"2024-09-10T10:44:00.500934+0000","last_clean":"2024-09-10T10:44:00.500934+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500934+0000","last_undegraded":"2024-09-10T10:44:00.500934+0000","last_fullsized":"2024-09-10T10:44:00.500934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440153+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:59.440153+0000","last_peered":"2024-09-10T10:44:59.440153+0000","last_clean":"2024-09-10T10:44:59.440153+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:59.440153+0000","last_undegraded":"2024-09-10T10:44:59.440153+0000","last_fullsized":"2024-09-10T10:44:59.440153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'18","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.270359+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:45:15.270359+0000","last_peered":"2024-09-10T10:45:15.270359+0000","last_clean":"2024-09-10T10:45:15.270359+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:45:15.270359+0000","last_undegraded":"2024-09-10T10:45:15.270359+0000","last_fullsized":"2024-09-10T10:45:15.270359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504339+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504339+0000","last_peered":"2024-09-10T10:44:00.504339+0000","last_clean":"2024-09-10T10:44:00.504339+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504339+0000","last_undegraded":"2024-09-10T10:44:00.504339+0000","last_fullsized":"2024-09-10T10:44:00.504339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443706+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:59.443706+0000","last_peered":"2024-09-10T10:44:59.443706+0000","last_clean":"2024-09-10T10:44:59.443706+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:59.443706+0000","last_undegraded":"2024-09-10T10:44:59.443706+0000","last_fullsized":"2024-09-10T10:44:59.443706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506338+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506338+0000","last_peered":"2024-09-10T10:44:00.506338+0000","last_clean":"2024-09-10T10:44:00.506338+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506338+0000","last_undegraded":"2024-09-10T10:44:00.506338+0000","last_fullsized":"2024-09-10T10:44:00.506338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503217+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503217+0000","last_peered":"2024-09-10T10:44:00.503217+0000","last_clean":"2024-09-10T10:44:00.503217+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503217+0000","last_undegraded":"2024-09-10T10:44:00.503217+0000","last_fullsized":"2024-09-10T10:44:00.503217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.424958+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:59.424958+0000","last_peered":"2024-09-10T10:44:59.424958+0000","last_clean":"2024-09-10T10:44:59.424958+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:59.424958+0000","last_undegraded":"2024-09-10T10:44:59.424958+0000","last_fullsized":"2024-09-10T10:44:59.424958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501376+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501376+0000","last_peered":"2024-09-10T10:44:00.501376+0000","last_clean":"2024-09-10T10:44:00.501376+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501376+0000","last_undegraded":"2024-09-10T10:44:00.501376+0000","last_fullsized":"2024-09-10T10:44:00.501376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505353+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505353+0000","last_peered":"2024-09-10T10:44:00.505353+0000","last_clean":"2024-09-10T10:44:00.505353+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505353+0000","last_undegraded":"2024-09-10T10:44:00.505353+0000","last_fullsized":"2024-09-10T10:44:00.505353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438858+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:59.438858+0000","last_peered":"2024-09-10T10:44:59.438858+0000","last_clean":"2024-09-10T10:44:59.438858+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:59.438858+0000","last_undegraded":"2024-09-10T10:44:59.438858+0000","last_fullsized":"2024-09-10T10:44:59.438858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:23.716080+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:45:23.716080+0000","last_peered":"2024-09-10T10:45:23.716080+0000","last_clean":"2024-09-10T10:45:23.716080+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:45:23.716080+0000","last_undegraded":"2024-09-10T10:45:23.716080+0000","last_fullsized":"2024-09-10T10:45:23.716080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502631+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502631+0000","last_peered":"2024-09-10T10:44:00.502631+0000","last_clean":"2024-09-10T10:44:00.502631+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502631+0000","last_undegraded":"2024-09-10T10:44:00.502631+0000","last_fullsized":"2024-09-10T10:44:00.502631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439106+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:59.439106+0000","last_peered":"2024-09-10T10:44:59.439106+0000","last_clean":"2024-09-10T10:44:59.439106+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:59.439106+0000","last_undegraded":"2024-09-10T10:44:59.439106+0000","last_fullsized":"2024-09-10T10:44:59.439106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:22.719923+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:45:22.719923+0000","last_peered":"2024-09-10T10:45:22.719923+0000","last_clean":"2024-09-10T10:45:22.719923+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:45:22.719923+0000","last_undegraded":"2024-09-10T10:45:22.719923+0000","last_fullsized":"2024-09-10T10:45:22.719923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502777+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502777+0000","last_peered":"2024-09-10T10:44:00.502777+0000","last_clean":"2024-09-10T10:44:00.502777+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502777+0000","last_undegraded":"2024-09-10T10:44:00.502777+0000","last_fullsized":"2024-09-10T10:44:00.502777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444099+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:59.444099+0000","last_peered":"2024-09-10T10:44:59.444099+0000","last_clean":"2024-09-10T10:44:59.444099+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:59.444099+0000","last_undegraded":"2024-09-10T10:44:59.444099+0000","last_fullsized":"2024-09-10T10:44:59.444099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506093+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506093+0000","last_peered":"2024-09-10T10:44:00.506093+0000","last_clean":"2024-09-10T10:44:00.506093+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506093+0000","last_undegraded":"2024-09-10T10:44:00.506093+0000","last_fullsized":"2024-09-10T10:44:00.506093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516092+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.516092+0000","last_peered":"2024-09-10T10:44:00.516092+0000","last_clean":"2024-09-10T10:44:00.516092+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.516092+0000","last_undegraded":"2024-09-10T10:44:00.516092+0000","last_fullsized":"2024-09-10T10:44:00.516092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443062+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:59.443062+0000","last_peered":"2024-09-10T10:44:59.443062+0000","last_clean":"2024-09-10T10:44:59.443062+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:59.443062+0000","last_undegraded":"2024-09-10T10:44:59.443062+0000","last_fullsized":"2024-09-10T10:44:59.443062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503953+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503953+0000","last_peered":"2024-09-10T10:44:00.503953+0000","last_clean":"2024-09-10T10:44:00.503953+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503953+0000","last_undegraded":"2024-09-10T10:44:00.503953+0000","last_fullsized":"2024-09-10T10:44:00.503953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507101+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507101+0000","last_peered":"2024-09-10T10:44:00.507101+0000","last_clean":"2024-09-10T10:44:00.507101+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507101+0000","last_undegraded":"2024-09-10T10:44:00.507101+0000","last_fullsized":"2024-09-10T10:44:00.507101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442450+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:59.442450+0000","last_peered":"2024-09-10T10:44:59.442450+0000","last_clean":"2024-09-10T10:44:59.442450+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:59.442450+0000","last_undegraded":"2024-09-10T10:44:59.442450+0000","last_fullsized":"2024-09-10T10:44:59.442450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:24.770712+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:45:24.770712+0000","last_peered":"2024-09-10T10:45:24.770712+0000","last_clean":"2024-09-10T10:45:24.770712+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:45:24.770712+0000","last_undegraded":"2024-09-10T10:45:24.770712+0000","last_fullsized":"2024-09-10T10:45:24.770712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509662+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509662+0000","last_peered":"2024-09-10T10:44:00.509662+0000","last_clean":"2024-09-10T10:44:00.509662+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509662+0000","last_undegraded":"2024-09-10T10:44:00.509662+0000","last_fullsized":"2024-09-10T10:44:00.509662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442843+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:59.442843+0000","last_peered":"2024-09-10T10:44:59.442843+0000","last_clean":"2024-09-10T10:44:59.442843+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:59.442843+0000","last_undegraded":"2024-09-10T10:44:59.442843+0000","last_fullsized":"2024-09-10T10:44:59.442843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516384+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516384+0000","last_peered":"2024-09-10T10:44:00.516384+0000","last_clean":"2024-09-10T10:44:00.516384+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516384+0000","last_undegraded":"2024-09-10T10:44:00.516384+0000","last_fullsized":"2024-09-10T10:44:00.516384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500645+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500645+0000","last_peered":"2024-09-10T10:44:00.500645+0000","last_clean":"2024-09-10T10:44:00.500645+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500645+0000","last_undegraded":"2024-09-10T10:44:00.500645+0000","last_fullsized":"2024-09-10T10:44:00.500645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440420+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:59.440420+0000","last_peered":"2024-09-10T10:44:59.440420+0000","last_clean":"2024-09-10T10:44:59.440420+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:59.440420+0000","last_undegraded":"2024-09-10T10:44:59.440420+0000","last_fullsized":"2024-09-10T10:44:59.440420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503016+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.503016+0000","last_peered":"2024-09-10T10:44:00.503016+0000","last_clean":"2024-09-10T10:44:00.503016+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.503016+0000","last_undegraded":"2024-09-10T10:44:00.503016+0000","last_fullsized":"2024-09-10T10:44:00.503016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515085+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515085+0000","last_peered":"2024-09-10T10:44:00.515085+0000","last_clean":"2024-09-10T10:44:00.515085+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515085+0000","last_undegraded":"2024-09-10T10:44:00.515085+0000","last_fullsized":"2024-09-10T10:44:00.515085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440308+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:59.440308+0000","last_peered":"2024-09-10T10:44:59.440308+0000","last_clean":"2024-09-10T10:44:59.440308+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:59.440308+0000","last_undegraded":"2024-09-10T10:44:59.440308+0000","last_fullsized":"2024-09-10T10:44:59.440308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:05.823762+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:45:05.823762+0000","last_peered":"2024-09-10T10:45:05.823762+0000","last_clean":"2024-09-10T10:45:05.823762+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:45:05.823762+0000","last_undegraded":"2024-09-10T10:45:05.823762+0000","last_fullsized":"2024-09-10T10:45:05.823762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515302+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515302+0000","last_peered":"2024-09-10T10:44:00.515302+0000","last_clean":"2024-09-10T10:44:00.515302+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515302+0000","last_undegraded":"2024-09-10T10:44:00.515302+0000","last_fullsized":"2024-09-10T10:44:00.515302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442616+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442616+0000","last_peered":"2024-09-10T10:44:59.442616+0000","last_clean":"2024-09-10T10:44:59.442616+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:59.442616+0000","last_undegraded":"2024-09-10T10:44:59.442616+0000","last_fullsized":"2024-09-10T10:44:59.442616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505428+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505428+0000","last_peered":"2024-09-10T10:44:00.505428+0000","last_clean":"2024-09-10T10:44:00.505428+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505428+0000","last_undegraded":"2024-09-10T10:44:00.505428+0000","last_fullsized":"2024-09-10T10:44:00.505428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507155+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.507155+0000","last_peered":"2024-09-10T10:44:00.507155+0000","last_clean":"2024-09-10T10:44:00.507155+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.507155+0000","last_undegraded":"2024-09-10T10:44:00.507155+0000","last_fullsized":"2024-09-10T10:44:00.507155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441344+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:59.441344+0000","last_peered":"2024-09-10T10:44:59.441344+0000","last_clean":"2024-09-10T10:44:59.441344+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:59.441344+0000","last_undegraded":"2024-09-10T10:44:59.441344+0000","last_fullsized":"2024-09-10T10:44:59.441344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:26.725721+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:45:26.725721+0000","last_peered":"2024-09-10T10:45:26.725721+0000","last_clean":"2024-09-10T10:45:26.725721+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:45:26.725721+0000","last_undegraded":"2024-09-10T10:45:26.725721+0000","last_fullsized":"2024-09-10T10:45:26.725721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500386+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500386+0000","last_peered":"2024-09-10T10:44:00.500386+0000","last_clean":"2024-09-10T10:44:00.500386+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500386+0000","last_undegraded":"2024-09-10T10:44:00.500386+0000","last_fullsized":"2024-09-10T10:44:00.500386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445872+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:59.445872+0000","last_peered":"2024-09-10T10:44:59.445872+0000","last_clean":"2024-09-10T10:44:59.445872+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:59.445872+0000","last_undegraded":"2024-09-10T10:44:59.445872+0000","last_fullsized":"2024-09-10T10:44:59.445872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515497+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515497+0000","last_peered":"2024-09-10T10:44:00.515497+0000","last_clean":"2024-09-10T10:44:00.515497+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515497+0000","last_undegraded":"2024-09-10T10:44:00.515497+0000","last_fullsized":"2024-09-10T10:44:00.515497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502910+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502910+0000","last_peered":"2024-09-10T10:44:00.502910+0000","last_clean":"2024-09-10T10:44:00.502910+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502910+0000","last_undegraded":"2024-09-10T10:44:00.502910+0000","last_fullsized":"2024-09-10T10:44:00.502910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435638+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:59.435638+0000","last_peered":"2024-09-10T10:44:59.435638+0000","last_clean":"2024-09-10T10:44:59.435638+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:59.435638+0000","last_undegraded":"2024-09-10T10:44:59.435638+0000","last_fullsized":"2024-09-10T10:44:59.435638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516482+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516482+0000","last_peered":"2024-09-10T10:44:00.516482+0000","last_clean":"2024-09-10T10:44:00.516482+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516482+0000","last_undegraded":"2024-09-10T10:44:00.516482+0000","last_fullsized":"2024-09-10T10:44:00.516482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":336,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.639756+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:45:25.639756+0000","last_peered":"2024-09-10T10:45:25.639756+0000","last_clean":"2024-09-10T10:45:25.639756+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:45:25.639756+0000","last_undegraded":"2024-09-10T10:45:25.639756+0000","last_fullsized":"2024-09-10T10:45:25.639756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70047,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":186,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500133+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500133+0000","last_peered":"2024-09-10T10:44:00.500133+0000","last_clean":"2024-09-10T10:44:00.500133+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500133+0000","last_undegraded":"2024-09-10T10:44:00.500133+0000","last_fullsized":"2024-09-10T10:44:00.500133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516289+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516289+0000","last_peered":"2024-09-10T10:44:00.516289+0000","last_clean":"2024-09-10T10:44:00.516289+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516289+0000","last_undegraded":"2024-09-10T10:44:00.516289+0000","last_fullsized":"2024-09-10T10:44:00.516289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503829+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503829+0000","last_peered":"2024-09-10T10:44:00.503829+0000","last_clean":"2024-09-10T10:44:00.503829+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503829+0000","last_undegraded":"2024-09-10T10:44:00.503829+0000","last_fullsized":"2024-09-10T10:44:00.503829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506656+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506656+0000","last_peered":"2024-09-10T10:44:00.506656+0000","last_clean":"2024-09-10T10:44:00.506656+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506656+0000","last_undegraded":"2024-09-10T10:44:00.506656+0000","last_fullsized":"2024-09-10T10:44:00.506656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499917+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499917+0000","last_peered":"2024-09-10T10:44:00.499917+0000","last_clean":"2024-09-10T10:44:00.499917+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499917+0000","last_undegraded":"2024-09-10T10:44:00.499917+0000","last_fullsized":"2024-09-10T10:44:00.499917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442320+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:59.442320+0000","last_peered":"2024-09-10T10:44:59.442320+0000","last_clean":"2024-09-10T10:44:59.442320+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:59.442320+0000","last_undegraded":"2024-09-10T10:44:59.442320+0000","last_fullsized":"2024-09-10T10:44:59.442320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":270,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.721017+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:45:25.721017+0000","last_peered":"2024-09-10T10:45:25.721017+0000","last_clean":"2024-09-10T10:45:25.721017+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:45:25.721017+0000","last_undegraded":"2024-09-10T10:45:25.721017+0000","last_fullsized":"2024-09-10T10:45:25.721017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504776+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504776+0000","last_peered":"2024-09-10T10:44:00.504776+0000","last_clean":"2024-09-10T10:44:00.504776+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504776+0000","last_undegraded":"2024-09-10T10:44:00.504776+0000","last_fullsized":"2024-09-10T10:44:00.504776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500929+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500929+0000","last_peered":"2024-09-10T10:44:00.500929+0000","last_clean":"2024-09-10T10:44:00.500929+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500929+0000","last_undegraded":"2024-09-10T10:44:00.500929+0000","last_fullsized":"2024-09-10T10:44:00.500929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.433074+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:59.433074+0000","last_peered":"2024-09-10T10:44:59.433074+0000","last_clean":"2024-09-10T10:44:59.433074+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:59.433074+0000","last_undegraded":"2024-09-10T10:44:59.433074+0000","last_fullsized":"2024-09-10T10:44:59.433074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515598+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515598+0000","last_peered":"2024-09-10T10:44:00.515598+0000","last_clean":"2024-09-10T10:44:00.515598+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515598+0000","last_undegraded":"2024-09-10T10:44:00.515598+0000","last_fullsized":"2024-09-10T10:44:00.515598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509926+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509926+0000","last_peered":"2024-09-10T10:44:00.509926+0000","last_clean":"2024-09-10T10:44:00.509926+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509926+0000","last_undegraded":"2024-09-10T10:44:00.509926+0000","last_fullsized":"2024-09-10T10:44:00.509926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.434606+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:59.434606+0000","last_peered":"2024-09-10T10:44:59.434606+0000","last_clean":"2024-09-10T10:44:59.434606+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:59.434606+0000","last_undegraded":"2024-09-10T10:44:59.434606+0000","last_fullsized":"2024-09-10T10:44:59.434606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502516+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502516+0000","last_peered":"2024-09-10T10:44:00.502516+0000","last_clean":"2024-09-10T10:44:00.502516+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502516+0000","last_undegraded":"2024-09-10T10:44:00.502516+0000","last_fullsized":"2024-09-10T10:44:00.502516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505395+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505395+0000","last_peered":"2024-09-10T10:44:00.505395+0000","last_clean":"2024-09-10T10:44:00.505395+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505395+0000","last_undegraded":"2024-09-10T10:44:00.505395+0000","last_fullsized":"2024-09-10T10:44:00.505395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438633+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:59.438633+0000","last_peered":"2024-09-10T10:44:59.438633+0000","last_clean":"2024-09-10T10:44:59.438633+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:59.438633+0000","last_undegraded":"2024-09-10T10:44:59.438633+0000","last_fullsized":"2024-09-10T10:44:59.438633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501301+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501301+0000","last_peered":"2024-09-10T10:44:00.501301+0000","last_clean":"2024-09-10T10:44:00.501301+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501301+0000","last_undegraded":"2024-09-10T10:44:00.501301+0000","last_fullsized":"2024-09-10T10:44:00.501301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.721739+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:45:25.721739+0000","last_peered":"2024-09-10T10:45:25.721739+0000","last_clean":"2024-09-10T10:45:25.721739+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:45:25.721739+0000","last_undegraded":"2024-09-10T10:45:25.721739+0000","last_fullsized":"2024-09-10T10:45:25.721739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439433+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:59.439433+0000","last_peered":"2024-09-10T10:44:59.439433+0000","last_clean":"2024-09-10T10:44:59.439433+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:59.439433+0000","last_undegraded":"2024-09-10T10:44:59.439433+0000","last_fullsized":"2024-09-10T10:44:59.439433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501304+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501304+0000","last_peered":"2024-09-10T10:44:00.501304+0000","last_clean":"2024-09-10T10:44:00.501304+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501304+0000","last_undegraded":"2024-09-10T10:44:00.501304+0000","last_fullsized":"2024-09-10T10:44:00.501304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440974+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:59.440974+0000","last_peered":"2024-09-10T10:44:59.440974+0000","last_clean":"2024-09-10T10:44:59.440974+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:59.440974+0000","last_undegraded":"2024-09-10T10:44:59.440974+0000","last_fullsized":"2024-09-10T10:44:59.440974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441221+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:59.441221+0000","last_peered":"2024-09-10T10:44:59.441221+0000","last_clean":"2024-09-10T10:44:59.441221+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:59.441221+0000","last_undegraded":"2024-09-10T10:44:59.441221+0000","last_fullsized":"2024-09-10T10:44:59.441221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440703+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:59.440703+0000","last_peered":"2024-09-10T10:44:59.440703+0000","last_clean":"2024-09-10T10:44:59.440703+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:59.440703+0000","last_undegraded":"2024-09-10T10:44:59.440703+0000","last_fullsized":"2024-09-10T10:44:59.440703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444329+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:59.444329+0000","last_peered":"2024-09-10T10:44:59.444329+0000","last_clean":"2024-09-10T10:44:59.444329+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:59.444329+0000","last_undegraded":"2024-09-10T10:44:59.444329+0000","last_fullsized":"2024-09-10T10:44:59.444329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438450+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:59.438450+0000","last_peered":"2024-09-10T10:44:59.438450+0000","last_clean":"2024-09-10T10:44:59.438450+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:59.438450+0000","last_undegraded":"2024-09-10T10:44:59.438450+0000","last_fullsized":"2024-09-10T10:44:59.438450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441275+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:59.441275+0000","last_peered":"2024-09-10T10:44:59.441275+0000","last_clean":"2024-09-10T10:44:59.441275+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:59.441275+0000","last_undegraded":"2024-09-10T10:44:59.441275+0000","last_fullsized":"2024-09-10T10:44:59.441275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437164+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:59.437164+0000","last_peered":"2024-09-10T10:44:59.437164+0000","last_clean":"2024-09-10T10:44:59.437164+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:59.437164+0000","last_undegraded":"2024-09-10T10:44:59.437164+0000","last_fullsized":"2024-09-10T10:44:59.437164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441514+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:59.441514+0000","last_peered":"2024-09-10T10:44:59.441514+0000","last_clean":"2024-09-10T10:44:59.441514+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:59.441514+0000","last_undegraded":"2024-09-10T10:44:59.441514+0000","last_fullsized":"2024-09-10T10:44:59.441514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442360+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:59.442360+0000","last_peered":"2024-09-10T10:44:59.442360+0000","last_clean":"2024-09-10T10:44:59.442360+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:59.442360+0000","last_undegraded":"2024-09-10T10:44:59.442360+0000","last_fullsized":"2024-09-10T10:44:59.442360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442124+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:59.442124+0000","last_peered":"2024-09-10T10:44:59.442124+0000","last_clean":"2024-09-10T10:44:59.442124+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:59.442124+0000","last_undegraded":"2024-09-10T10:44:59.442124+0000","last_fullsized":"2024-09-10T10:44:59.442124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442058+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:59.442058+0000","last_peered":"2024-09-10T10:44:59.442058+0000","last_clean":"2024-09-10T10:44:59.442058+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:59.442058+0000","last_undegraded":"2024-09-10T10:44:59.442058+0000","last_fullsized":"2024-09-10T10:44:59.442058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438851+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:59.438851+0000","last_peered":"2024-09-10T10:44:59.438851+0000","last_clean":"2024-09-10T10:44:59.438851+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:59.438851+0000","last_undegraded":"2024-09-10T10:44:59.438851+0000","last_fullsized":"2024-09-10T10:44:59.438851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429557+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:59.429557+0000","last_peered":"2024-09-10T10:44:59.429557+0000","last_clean":"2024-09-10T10:44:59.429557+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:59.429557+0000","last_undegraded":"2024-09-10T10:44:59.429557+0000","last_fullsized":"2024-09-10T10:44:59.429557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440551+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:59.440551+0000","last_peered":"2024-09-10T10:44:59.440551+0000","last_clean":"2024-09-10T10:44:59.440551+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:59.440551+0000","last_undegraded":"2024-09-10T10:44:59.440551+0000","last_fullsized":"2024-09-10T10:44:59.440551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437183+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:59.437183+0000","last_peered":"2024-09-10T10:44:59.437183+0000","last_clean":"2024-09-10T10:44:59.437183+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:59.437183+0000","last_undegraded":"2024-09-10T10:44:59.437183+0000","last_fullsized":"2024-09-10T10:44:59.437183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439780+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:59.439780+0000","last_peered":"2024-09-10T10:44:59.439780+0000","last_clean":"2024-09-10T10:44:59.439780+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:59.439780+0000","last_undegraded":"2024-09-10T10:44:59.439780+0000","last_fullsized":"2024-09-10T10:44:59.439780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441856+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:59.441856+0000","last_peered":"2024-09-10T10:44:59.441856+0000","last_clean":"2024-09-10T10:44:59.441856+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:59.441856+0000","last_undegraded":"2024-09-10T10:44:59.441856+0000","last_fullsized":"2024-09-10T10:44:59.441856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437328+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:59.437328+0000","last_peered":"2024-09-10T10:44:59.437328+0000","last_clean":"2024-09-10T10:44:59.437328+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:59.437328+0000","last_undegraded":"2024-09-10T10:44:59.437328+0000","last_fullsized":"2024-09-10T10:44:59.437328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438618+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:59.438618+0000","last_peered":"2024-09-10T10:44:59.438618+0000","last_clean":"2024-09-10T10:44:59.438618+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:59.438618+0000","last_undegraded":"2024-09-10T10:44:59.438618+0000","last_fullsized":"2024-09-10T10:44:59.438618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444056+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:59.444056+0000","last_peered":"2024-09-10T10:44:59.444056+0000","last_clean":"2024-09-10T10:44:59.444056+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:59.444056+0000","last_undegraded":"2024-09-10T10:44:59.444056+0000","last_fullsized":"2024-09-10T10:44:59.444056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442679+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:59.442679+0000","last_peered":"2024-09-10T10:44:59.442679+0000","last_clean":"2024-09-10T10:44:59.442679+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:59.442679+0000","last_undegraded":"2024-09-10T10:44:59.442679+0000","last_fullsized":"2024-09-10T10:44:59.442679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.433231+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:59.433231+0000","last_peered":"2024-09-10T10:44:59.433231+0000","last_clean":"2024-09-10T10:44:59.433231+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:59.433231+0000","last_undegraded":"2024-09-10T10:44:59.433231+0000","last_fullsized":"2024-09-10T10:44:59.433231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442732+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442732+0000","last_peered":"2024-09-10T10:44:59.442732+0000","last_clean":"2024-09-10T10:44:59.442732+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:59.442732+0000","last_undegraded":"2024-09-10T10:44:59.442732+0000","last_fullsized":"2024-09-10T10:44:59.442732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440910+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:59.440910+0000","last_peered":"2024-09-10T10:44:59.440910+0000","last_clean":"2024-09-10T10:44:59.440910+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:59.440910+0000","last_undegraded":"2024-09-10T10:44:59.440910+0000","last_fullsized":"2024-09-10T10:44:59.440910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445105+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:59.445105+0000","last_peered":"2024-09-10T10:44:59.445105+0000","last_clean":"2024-09-10T10:44:59.445105+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:59.445105+0000","last_undegraded":"2024-09-10T10:44:59.445105+0000","last_fullsized":"2024-09-10T10:44:59.445105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428701+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:59.428701+0000","last_peered":"2024-09-10T10:44:59.428701+0000","last_clean":"2024-09-10T10:44:59.428701+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:59.428701+0000","last_undegraded":"2024-09-10T10:44:59.428701+0000","last_fullsized":"2024-09-10T10:44:59.428701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429050+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:59.429050+0000","last_peered":"2024-09-10T10:44:59.429050+0000","last_clean":"2024-09-10T10:44:59.429050+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:59.429050+0000","last_undegraded":"2024-09-10T10:44:59.429050+0000","last_fullsized":"2024-09-10T10:44:59.429050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441024+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:59.441024+0000","last_peered":"2024-09-10T10:44:59.441024+0000","last_clean":"2024-09-10T10:44:59.441024+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:59.441024+0000","last_undegraded":"2024-09-10T10:44:59.441024+0000","last_fullsized":"2024-09-10T10:44:59.441024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441764+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:59.441764+0000","last_peered":"2024-09-10T10:44:59.441764+0000","last_clean":"2024-09-10T10:44:59.441764+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:59.441764+0000","last_undegraded":"2024-09-10T10:44:59.441764+0000","last_fullsized":"2024-09-10T10:44:59.441764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439409+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:59.439409+0000","last_peered":"2024-09-10T10:44:59.439409+0000","last_clean":"2024-09-10T10:44:59.439409+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:59.439409+0000","last_undegraded":"2024-09-10T10:44:59.439409+0000","last_fullsized":"2024-09-10T10:44:59.439409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439118+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:59.439118+0000","last_peered":"2024-09-10T10:44:59.439118+0000","last_clean":"2024-09-10T10:44:59.439118+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:59.439118+0000","last_undegraded":"2024-09-10T10:44:59.439118+0000","last_fullsized":"2024-09-10T10:44:59.439118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79910,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1182,"num_read_kb":678,"num_write":280,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":239730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547243,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":501134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776178,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1162025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168636,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397569,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1098863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":496359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790028,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":570621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215190,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1024000,"data_stored":498177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":567436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:31.218 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:45:31.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:45:31.469 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:31.888 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:31 smithi143 ceph-mon[31057]: pgmap v365: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T10:45:31.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:31 smithi178 ceph-mon[34813]: pgmap v365: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T10:45:31.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:31 smithi144 ceph-mon[34230]: pgmap v365: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T10:45:31.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 -- 172.21.15.143:0/4049834012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e810c180 msgr2=0x7f81e810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 --2- 172.21.15.143:0/4049834012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e810c180 0x7f81e810e600 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009ca0 tx=0x7f81e002a950 comp rx=0 tx=0).stop 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 -- 172.21.15.143:0/4049834012 shutdown_connections 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 --2- 172.21.15.143:0/4049834012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e810c180 0x7f81e810e600 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 --2- 172.21.15.143:0/4049834012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e81036c0 0x7f81e810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 --2- 172.21.15.143:0/4049834012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81e8102da0 0x7f81e8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.966+0000 7f81ed040640 1 -- 172.21.15.143:0/4049834012 >> 172.21.15.143:0/4049834012 conn(0x7f81e80fc7a0 msgr2=0x7f81e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.967+0000 7f81ed040640 1 -- 172.21.15.143:0/4049834012 shutdown_connections 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.967+0000 7f81ed040640 1 -- 172.21.15.143:0/4049834012 wait complete. 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.967+0000 7f81ed040640 1 Processor -- start 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 -- start start 2024-09-10T10:45:31.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81e8102da0 0x7f81e8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 0x7f81e8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e810c180 0x7f81e819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8111df0 con 0x7f81e81036c0 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8111c70 con 0x7f81e8102da0 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81ed040640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8111f70 con 0x7f81e810c180 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 0x7f81e8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 0x7f81e8198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50254/0 (socket says 172.21.15.143:50254) 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 -- 172.21.15.143:0/4069888578 learned_addr learned my addr 172.21.15.143:0/4069888578 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81e6d9d640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81e8102da0 0x7f81e8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 -- 172.21.15.143:0/4069888578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e810c180 msgr2=0x7f81e819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e810c180 0x7f81e819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 -- 172.21.15.143:0/4069888578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81e8102da0 msgr2=0x7f81e8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81e8102da0 0x7f81e8198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.968+0000 7f81de59c640 1 -- 172.21.15.143:0/4069888578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e819d2d0 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.969+0000 7f81de59c640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 0x7f81e8198860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c970 tx=0x7f81d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.969+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0007bb0 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.969+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81d0007d50 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.969+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e819d5c0 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.969+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d000f450 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.969+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81a4ea0 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.970+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e8103180 con 0x7f81e81036c0 2024-09-10T10:45:32.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.971+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f81d0004f20 con 0x7f81e81036c0 2024-09-10T10:45:32.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.972+0000 7f81e4d99640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81d40a2c10 0x7f81d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:32.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.972+0000 7f81e6d9d640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81d40a2c10 0x7f81d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:32.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.972+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f81d00cf2d0 con 0x7f81e81036c0 2024-09-10T10:45:32.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.972+0000 7f81e6d9d640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81d40a2c10 0x7f81d40a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005fd0 tx=0x7f81d8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:32.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:31.976+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81d0014030 con 0x7f81e81036c0 2024-09-10T10:45:32.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.138+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 --> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81e8107280 con 0x7f81d40a2c10 2024-09-10T10:45:32.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.146+0000 7f81e4d99640 1 -- 172.21.15.143:0/4069888578 <== mgr.14150 v2:172.21.15.143:6800/1095035473 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340562 (secure 0 0 0) 0x7f81e8107280 con 0x7f81d40a2c10 2024-09-10T10:45:32.152 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:32.153 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81d40a2c10 msgr2=0x7f81d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81d40a2c10 0x7f81d40a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005fd0 tx=0x7f81d8005850 comp rx=0 tx=0).stop 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 msgr2=0x7f81e8198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 0x7f81e8198860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c970 tx=0x7f81d000ce30 comp rx=0 tx=0).stop 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 shutdown_connections 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81d40a2c10 0x7f81d40a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:32.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81e810c180 0x7f81e819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e81036c0 0x7f81e8198860 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 --2- 172.21.15.143:0/4069888578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81e8102da0 0x7f81e8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 >> 172.21.15.143:0/4069888578 conn(0x7f81e80fc7a0 msgr2=0x7f81e80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 shutdown_connections 2024-09-10T10:45:32.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:32.151+0000 7f81ed040640 1 -- 172.21.15.143:0/4069888578 wait complete. 2024-09-10T10:45:32.861 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:32 smithi143 ceph-mon[31057]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:32.891 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-10T10:45:31.472013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670512,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1323,"num_read_kb":792,"num_write":463,"num_write_kb":3112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1242997,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375788,"kb_used_data":14060,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566036,"statfs":{"total":1151940427776,"available":1151555620864,"internally_reserved":0,"allocated":14397440,"data_stored":7980590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7620,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":67,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003151"},"pg_stats":[{"pgid":"2.1f","version":"89'14","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.718019+0000","last_change":"2024-09-10T10:42:51.224020+0000","last_active":"2024-09-10T10:45:25.718019+0000","last_peered":"2024-09-10T10:45:25.718019+0000","last_clean":"2024-09-10T10:45:25.718019+0000","last_became_active":"2024-09-10T10:42:51.223675+0000","last_became_peered":"2024-09-10T10:42:51.223675+0000","last_unstale":"2024-09-10T10:45:25.718019+0000","last_undegraded":"2024-09-10T10:45:25.718019+0000","last_fullsized":"2024-09-10T10:45:25.718019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:19.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441227+0000","last_change":"2024-09-10T10:42:53.240550+0000","last_active":"2024-09-10T10:44:59.441227+0000","last_peered":"2024-09-10T10:44:59.441227+0000","last_clean":"2024-09-10T10:44:59.441227+0000","last_became_active":"2024-09-10T10:42:53.240250+0000","last_became_peered":"2024-09-10T10:42:53.240250+0000","last_unstale":"2024-09-10T10:44:59.441227+0000","last_undegraded":"2024-09-10T10:44:59.441227+0000","last_fullsized":"2024-09-10T10:44:59.441227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:46.225757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500815+0000","last_change":"2024-09-10T10:43:00.286832+0000","last_active":"2024-09-10T10:44:00.500815+0000","last_peered":"2024-09-10T10:44:00.500815+0000","last_clean":"2024-09-10T10:44:00.500815+0000","last_became_active":"2024-09-10T10:43:00.286422+0000","last_became_peered":"2024-09-10T10:43:00.286422+0000","last_unstale":"2024-09-10T10:44:00.500815+0000","last_undegraded":"2024-09-10T10:44:00.500815+0000","last_fullsized":"2024-09-10T10:44:00.500815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:13.464460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505312+0000","last_change":"2024-09-10T10:42:51.400058+0000","last_active":"2024-09-10T10:44:00.505312+0000","last_peered":"2024-09-10T10:44:00.505312+0000","last_clean":"2024-09-10T10:44:00.505312+0000","last_became_active":"2024-09-10T10:42:51.399560+0000","last_became_peered":"2024-09-10T10:42:51.399560+0000","last_unstale":"2024-09-10T10:44:00.505312+0000","last_undegraded":"2024-09-10T10:44:00.505312+0000","last_fullsized":"2024-09-10T10:44:00.505312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:30.028219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440037+0000","last_change":"2024-09-10T10:42:53.247618+0000","last_active":"2024-09-10T10:44:59.440037+0000","last_peered":"2024-09-10T10:44:59.440037+0000","last_clean":"2024-09-10T10:44:59.440037+0000","last_became_active":"2024-09-10T10:42:53.247355+0000","last_became_peered":"2024-09-10T10:42:53.247355+0000","last_unstale":"2024-09-10T10:44:59.440037+0000","last_undegraded":"2024-09-10T10:44:59.440037+0000","last_fullsized":"2024-09-10T10:44:59.440037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:15.091710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503448+0000","last_change":"2024-09-10T10:43:00.294884+0000","last_active":"2024-09-10T10:44:00.503448+0000","last_peered":"2024-09-10T10:44:00.503448+0000","last_clean":"2024-09-10T10:44:00.503448+0000","last_became_active":"2024-09-10T10:43:00.294479+0000","last_became_peered":"2024-09-10T10:43:00.294479+0000","last_unstale":"2024-09-10T10:44:00.503448+0000","last_undegraded":"2024-09-10T10:44:00.503448+0000","last_fullsized":"2024-09-10T10:44:00.503448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:27.932298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:18.560860+0000","last_change":"2024-09-10T10:42:51.229829+0000","last_active":"2024-09-10T10:45:18.560860+0000","last_peered":"2024-09-10T10:45:18.560860+0000","last_clean":"2024-09-10T10:45:18.560860+0000","last_became_active":"2024-09-10T10:42:51.229447+0000","last_became_peered":"2024-09-10T10:42:51.229447+0000","last_unstale":"2024-09-10T10:45:18.560860+0000","last_undegraded":"2024-09-10T10:45:18.560860+0000","last_fullsized":"2024-09-10T10:45:18.560860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:23.482612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442992+0000","last_change":"2024-09-10T10:42:53.242939+0000","last_active":"2024-09-10T10:44:59.442992+0000","last_peered":"2024-09-10T10:44:59.442992+0000","last_clean":"2024-09-10T10:44:59.442992+0000","last_became_active":"2024-09-10T10:42:53.242548+0000","last_became_peered":"2024-09-10T10:42:53.242548+0000","last_unstale":"2024-09-10T10:44:59.442992+0000","last_undegraded":"2024-09-10T10:44:59.442992+0000","last_fullsized":"2024-09-10T10:44:59.442992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.477091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507477+0000","last_change":"2024-09-10T10:44:00.507477+0000","last_active":"2024-09-10T10:44:00.507477+0000","last_peered":"2024-09-10T10:44:00.507477+0000","last_clean":"2024-09-10T10:44:00.507477+0000","last_became_active":"2024-09-10T10:44:00.507012+0000","last_became_peered":"2024-09-10T10:44:00.507012+0000","last_unstale":"2024-09-10T10:44:00.507477+0000","last_undegraded":"2024-09-10T10:44:00.507477+0000","last_fullsized":"2024-09-10T10:44:00.507477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:56.262649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505985+0000","last_change":"2024-09-10T10:42:51.226570+0000","last_active":"2024-09-10T10:44:00.505985+0000","last_peered":"2024-09-10T10:44:00.505985+0000","last_clean":"2024-09-10T10:44:00.505985+0000","last_became_active":"2024-09-10T10:42:51.226151+0000","last_became_peered":"2024-09-10T10:42:51.226151+0000","last_unstale":"2024-09-10T10:44:00.505985+0000","last_undegraded":"2024-09-10T10:44:00.505985+0000","last_fullsized":"2024-09-10T10:44:00.505985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:50.925046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442911+0000","last_change":"2024-09-10T10:42:53.241094+0000","last_active":"2024-09-10T10:44:59.442911+0000","last_peered":"2024-09-10T10:44:59.442911+0000","last_clean":"2024-09-10T10:44:59.442911+0000","last_became_active":"2024-09-10T10:42:53.240764+0000","last_became_peered":"2024-09-10T10:42:53.240764+0000","last_unstale":"2024-09-10T10:44:59.442911+0000","last_undegraded":"2024-09-10T10:44:59.442911+0000","last_fullsized":"2024-09-10T10:44:59.442911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:04.633501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503343+0000","last_change":"2024-09-10T10:43:00.288830+0000","last_active":"2024-09-10T10:44:00.503343+0000","last_peered":"2024-09-10T10:44:00.503343+0000","last_clean":"2024-09-10T10:44:00.503343+0000","last_became_active":"2024-09-10T10:43:00.288405+0000","last_became_peered":"2024-09-10T10:43:00.288405+0000","last_unstale":"2024-09-10T10:44:00.503343+0000","last_undegraded":"2024-09-10T10:44:00.503343+0000","last_fullsized":"2024-09-10T10:44:00.503343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:49.997494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442546+0000","last_change":"2024-09-10T10:42:53.251251+0000","last_active":"2024-09-10T10:44:59.442546+0000","last_peered":"2024-09-10T10:44:59.442546+0000","last_clean":"2024-09-10T10:44:59.442546+0000","last_became_active":"2024-09-10T10:42:53.250952+0000","last_became_peered":"2024-09-10T10:42:53.250952+0000","last_unstale":"2024-09-10T10:44:59.442546+0000","last_undegraded":"2024-09-10T10:44:59.442546+0000","last_fullsized":"2024-09-10T10:44:59.442546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:23.491022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.522738+0000","last_change":"2024-09-10T10:44:00.522406+0000","last_active":"2024-09-10T10:44:00.522738+0000","last_peered":"2024-09-10T10:44:00.522738+0000","last_clean":"2024-09-10T10:44:00.522738+0000","last_became_active":"2024-09-10T10:44:00.508375+0000","last_became_peered":"2024-09-10T10:44:00.508375+0000","last_unstale":"2024-09-10T10:44:00.522738+0000","last_undegraded":"2024-09-10T10:44:00.522738+0000","last_fullsized":"2024-09-10T10:44:00.522738+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:38.313563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503983+0000","last_change":"2024-09-10T10:43:00.293628+0000","last_active":"2024-09-10T10:44:00.503983+0000","last_peered":"2024-09-10T10:44:00.503983+0000","last_clean":"2024-09-10T10:44:00.503983+0000","last_became_active":"2024-09-10T10:43:00.293103+0000","last_became_peered":"2024-09-10T10:43:00.293103+0000","last_unstale":"2024-09-10T10:44:00.503983+0000","last_undegraded":"2024-09-10T10:44:00.503983+0000","last_fullsized":"2024-09-10T10:44:00.503983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.324860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435612+0000","last_change":"2024-09-10T10:42:53.242113+0000","last_active":"2024-09-10T10:44:59.435612+0000","last_peered":"2024-09-10T10:44:59.435612+0000","last_clean":"2024-09-10T10:44:59.435612+0000","last_became_active":"2024-09-10T10:42:53.241818+0000","last_became_peered":"2024-09-10T10:42:53.241818+0000","last_unstale":"2024-09-10T10:44:59.435612+0000","last_undegraded":"2024-09-10T10:44:59.435612+0000","last_fullsized":"2024-09-10T10:44:59.435612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:45.243644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:01.106863+0000","last_change":"2024-09-10T10:42:51.226806+0000","last_active":"2024-09-10T10:45:01.106863+0000","last_peered":"2024-09-10T10:45:01.106863+0000","last_clean":"2024-09-10T10:45:01.106863+0000","last_became_active":"2024-09-10T10:42:51.226438+0000","last_became_peered":"2024-09-10T10:42:51.226438+0000","last_unstale":"2024-09-10T10:45:01.106863+0000","last_undegraded":"2024-09-10T10:45:01.106863+0000","last_fullsized":"2024-09-10T10:45:01.106863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:53.465629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.524134+0000","last_change":"2024-09-10T10:44:00.523722+0000","last_active":"2024-09-10T10:44:00.524134+0000","last_peered":"2024-09-10T10:44:00.524134+0000","last_clean":"2024-09-10T10:44:00.524134+0000","last_became_active":"2024-09-10T10:44:00.511814+0000","last_became_peered":"2024-09-10T10:44:00.511814+0000","last_unstale":"2024-09-10T10:44:00.524134+0000","last_undegraded":"2024-09-10T10:44:00.524134+0000","last_fullsized":"2024-09-10T10:44:00.524134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:22.643202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437632+0000","last_change":"2024-09-10T10:42:53.246778+0000","last_active":"2024-09-10T10:44:59.437632+0000","last_peered":"2024-09-10T10:44:59.437632+0000","last_clean":"2024-09-10T10:44:59.437632+0000","last_became_active":"2024-09-10T10:42:53.246446+0000","last_became_peered":"2024-09-10T10:42:53.246446+0000","last_unstale":"2024-09-10T10:44:59.437632+0000","last_undegraded":"2024-09-10T10:44:59.437632+0000","last_fullsized":"2024-09-10T10:44:59.437632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:55.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504947+0000","last_change":"2024-09-10T10:42:51.224374+0000","last_active":"2024-09-10T10:44:00.504947+0000","last_peered":"2024-09-10T10:44:00.504947+0000","last_clean":"2024-09-10T10:44:00.504947+0000","last_became_active":"2024-09-10T10:42:51.224046+0000","last_became_peered":"2024-09-10T10:42:51.224046+0000","last_unstale":"2024-09-10T10:44:00.504947+0000","last_undegraded":"2024-09-10T10:44:00.504947+0000","last_fullsized":"2024-09-10T10:44:00.504947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:21.460745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500343+0000","last_change":"2024-09-10T10:43:00.294058+0000","last_active":"2024-09-10T10:44:00.500343+0000","last_peered":"2024-09-10T10:44:00.500343+0000","last_clean":"2024-09-10T10:44:00.500343+0000","last_became_active":"2024-09-10T10:43:00.293717+0000","last_became_peered":"2024-09-10T10:43:00.293717+0000","last_unstale":"2024-09-10T10:44:00.500343+0000","last_undegraded":"2024-09-10T10:44:00.500343+0000","last_fullsized":"2024-09-10T10:44:00.500343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:46.447847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.422791+0000","last_change":"2024-09-10T10:42:53.243939+0000","last_active":"2024-09-10T10:44:59.422791+0000","last_peered":"2024-09-10T10:44:59.422791+0000","last_clean":"2024-09-10T10:44:59.422791+0000","last_became_active":"2024-09-10T10:42:53.243528+0000","last_became_peered":"2024-09-10T10:42:53.243528+0000","last_unstale":"2024-09-10T10:44:59.422791+0000","last_undegraded":"2024-09-10T10:44:59.422791+0000","last_fullsized":"2024-09-10T10:44:59.422791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:08.097617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.510061+0000","last_change":"2024-09-10T10:42:51.400419+0000","last_active":"2024-09-10T10:44:00.510061+0000","last_peered":"2024-09-10T10:44:00.510061+0000","last_clean":"2024-09-10T10:44:00.510061+0000","last_became_active":"2024-09-10T10:42:51.400047+0000","last_became_peered":"2024-09-10T10:42:51.400047+0000","last_unstale":"2024-09-10T10:44:00.510061+0000","last_undegraded":"2024-09-10T10:44:00.510061+0000","last_fullsized":"2024-09-10T10:44:00.510061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:30.005690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504098+0000","last_change":"2024-09-10T10:43:00.291089+0000","last_active":"2024-09-10T10:44:00.504098+0000","last_peered":"2024-09-10T10:44:00.504098+0000","last_clean":"2024-09-10T10:44:00.504098+0000","last_became_active":"2024-09-10T10:43:00.290581+0000","last_became_peered":"2024-09-10T10:43:00.290581+0000","last_unstale":"2024-09-10T10:44:00.504098+0000","last_undegraded":"2024-09-10T10:44:00.504098+0000","last_fullsized":"2024-09-10T10:44:00.504098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:06.738961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441460+0000","last_change":"2024-09-10T10:42:53.253199+0000","last_active":"2024-09-10T10:44:59.441460+0000","last_peered":"2024-09-10T10:44:59.441460+0000","last_clean":"2024-09-10T10:44:59.441460+0000","last_became_active":"2024-09-10T10:42:53.252430+0000","last_became_peered":"2024-09-10T10:42:53.252430+0000","last_unstale":"2024-09-10T10:44:59.441460+0000","last_undegraded":"2024-09-10T10:44:59.441460+0000","last_fullsized":"2024-09-10T10:44:59.441460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:16.291818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504342+0000","last_change":"2024-09-10T10:42:51.314602+0000","last_active":"2024-09-10T10:44:00.504342+0000","last_peered":"2024-09-10T10:44:00.504342+0000","last_clean":"2024-09-10T10:44:00.504342+0000","last_became_active":"2024-09-10T10:42:51.314322+0000","last_became_peered":"2024-09-10T10:42:51.314322+0000","last_unstale":"2024-09-10T10:44:00.504342+0000","last_undegraded":"2024-09-10T10:44:00.504342+0000","last_fullsized":"2024-09-10T10:44:00.504342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:11.839261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500934+0000","last_change":"2024-09-10T10:43:00.290195+0000","last_active":"2024-09-10T10:44:00.500934+0000","last_peered":"2024-09-10T10:44:00.500934+0000","last_clean":"2024-09-10T10:44:00.500934+0000","last_became_active":"2024-09-10T10:43:00.289786+0000","last_became_peered":"2024-09-10T10:43:00.289786+0000","last_unstale":"2024-09-10T10:44:00.500934+0000","last_undegraded":"2024-09-10T10:44:00.500934+0000","last_fullsized":"2024-09-10T10:44:00.500934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:04.303662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440153+0000","last_change":"2024-09-10T10:42:53.244803+0000","last_active":"2024-09-10T10:44:59.440153+0000","last_peered":"2024-09-10T10:44:59.440153+0000","last_clean":"2024-09-10T10:44:59.440153+0000","last_became_active":"2024-09-10T10:42:53.243981+0000","last_became_peered":"2024-09-10T10:42:53.243981+0000","last_unstale":"2024-09-10T10:44:59.440153+0000","last_undegraded":"2024-09-10T10:44:59.440153+0000","last_fullsized":"2024-09-10T10:44:59.440153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:41.046064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'18","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:15.270359+0000","last_change":"2024-09-10T10:42:51.223729+0000","last_active":"2024-09-10T10:45:15.270359+0000","last_peered":"2024-09-10T10:45:15.270359+0000","last_clean":"2024-09-10T10:45:15.270359+0000","last_became_active":"2024-09-10T10:42:51.223311+0000","last_became_peered":"2024-09-10T10:42:51.223311+0000","last_unstale":"2024-09-10T10:45:15.270359+0000","last_undegraded":"2024-09-10T10:45:15.270359+0000","last_fullsized":"2024-09-10T10:45:15.270359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:27.302347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504339+0000","last_change":"2024-09-10T10:43:00.289046+0000","last_active":"2024-09-10T10:44:00.504339+0000","last_peered":"2024-09-10T10:44:00.504339+0000","last_clean":"2024-09-10T10:44:00.504339+0000","last_became_active":"2024-09-10T10:43:00.288709+0000","last_became_peered":"2024-09-10T10:43:00.288709+0000","last_unstale":"2024-09-10T10:44:00.504339+0000","last_undegraded":"2024-09-10T10:44:00.504339+0000","last_fullsized":"2024-09-10T10:44:00.504339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:17.912722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443706+0000","last_change":"2024-09-10T10:42:53.244902+0000","last_active":"2024-09-10T10:44:59.443706+0000","last_peered":"2024-09-10T10:44:59.443706+0000","last_clean":"2024-09-10T10:44:59.443706+0000","last_became_active":"2024-09-10T10:42:53.244599+0000","last_became_peered":"2024-09-10T10:42:53.244599+0000","last_unstale":"2024-09-10T10:44:59.443706+0000","last_undegraded":"2024-09-10T10:44:59.443706+0000","last_fullsized":"2024-09-10T10:44:59.443706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:13.216120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506338+0000","last_change":"2024-09-10T10:42:51.400448+0000","last_active":"2024-09-10T10:44:00.506338+0000","last_peered":"2024-09-10T10:44:00.506338+0000","last_clean":"2024-09-10T10:44:00.506338+0000","last_became_active":"2024-09-10T10:42:51.400147+0000","last_became_peered":"2024-09-10T10:42:51.400147+0000","last_unstale":"2024-09-10T10:44:00.506338+0000","last_undegraded":"2024-09-10T10:44:00.506338+0000","last_fullsized":"2024-09-10T10:44:00.506338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:03.852868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503217+0000","last_change":"2024-09-10T10:43:00.287870+0000","last_active":"2024-09-10T10:44:00.503217+0000","last_peered":"2024-09-10T10:44:00.503217+0000","last_clean":"2024-09-10T10:44:00.503217+0000","last_became_active":"2024-09-10T10:43:00.287303+0000","last_became_peered":"2024-09-10T10:43:00.287303+0000","last_unstale":"2024-09-10T10:44:00.503217+0000","last_undegraded":"2024-09-10T10:44:00.503217+0000","last_fullsized":"2024-09-10T10:44:00.503217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:36.131012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.424958+0000","last_change":"2024-09-10T10:42:53.245975+0000","last_active":"2024-09-10T10:44:59.424958+0000","last_peered":"2024-09-10T10:44:59.424958+0000","last_clean":"2024-09-10T10:44:59.424958+0000","last_became_active":"2024-09-10T10:42:53.244998+0000","last_became_peered":"2024-09-10T10:42:53.244998+0000","last_unstale":"2024-09-10T10:44:59.424958+0000","last_undegraded":"2024-09-10T10:44:59.424958+0000","last_fullsized":"2024-09-10T10:44:59.424958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:11.293303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501376+0000","last_change":"2024-09-10T10:42:51.400267+0000","last_active":"2024-09-10T10:44:00.501376+0000","last_peered":"2024-09-10T10:44:00.501376+0000","last_clean":"2024-09-10T10:44:00.501376+0000","last_became_active":"2024-09-10T10:42:51.399946+0000","last_became_peered":"2024-09-10T10:42:51.399946+0000","last_unstale":"2024-09-10T10:44:00.501376+0000","last_undegraded":"2024-09-10T10:44:00.501376+0000","last_fullsized":"2024-09-10T10:44:00.501376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:35.386028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505353+0000","last_change":"2024-09-10T10:43:00.294308+0000","last_active":"2024-09-10T10:44:00.505353+0000","last_peered":"2024-09-10T10:44:00.505353+0000","last_clean":"2024-09-10T10:44:00.505353+0000","last_became_active":"2024-09-10T10:43:00.293817+0000","last_became_peered":"2024-09-10T10:43:00.293817+0000","last_unstale":"2024-09-10T10:44:00.505353+0000","last_undegraded":"2024-09-10T10:44:00.505353+0000","last_fullsized":"2024-09-10T10:44:00.505353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:25.926156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438858+0000","last_change":"2024-09-10T10:42:53.252187+0000","last_active":"2024-09-10T10:44:59.438858+0000","last_peered":"2024-09-10T10:44:59.438858+0000","last_clean":"2024-09-10T10:44:59.438858+0000","last_became_active":"2024-09-10T10:42:53.251835+0000","last_became_peered":"2024-09-10T10:42:53.251835+0000","last_unstale":"2024-09-10T10:44:59.438858+0000","last_undegraded":"2024-09-10T10:44:59.438858+0000","last_fullsized":"2024-09-10T10:44:59.438858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:45.151977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:28.722645+0000","last_change":"2024-09-10T10:42:51.229591+0000","last_active":"2024-09-10T10:45:28.722645+0000","last_peered":"2024-09-10T10:45:28.722645+0000","last_clean":"2024-09-10T10:45:28.722645+0000","last_became_active":"2024-09-10T10:42:51.229150+0000","last_became_peered":"2024-09-10T10:42:51.229150+0000","last_unstale":"2024-09-10T10:45:28.722645+0000","last_undegraded":"2024-09-10T10:45:28.722645+0000","last_fullsized":"2024-09-10T10:45:28.722645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:40.049927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1476,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502631+0000","last_change":"2024-09-10T10:43:00.289233+0000","last_active":"2024-09-10T10:44:00.502631+0000","last_peered":"2024-09-10T10:44:00.502631+0000","last_clean":"2024-09-10T10:44:00.502631+0000","last_became_active":"2024-09-10T10:43:00.288870+0000","last_became_peered":"2024-09-10T10:43:00.288870+0000","last_unstale":"2024-09-10T10:44:00.502631+0000","last_undegraded":"2024-09-10T10:44:00.502631+0000","last_fullsized":"2024-09-10T10:44:00.502631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:22.184638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439106+0000","last_change":"2024-09-10T10:42:53.245348+0000","last_active":"2024-09-10T10:44:59.439106+0000","last_peered":"2024-09-10T10:44:59.439106+0000","last_clean":"2024-09-10T10:44:59.439106+0000","last_became_active":"2024-09-10T10:42:53.244859+0000","last_became_peered":"2024-09-10T10:42:53.244859+0000","last_unstale":"2024-09-10T10:44:59.439106+0000","last_undegraded":"2024-09-10T10:44:59.439106+0000","last_fullsized":"2024-09-10T10:44:59.439106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:36.872413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:27.726496+0000","last_change":"2024-09-10T10:42:51.399912+0000","last_active":"2024-09-10T10:45:27.726496+0000","last_peered":"2024-09-10T10:45:27.726496+0000","last_clean":"2024-09-10T10:45:27.726496+0000","last_became_active":"2024-09-10T10:42:51.399556+0000","last_became_peered":"2024-09-10T10:42:51.399556+0000","last_unstale":"2024-09-10T10:45:27.726496+0000","last_undegraded":"2024-09-10T10:45:27.726496+0000","last_fullsized":"2024-09-10T10:45:27.726496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502777+0000","last_change":"2024-09-10T10:43:00.291654+0000","last_active":"2024-09-10T10:44:00.502777+0000","last_peered":"2024-09-10T10:44:00.502777+0000","last_clean":"2024-09-10T10:44:00.502777+0000","last_became_active":"2024-09-10T10:43:00.291263+0000","last_became_peered":"2024-09-10T10:43:00.291263+0000","last_unstale":"2024-09-10T10:44:00.502777+0000","last_undegraded":"2024-09-10T10:44:00.502777+0000","last_fullsized":"2024-09-10T10:44:00.502777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:21.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444099+0000","last_change":"2024-09-10T10:42:53.249826+0000","last_active":"2024-09-10T10:44:59.444099+0000","last_peered":"2024-09-10T10:44:59.444099+0000","last_clean":"2024-09-10T10:44:59.444099+0000","last_became_active":"2024-09-10T10:42:53.249533+0000","last_became_peered":"2024-09-10T10:42:53.249533+0000","last_unstale":"2024-09-10T10:44:59.444099+0000","last_undegraded":"2024-09-10T10:44:59.444099+0000","last_fullsized":"2024-09-10T10:44:59.444099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:13.672702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506093+0000","last_change":"2024-09-10T10:42:51.227085+0000","last_active":"2024-09-10T10:44:00.506093+0000","last_peered":"2024-09-10T10:44:00.506093+0000","last_clean":"2024-09-10T10:44:00.506093+0000","last_became_active":"2024-09-10T10:42:51.226702+0000","last_became_peered":"2024-09-10T10:42:51.226702+0000","last_unstale":"2024-09-10T10:44:00.506093+0000","last_undegraded":"2024-09-10T10:44:00.506093+0000","last_fullsized":"2024-09-10T10:44:00.506093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:53.074740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516092+0000","last_change":"2024-09-10T10:43:00.292867+0000","last_active":"2024-09-10T10:44:00.516092+0000","last_peered":"2024-09-10T10:44:00.516092+0000","last_clean":"2024-09-10T10:44:00.516092+0000","last_became_active":"2024-09-10T10:43:00.292453+0000","last_became_peered":"2024-09-10T10:43:00.292453+0000","last_unstale":"2024-09-10T10:44:00.516092+0000","last_undegraded":"2024-09-10T10:44:00.516092+0000","last_fullsized":"2024-09-10T10:44:00.516092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:07.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443062+0000","last_change":"2024-09-10T10:44:00.516146+0000","last_active":"2024-09-10T10:44:59.443062+0000","last_peered":"2024-09-10T10:44:59.443062+0000","last_clean":"2024-09-10T10:44:59.443062+0000","last_became_active":"2024-09-10T10:44:00.515823+0000","last_became_peered":"2024-09-10T10:44:00.515823+0000","last_unstale":"2024-09-10T10:44:59.443062+0000","last_undegraded":"2024-09-10T10:44:59.443062+0000","last_fullsized":"2024-09-10T10:44:59.443062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:31.169515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503953+0000","last_change":"2024-09-10T10:42:51.224592+0000","last_active":"2024-09-10T10:44:00.503953+0000","last_peered":"2024-09-10T10:44:00.503953+0000","last_clean":"2024-09-10T10:44:00.503953+0000","last_became_active":"2024-09-10T10:42:51.224205+0000","last_became_peered":"2024-09-10T10:42:51.224205+0000","last_unstale":"2024-09-10T10:44:00.503953+0000","last_undegraded":"2024-09-10T10:44:00.503953+0000","last_fullsized":"2024-09-10T10:44:00.503953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.535058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506278+0000","last_change":"2024-09-10T10:41:37.904214+0000","last_active":"2024-09-10T10:44:00.506278+0000","last_peered":"2024-09-10T10:44:00.506278+0000","last_clean":"2024-09-10T10:44:00.506278+0000","last_became_active":"2024-09-10T10:41:37.903809+0000","last_became_peered":"2024-09-10T10:41:37.903809+0000","last_unstale":"2024-09-10T10:44:00.506278+0000","last_undegraded":"2024-09-10T10:44:00.506278+0000","last_fullsized":"2024-09-10T10:44:00.506278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:38:11.645617+0000","last_clean_scrub_stamp":"2024-09-10T10:38:11.645617+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:57.850921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507101+0000","last_change":"2024-09-10T10:43:00.294419+0000","last_active":"2024-09-10T10:44:00.507101+0000","last_peered":"2024-09-10T10:44:00.507101+0000","last_clean":"2024-09-10T10:44:00.507101+0000","last_became_active":"2024-09-10T10:43:00.293990+0000","last_became_peered":"2024-09-10T10:43:00.293990+0000","last_unstale":"2024-09-10T10:44:00.507101+0000","last_undegraded":"2024-09-10T10:44:00.507101+0000","last_fullsized":"2024-09-10T10:44:00.507101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:22:10.068114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442450+0000","last_change":"2024-09-10T10:42:53.242922+0000","last_active":"2024-09-10T10:44:59.442450+0000","last_peered":"2024-09-10T10:44:59.442450+0000","last_clean":"2024-09-10T10:44:59.442450+0000","last_became_active":"2024-09-10T10:42:53.242612+0000","last_became_peered":"2024-09-10T10:42:53.242612+0000","last_unstale":"2024-09-10T10:44:59.442450+0000","last_undegraded":"2024-09-10T10:44:59.442450+0000","last_fullsized":"2024-09-10T10:44:59.442450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:14.784711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":167,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:29.778065+0000","last_change":"2024-09-10T10:42:51.400658+0000","last_active":"2024-09-10T10:45:29.778065+0000","last_peered":"2024-09-10T10:45:29.778065+0000","last_clean":"2024-09-10T10:45:29.778065+0000","last_became_active":"2024-09-10T10:42:51.400219+0000","last_became_peered":"2024-09-10T10:42:51.400219+0000","last_unstale":"2024-09-10T10:45:29.778065+0000","last_undegraded":"2024-09-10T10:45:29.778065+0000","last_fullsized":"2024-09-10T10:45:29.778065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:19.501549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509662+0000","last_change":"2024-09-10T10:43:00.288489+0000","last_active":"2024-09-10T10:44:00.509662+0000","last_peered":"2024-09-10T10:44:00.509662+0000","last_clean":"2024-09-10T10:44:00.509662+0000","last_became_active":"2024-09-10T10:43:00.288062+0000","last_became_peered":"2024-09-10T10:43:00.288062+0000","last_unstale":"2024-09-10T10:44:00.509662+0000","last_undegraded":"2024-09-10T10:44:00.509662+0000","last_fullsized":"2024-09-10T10:44:00.509662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:32.630437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442843+0000","last_change":"2024-09-10T10:42:53.247210+0000","last_active":"2024-09-10T10:44:59.442843+0000","last_peered":"2024-09-10T10:44:59.442843+0000","last_clean":"2024-09-10T10:44:59.442843+0000","last_became_active":"2024-09-10T10:42:53.246869+0000","last_became_peered":"2024-09-10T10:42:53.246869+0000","last_unstale":"2024-09-10T10:44:59.442843+0000","last_undegraded":"2024-09-10T10:44:59.442843+0000","last_fullsized":"2024-09-10T10:44:59.442843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:18.662705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516384+0000","last_change":"2024-09-10T10:42:51.225300+0000","last_active":"2024-09-10T10:44:00.516384+0000","last_peered":"2024-09-10T10:44:00.516384+0000","last_clean":"2024-09-10T10:44:00.516384+0000","last_became_active":"2024-09-10T10:42:51.224980+0000","last_became_peered":"2024-09-10T10:42:51.224980+0000","last_unstale":"2024-09-10T10:44:00.516384+0000","last_undegraded":"2024-09-10T10:44:00.516384+0000","last_fullsized":"2024-09-10T10:44:00.516384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:35.627371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500645+0000","last_change":"2024-09-10T10:43:00.294190+0000","last_active":"2024-09-10T10:44:00.500645+0000","last_peered":"2024-09-10T10:44:00.500645+0000","last_clean":"2024-09-10T10:44:00.500645+0000","last_became_active":"2024-09-10T10:43:00.293835+0000","last_became_peered":"2024-09-10T10:43:00.293835+0000","last_unstale":"2024-09-10T10:44:00.500645+0000","last_undegraded":"2024-09-10T10:44:00.500645+0000","last_fullsized":"2024-09-10T10:44:00.500645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:28.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440420+0000","last_change":"2024-09-10T10:42:53.249612+0000","last_active":"2024-09-10T10:44:59.440420+0000","last_peered":"2024-09-10T10:44:59.440420+0000","last_clean":"2024-09-10T10:44:59.440420+0000","last_became_active":"2024-09-10T10:42:53.249282+0000","last_became_peered":"2024-09-10T10:42:53.249282+0000","last_unstale":"2024-09-10T10:44:59.440420+0000","last_undegraded":"2024-09-10T10:44:59.440420+0000","last_fullsized":"2024-09-10T10:44:59.440420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:11.017565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503016+0000","last_change":"2024-09-10T10:42:51.221897+0000","last_active":"2024-09-10T10:44:00.503016+0000","last_peered":"2024-09-10T10:44:00.503016+0000","last_clean":"2024-09-10T10:44:00.503016+0000","last_became_active":"2024-09-10T10:42:51.221396+0000","last_became_peered":"2024-09-10T10:42:51.221396+0000","last_unstale":"2024-09-10T10:44:00.503016+0000","last_undegraded":"2024-09-10T10:44:00.503016+0000","last_fullsized":"2024-09-10T10:44:00.503016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:56.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515085+0000","last_change":"2024-09-10T10:43:00.289489+0000","last_active":"2024-09-10T10:44:00.515085+0000","last_peered":"2024-09-10T10:44:00.515085+0000","last_clean":"2024-09-10T10:44:00.515085+0000","last_became_active":"2024-09-10T10:43:00.289058+0000","last_became_peered":"2024-09-10T10:43:00.289058+0000","last_unstale":"2024-09-10T10:44:00.515085+0000","last_undegraded":"2024-09-10T10:44:00.515085+0000","last_fullsized":"2024-09-10T10:44:00.515085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:29.386895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440308+0000","last_change":"2024-09-10T10:42:53.255457+0000","last_active":"2024-09-10T10:44:59.440308+0000","last_peered":"2024-09-10T10:44:59.440308+0000","last_clean":"2024-09-10T10:44:59.440308+0000","last_became_active":"2024-09-10T10:42:53.254937+0000","last_became_peered":"2024-09-10T10:42:53.254937+0000","last_unstale":"2024-09-10T10:44:59.440308+0000","last_undegraded":"2024-09-10T10:44:59.440308+0000","last_fullsized":"2024-09-10T10:44:59.440308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:49.363836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:05.823762+0000","last_change":"2024-09-10T10:42:51.223502+0000","last_active":"2024-09-10T10:45:05.823762+0000","last_peered":"2024-09-10T10:45:05.823762+0000","last_clean":"2024-09-10T10:45:05.823762+0000","last_became_active":"2024-09-10T10:42:51.222664+0000","last_became_peered":"2024-09-10T10:42:51.222664+0000","last_unstale":"2024-09-10T10:45:05.823762+0000","last_undegraded":"2024-09-10T10:45:05.823762+0000","last_fullsized":"2024-09-10T10:45:05.823762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:14.670404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515302+0000","last_change":"2024-09-10T10:43:00.291040+0000","last_active":"2024-09-10T10:44:00.515302+0000","last_peered":"2024-09-10T10:44:00.515302+0000","last_clean":"2024-09-10T10:44:00.515302+0000","last_became_active":"2024-09-10T10:43:00.290613+0000","last_became_peered":"2024-09-10T10:43:00.290613+0000","last_unstale":"2024-09-10T10:44:00.515302+0000","last_undegraded":"2024-09-10T10:44:00.515302+0000","last_fullsized":"2024-09-10T10:44:00.515302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:47.222997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442616+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442616+0000","last_peered":"2024-09-10T10:44:59.442616+0000","last_clean":"2024-09-10T10:44:59.442616+0000","last_became_active":"2024-09-10T10:42:53.244617+0000","last_became_peered":"2024-09-10T10:42:53.244617+0000","last_unstale":"2024-09-10T10:44:59.442616+0000","last_undegraded":"2024-09-10T10:44:59.442616+0000","last_fullsized":"2024-09-10T10:44:59.442616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:56:37.431438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505428+0000","last_change":"2024-09-10T10:42:51.400577+0000","last_active":"2024-09-10T10:44:00.505428+0000","last_peered":"2024-09-10T10:44:00.505428+0000","last_clean":"2024-09-10T10:44:00.505428+0000","last_became_active":"2024-09-10T10:42:51.400116+0000","last_became_peered":"2024-09-10T10:42:51.400116+0000","last_unstale":"2024-09-10T10:44:00.505428+0000","last_undegraded":"2024-09-10T10:44:00.505428+0000","last_fullsized":"2024-09-10T10:44:00.505428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:28.092092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.507155+0000","last_change":"2024-09-10T10:44:00.506639+0000","last_active":"2024-09-10T10:44:00.507155+0000","last_peered":"2024-09-10T10:44:00.507155+0000","last_clean":"2024-09-10T10:44:00.507155+0000","last_became_active":"2024-09-10T10:44:00.506218+0000","last_became_peered":"2024-09-10T10:44:00.506218+0000","last_unstale":"2024-09-10T10:44:00.507155+0000","last_undegraded":"2024-09-10T10:44:00.507155+0000","last_fullsized":"2024-09-10T10:44:00.507155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:08.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441344+0000","last_change":"2024-09-10T10:42:53.247993+0000","last_active":"2024-09-10T10:44:59.441344+0000","last_peered":"2024-09-10T10:44:59.441344+0000","last_clean":"2024-09-10T10:44:59.441344+0000","last_became_active":"2024-09-10T10:42:53.247682+0000","last_became_peered":"2024-09-10T10:42:53.247682+0000","last_unstale":"2024-09-10T10:44:59.441344+0000","last_undegraded":"2024-09-10T10:44:59.441344+0000","last_fullsized":"2024-09-10T10:44:59.441344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:05.207410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:26.725721+0000","last_change":"2024-09-10T10:42:51.224805+0000","last_active":"2024-09-10T10:45:26.725721+0000","last_peered":"2024-09-10T10:45:26.725721+0000","last_clean":"2024-09-10T10:45:26.725721+0000","last_became_active":"2024-09-10T10:42:51.224119+0000","last_became_peered":"2024-09-10T10:42:51.224119+0000","last_unstale":"2024-09-10T10:45:26.725721+0000","last_undegraded":"2024-09-10T10:45:26.725721+0000","last_fullsized":"2024-09-10T10:45:26.725721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:28.061394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500386+0000","last_change":"2024-09-10T10:43:00.293276+0000","last_active":"2024-09-10T10:44:00.500386+0000","last_peered":"2024-09-10T10:44:00.500386+0000","last_clean":"2024-09-10T10:44:00.500386+0000","last_became_active":"2024-09-10T10:43:00.292892+0000","last_became_peered":"2024-09-10T10:43:00.292892+0000","last_unstale":"2024-09-10T10:44:00.500386+0000","last_undegraded":"2024-09-10T10:44:00.500386+0000","last_fullsized":"2024-09-10T10:44:00.500386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:56.041097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445872+0000","last_change":"2024-09-10T10:44:00.509410+0000","last_active":"2024-09-10T10:44:59.445872+0000","last_peered":"2024-09-10T10:44:59.445872+0000","last_clean":"2024-09-10T10:44:59.445872+0000","last_became_active":"2024-09-10T10:44:00.509041+0000","last_became_peered":"2024-09-10T10:44:00.509041+0000","last_unstale":"2024-09-10T10:44:59.445872+0000","last_undegraded":"2024-09-10T10:44:59.445872+0000","last_fullsized":"2024-09-10T10:44:59.445872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:36.231783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515497+0000","last_change":"2024-09-10T10:42:51.223872+0000","last_active":"2024-09-10T10:44:00.515497+0000","last_peered":"2024-09-10T10:44:00.515497+0000","last_clean":"2024-09-10T10:44:00.515497+0000","last_became_active":"2024-09-10T10:42:51.223528+0000","last_became_peered":"2024-09-10T10:42:51.223528+0000","last_unstale":"2024-09-10T10:44:00.515497+0000","last_undegraded":"2024-09-10T10:44:00.515497+0000","last_fullsized":"2024-09-10T10:44:00.515497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:26.926556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502910+0000","last_change":"2024-09-10T10:43:00.288773+0000","last_active":"2024-09-10T10:44:00.502910+0000","last_peered":"2024-09-10T10:44:00.502910+0000","last_clean":"2024-09-10T10:44:00.502910+0000","last_became_active":"2024-09-10T10:43:00.288311+0000","last_became_peered":"2024-09-10T10:43:00.288311+0000","last_unstale":"2024-09-10T10:44:00.502910+0000","last_undegraded":"2024-09-10T10:44:00.502910+0000","last_fullsized":"2024-09-10T10:44:00.502910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:54.673316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.435638+0000","last_change":"2024-09-10T10:42:53.246149+0000","last_active":"2024-09-10T10:44:59.435638+0000","last_peered":"2024-09-10T10:44:59.435638+0000","last_clean":"2024-09-10T10:44:59.435638+0000","last_became_active":"2024-09-10T10:42:53.245862+0000","last_became_peered":"2024-09-10T10:42:53.245862+0000","last_unstale":"2024-09-10T10:44:59.435638+0000","last_undegraded":"2024-09-10T10:44:59.435638+0000","last_fullsized":"2024-09-10T10:44:59.435638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:42.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.519165+0000","last_change":"2024-09-10T10:44:00.518795+0000","last_active":"2024-09-10T10:44:00.519165+0000","last_peered":"2024-09-10T10:44:00.519165+0000","last_clean":"2024-09-10T10:44:00.519165+0000","last_became_active":"2024-09-10T10:44:00.507958+0000","last_became_peered":"2024-09-10T10:44:00.507958+0000","last_unstale":"2024-09-10T10:44:00.519165+0000","last_undegraded":"2024-09-10T10:44:00.519165+0000","last_fullsized":"2024-09-10T10:44:00.519165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:43.790274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516482+0000","last_change":"2024-09-10T10:43:00.293107+0000","last_active":"2024-09-10T10:44:00.516482+0000","last_peered":"2024-09-10T10:44:00.516482+0000","last_clean":"2024-09-10T10:44:00.516482+0000","last_became_active":"2024-09-10T10:43:00.292062+0000","last_became_peered":"2024-09-10T10:43:00.292062+0000","last_unstale":"2024-09-10T10:44:00.516482+0000","last_undegraded":"2024-09-10T10:44:00.516482+0000","last_fullsized":"2024-09-10T10:44:00.516482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:22.146059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:42:53.254008+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:42:53.253651+0000","last_became_peered":"2024-09-10T10:42:53.253651+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:43.725612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":336,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.639756+0000","last_change":"2024-09-10T10:44:00.519644+0000","last_active":"2024-09-10T10:45:25.639756+0000","last_peered":"2024-09-10T10:45:25.639756+0000","last_clean":"2024-09-10T10:45:25.639756+0000","last_became_active":"2024-09-10T10:44:00.512006+0000","last_became_peered":"2024-09-10T10:44:00.512006+0000","last_unstale":"2024-09-10T10:45:25.639756+0000","last_undegraded":"2024-09-10T10:45:25.639756+0000","last_fullsized":"2024-09-10T10:45:25.639756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:53.732160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70047,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":186,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":62076,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500133+0000","last_change":"2024-09-10T10:43:00.290326+0000","last_active":"2024-09-10T10:44:00.500133+0000","last_peered":"2024-09-10T10:44:00.500133+0000","last_clean":"2024-09-10T10:44:00.500133+0000","last_became_active":"2024-09-10T10:43:00.289973+0000","last_became_peered":"2024-09-10T10:43:00.289973+0000","last_unstale":"2024-09-10T10:44:00.500133+0000","last_undegraded":"2024-09-10T10:44:00.500133+0000","last_fullsized":"2024-09-10T10:44:00.500133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:18.664098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.245884+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.245573+0000","last_became_peered":"2024-09-10T10:42:53.245573+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:50.387829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.516289+0000","last_change":"2024-09-10T10:42:51.400847+0000","last_active":"2024-09-10T10:44:00.516289+0000","last_peered":"2024-09-10T10:44:00.516289+0000","last_clean":"2024-09-10T10:44:00.516289+0000","last_became_active":"2024-09-10T10:42:51.400382+0000","last_became_peered":"2024-09-10T10:42:51.400382+0000","last_unstale":"2024-09-10T10:44:00.516289+0000","last_undegraded":"2024-09-10T10:44:00.516289+0000","last_fullsized":"2024-09-10T10:44:00.516289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.101504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.503829+0000","last_change":"2024-09-10T10:43:00.291754+0000","last_active":"2024-09-10T10:44:00.503829+0000","last_peered":"2024-09-10T10:44:00.503829+0000","last_clean":"2024-09-10T10:44:00.503829+0000","last_became_active":"2024-09-10T10:43:00.291174+0000","last_became_peered":"2024-09-10T10:43:00.291174+0000","last_unstale":"2024-09-10T10:44:00.503829+0000","last_undegraded":"2024-09-10T10:44:00.503829+0000","last_fullsized":"2024-09-10T10:44:00.503829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:08.841954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.443668+0000","last_change":"2024-09-10T10:42:53.251985+0000","last_active":"2024-09-10T10:44:59.443668+0000","last_peered":"2024-09-10T10:44:59.443668+0000","last_clean":"2024-09-10T10:44:59.443668+0000","last_became_active":"2024-09-10T10:42:53.251678+0000","last_became_peered":"2024-09-10T10:42:53.251678+0000","last_unstale":"2024-09-10T10:44:59.443668+0000","last_undegraded":"2024-09-10T10:44:59.443668+0000","last_fullsized":"2024-09-10T10:44:59.443668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:24.542856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.506656+0000","last_change":"2024-09-10T10:42:51.227831+0000","last_active":"2024-09-10T10:44:00.506656+0000","last_peered":"2024-09-10T10:44:00.506656+0000","last_clean":"2024-09-10T10:44:00.506656+0000","last_became_active":"2024-09-10T10:42:51.227517+0000","last_became_peered":"2024-09-10T10:42:51.227517+0000","last_unstale":"2024-09-10T10:44:00.506656+0000","last_undegraded":"2024-09-10T10:44:00.506656+0000","last_fullsized":"2024-09-10T10:44:00.506656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:11.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.499917+0000","last_change":"2024-09-10T10:43:00.288448+0000","last_active":"2024-09-10T10:44:00.499917+0000","last_peered":"2024-09-10T10:44:00.499917+0000","last_clean":"2024-09-10T10:44:00.499917+0000","last_became_active":"2024-09-10T10:43:00.288121+0000","last_became_peered":"2024-09-10T10:43:00.288121+0000","last_unstale":"2024-09-10T10:44:00.499917+0000","last_undegraded":"2024-09-10T10:44:00.499917+0000","last_fullsized":"2024-09-10T10:44:00.499917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:26.121272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442320+0000","last_change":"2024-09-10T10:42:53.243669+0000","last_active":"2024-09-10T10:44:59.442320+0000","last_peered":"2024-09-10T10:44:59.442320+0000","last_clean":"2024-09-10T10:44:59.442320+0000","last_became_active":"2024-09-10T10:42:53.243323+0000","last_became_peered":"2024-09-10T10:42:53.243323+0000","last_unstale":"2024-09-10T10:44:59.442320+0000","last_undegraded":"2024-09-10T10:44:59.442320+0000","last_fullsized":"2024-09-10T10:44:59.442320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:09.867949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":270,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.721017+0000","last_change":"2024-09-10T10:42:51.228783+0000","last_active":"2024-09-10T10:45:25.721017+0000","last_peered":"2024-09-10T10:45:25.721017+0000","last_clean":"2024-09-10T10:45:25.721017+0000","last_became_active":"2024-09-10T10:42:51.228450+0000","last_became_peered":"2024-09-10T10:42:51.228450+0000","last_unstale":"2024-09-10T10:45:25.721017+0000","last_undegraded":"2024-09-10T10:45:25.721017+0000","last_fullsized":"2024-09-10T10:45:25.721017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:34.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.504776+0000","last_change":"2024-09-10T10:43:00.292962+0000","last_active":"2024-09-10T10:44:00.504776+0000","last_peered":"2024-09-10T10:44:00.504776+0000","last_clean":"2024-09-10T10:44:00.504776+0000","last_became_active":"2024-09-10T10:43:00.292621+0000","last_became_peered":"2024-09-10T10:43:00.292621+0000","last_unstale":"2024-09-10T10:44:00.504776+0000","last_undegraded":"2024-09-10T10:44:00.504776+0000","last_fullsized":"2024-09-10T10:44:00.504776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:03.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.500929+0000","last_change":"2024-09-10T10:43:00.290958+0000","last_active":"2024-09-10T10:44:00.500929+0000","last_peered":"2024-09-10T10:44:00.500929+0000","last_clean":"2024-09-10T10:44:00.500929+0000","last_became_active":"2024-09-10T10:43:00.290607+0000","last_became_peered":"2024-09-10T10:43:00.290607+0000","last_unstale":"2024-09-10T10:44:00.500929+0000","last_undegraded":"2024-09-10T10:44:00.500929+0000","last_fullsized":"2024-09-10T10:44:00.500929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:17.704711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.433074+0000","last_change":"2024-09-10T10:42:53.244983+0000","last_active":"2024-09-10T10:44:59.433074+0000","last_peered":"2024-09-10T10:44:59.433074+0000","last_clean":"2024-09-10T10:44:59.433074+0000","last_became_active":"2024-09-10T10:42:53.244681+0000","last_became_peered":"2024-09-10T10:42:53.244681+0000","last_unstale":"2024-09-10T10:44:59.433074+0000","last_undegraded":"2024-09-10T10:44:59.433074+0000","last_fullsized":"2024-09-10T10:44:59.433074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:34.813253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.515598+0000","last_change":"2024-09-10T10:42:51.226363+0000","last_active":"2024-09-10T10:44:00.515598+0000","last_peered":"2024-09-10T10:44:00.515598+0000","last_clean":"2024-09-10T10:44:00.515598+0000","last_became_active":"2024-09-10T10:42:51.225985+0000","last_became_peered":"2024-09-10T10:42:51.225985+0000","last_unstale":"2024-09-10T10:44:00.515598+0000","last_undegraded":"2024-09-10T10:44:00.515598+0000","last_fullsized":"2024-09-10T10:44:00.515598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:10.902980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.509926+0000","last_change":"2024-09-10T10:44:00.509118+0000","last_active":"2024-09-10T10:44:00.509926+0000","last_peered":"2024-09-10T10:44:00.509926+0000","last_clean":"2024-09-10T10:44:00.509926+0000","last_became_active":"2024-09-10T10:44:00.508600+0000","last_became_peered":"2024-09-10T10:44:00.508600+0000","last_unstale":"2024-09-10T10:44:00.509926+0000","last_undegraded":"2024-09-10T10:44:00.509926+0000","last_fullsized":"2024-09-10T10:44:00.509926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:16.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.434606+0000","last_change":"2024-09-10T10:42:53.243371+0000","last_active":"2024-09-10T10:44:59.434606+0000","last_peered":"2024-09-10T10:44:59.434606+0000","last_clean":"2024-09-10T10:44:59.434606+0000","last_became_active":"2024-09-10T10:42:53.243054+0000","last_became_peered":"2024-09-10T10:42:53.243054+0000","last_unstale":"2024-09-10T10:44:59.434606+0000","last_undegraded":"2024-09-10T10:44:59.434606+0000","last_fullsized":"2024-09-10T10:44:59.434606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:19.513111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.502516+0000","last_change":"2024-09-10T10:42:51.221509+0000","last_active":"2024-09-10T10:44:00.502516+0000","last_peered":"2024-09-10T10:44:00.502516+0000","last_clean":"2024-09-10T10:44:00.502516+0000","last_became_active":"2024-09-10T10:42:51.221119+0000","last_became_peered":"2024-09-10T10:42:51.221119+0000","last_unstale":"2024-09-10T10:44:00.502516+0000","last_undegraded":"2024-09-10T10:44:00.502516+0000","last_fullsized":"2024-09-10T10:44:00.502516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.113514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.505395+0000","last_change":"2024-09-10T10:42:51.226833+0000","last_active":"2024-09-10T10:44:00.505395+0000","last_peered":"2024-09-10T10:44:00.505395+0000","last_clean":"2024-09-10T10:44:00.505395+0000","last_became_active":"2024-09-10T10:42:51.226437+0000","last_became_peered":"2024-09-10T10:42:51.226437+0000","last_unstale":"2024-09-10T10:44:00.505395+0000","last_undegraded":"2024-09-10T10:44:00.505395+0000","last_fullsized":"2024-09-10T10:44:00.505395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:35.953123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438633+0000","last_change":"2024-09-10T10:42:53.238346+0000","last_active":"2024-09-10T10:44:59.438633+0000","last_peered":"2024-09-10T10:44:59.438633+0000","last_clean":"2024-09-10T10:44:59.438633+0000","last_became_active":"2024-09-10T10:42:53.237999+0000","last_became_peered":"2024-09-10T10:42:53.237999+0000","last_unstale":"2024-09-10T10:44:59.438633+0000","last_undegraded":"2024-09-10T10:44:59.438633+0000","last_fullsized":"2024-09-10T10:44:59.438633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:44:41.501520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501401+0000","last_change":"2024-09-10T10:43:00.290568+0000","last_active":"2024-09-10T10:44:00.501401+0000","last_peered":"2024-09-10T10:44:00.501401+0000","last_clean":"2024-09-10T10:44:00.501401+0000","last_became_active":"2024-09-10T10:43:00.287908+0000","last_became_peered":"2024-09-10T10:43:00.287908+0000","last_unstale":"2024-09-10T10:44:00.501401+0000","last_undegraded":"2024-09-10T10:44:00.501401+0000","last_fullsized":"2024-09-10T10:44:00.501401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:35.370283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:45:25.721739+0000","last_change":"2024-09-10T10:42:51.225041+0000","last_active":"2024-09-10T10:45:25.721739+0000","last_peered":"2024-09-10T10:45:25.721739+0000","last_clean":"2024-09-10T10:45:25.721739+0000","last_became_active":"2024-09-10T10:42:51.224676+0000","last_became_peered":"2024-09-10T10:42:51.224676+0000","last_unstale":"2024-09-10T10:45:25.721739+0000","last_undegraded":"2024-09-10T10:45:25.721739+0000","last_fullsized":"2024-09-10T10:45:25.721739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:50.203771+0000","last_clean_scrub_stamp":"2024-09-10T10:42:50.203771+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:09.168351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439433+0000","last_change":"2024-09-10T10:42:53.245263+0000","last_active":"2024-09-10T10:44:59.439433+0000","last_peered":"2024-09-10T10:44:59.439433+0000","last_clean":"2024-09-10T10:44:59.439433+0000","last_became_active":"2024-09-10T10:42:53.244910+0000","last_became_peered":"2024-09-10T10:42:53.244910+0000","last_unstale":"2024-09-10T10:44:59.439433+0000","last_undegraded":"2024-09-10T10:44:59.439433+0000","last_fullsized":"2024-09-10T10:44:59.439433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:00.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:00.501408+0000","last_change":"2024-09-10T10:43:00.291391+0000","last_active":"2024-09-10T10:44:00.501408+0000","last_peered":"2024-09-10T10:44:00.501408+0000","last_clean":"2024-09-10T10:44:00.501408+0000","last_became_active":"2024-09-10T10:43:00.289599+0000","last_became_peered":"2024-09-10T10:43:00.289599+0000","last_unstale":"2024-09-10T10:44:00.501408+0000","last_undegraded":"2024-09-10T10:44:00.501408+0000","last_fullsized":"2024-09-10T10:44:00.501408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:59.260345+0000","last_clean_scrub_stamp":"2024-09-10T10:42:59.260345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:27:14.444154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440974+0000","last_change":"2024-09-10T10:42:53.251417+0000","last_active":"2024-09-10T10:44:59.440974+0000","last_peered":"2024-09-10T10:44:59.440974+0000","last_clean":"2024-09-10T10:44:59.440974+0000","last_became_active":"2024-09-10T10:42:53.251074+0000","last_became_peered":"2024-09-10T10:42:53.251074+0000","last_unstale":"2024-09-10T10:44:59.440974+0000","last_undegraded":"2024-09-10T10:44:59.440974+0000","last_fullsized":"2024-09-10T10:44:59.440974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:11.838550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441221+0000","last_change":"2024-09-10T10:42:53.245753+0000","last_active":"2024-09-10T10:44:59.441221+0000","last_peered":"2024-09-10T10:44:59.441221+0000","last_clean":"2024-09-10T10:44:59.441221+0000","last_became_active":"2024-09-10T10:42:53.245410+0000","last_became_peered":"2024-09-10T10:42:53.245410+0000","last_unstale":"2024-09-10T10:44:59.441221+0000","last_undegraded":"2024-09-10T10:44:59.441221+0000","last_fullsized":"2024-09-10T10:44:59.441221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:58.327391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440703+0000","last_change":"2024-09-10T10:42:53.246449+0000","last_active":"2024-09-10T10:44:59.440703+0000","last_peered":"2024-09-10T10:44:59.440703+0000","last_clean":"2024-09-10T10:44:59.440703+0000","last_became_active":"2024-09-10T10:42:53.246129+0000","last_became_peered":"2024-09-10T10:42:53.246129+0000","last_unstale":"2024-09-10T10:44:59.440703+0000","last_undegraded":"2024-09-10T10:44:59.440703+0000","last_fullsized":"2024-09-10T10:44:59.440703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:41.141183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444329+0000","last_change":"2024-09-10T10:42:53.253623+0000","last_active":"2024-09-10T10:44:59.444329+0000","last_peered":"2024-09-10T10:44:59.444329+0000","last_clean":"2024-09-10T10:44:59.444329+0000","last_became_active":"2024-09-10T10:42:53.253098+0000","last_became_peered":"2024-09-10T10:42:53.253098+0000","last_unstale":"2024-09-10T10:44:59.444329+0000","last_undegraded":"2024-09-10T10:44:59.444329+0000","last_fullsized":"2024-09-10T10:44:59.444329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:10.524355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438450+0000","last_change":"2024-09-10T10:42:53.248805+0000","last_active":"2024-09-10T10:44:59.438450+0000","last_peered":"2024-09-10T10:44:59.438450+0000","last_clean":"2024-09-10T10:44:59.438450+0000","last_became_active":"2024-09-10T10:42:53.248478+0000","last_became_peered":"2024-09-10T10:42:53.248478+0000","last_unstale":"2024-09-10T10:44:59.438450+0000","last_undegraded":"2024-09-10T10:44:59.438450+0000","last_fullsized":"2024-09-10T10:44:59.438450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:38.272899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441275+0000","last_change":"2024-09-10T10:42:53.252527+0000","last_active":"2024-09-10T10:44:59.441275+0000","last_peered":"2024-09-10T10:44:59.441275+0000","last_clean":"2024-09-10T10:44:59.441275+0000","last_became_active":"2024-09-10T10:42:53.252204+0000","last_became_peered":"2024-09-10T10:42:53.252204+0000","last_unstale":"2024-09-10T10:44:59.441275+0000","last_undegraded":"2024-09-10T10:44:59.441275+0000","last_fullsized":"2024-09-10T10:44:59.441275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:57.852142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437164+0000","last_change":"2024-09-10T10:42:53.246320+0000","last_active":"2024-09-10T10:44:59.437164+0000","last_peered":"2024-09-10T10:44:59.437164+0000","last_clean":"2024-09-10T10:44:59.437164+0000","last_became_active":"2024-09-10T10:42:53.246001+0000","last_became_peered":"2024-09-10T10:42:53.246001+0000","last_unstale":"2024-09-10T10:44:59.437164+0000","last_undegraded":"2024-09-10T10:44:59.437164+0000","last_fullsized":"2024-09-10T10:44:59.437164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:55.973647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441514+0000","last_change":"2024-09-10T10:42:53.243112+0000","last_active":"2024-09-10T10:44:59.441514+0000","last_peered":"2024-09-10T10:44:59.441514+0000","last_clean":"2024-09-10T10:44:59.441514+0000","last_became_active":"2024-09-10T10:42:53.242815+0000","last_became_peered":"2024-09-10T10:42:53.242815+0000","last_unstale":"2024-09-10T10:44:59.441514+0000","last_undegraded":"2024-09-10T10:44:59.441514+0000","last_fullsized":"2024-09-10T10:44:59.441514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:45.403173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442360+0000","last_change":"2024-09-10T10:42:53.248330+0000","last_active":"2024-09-10T10:44:59.442360+0000","last_peered":"2024-09-10T10:44:59.442360+0000","last_clean":"2024-09-10T10:44:59.442360+0000","last_became_active":"2024-09-10T10:42:53.247970+0000","last_became_peered":"2024-09-10T10:42:53.247970+0000","last_unstale":"2024-09-10T10:44:59.442360+0000","last_undegraded":"2024-09-10T10:44:59.442360+0000","last_fullsized":"2024-09-10T10:44:59.442360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:59.157401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442124+0000","last_change":"2024-09-10T10:42:53.242134+0000","last_active":"2024-09-10T10:44:59.442124+0000","last_peered":"2024-09-10T10:44:59.442124+0000","last_clean":"2024-09-10T10:44:59.442124+0000","last_became_active":"2024-09-10T10:42:53.241849+0000","last_became_peered":"2024-09-10T10:42:53.241849+0000","last_unstale":"2024-09-10T10:44:59.442124+0000","last_undegraded":"2024-09-10T10:44:59.442124+0000","last_fullsized":"2024-09-10T10:44:59.442124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:27.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442058+0000","last_change":"2024-09-10T10:42:53.248394+0000","last_active":"2024-09-10T10:44:59.442058+0000","last_peered":"2024-09-10T10:44:59.442058+0000","last_clean":"2024-09-10T10:44:59.442058+0000","last_became_active":"2024-09-10T10:42:53.248108+0000","last_became_peered":"2024-09-10T10:42:53.248108+0000","last_unstale":"2024-09-10T10:44:59.442058+0000","last_undegraded":"2024-09-10T10:44:59.442058+0000","last_fullsized":"2024-09-10T10:44:59.442058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:20.736222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438851+0000","last_change":"2024-09-10T10:42:53.245235+0000","last_active":"2024-09-10T10:44:59.438851+0000","last_peered":"2024-09-10T10:44:59.438851+0000","last_clean":"2024-09-10T10:44:59.438851+0000","last_became_active":"2024-09-10T10:42:53.244567+0000","last_became_peered":"2024-09-10T10:42:53.244567+0000","last_unstale":"2024-09-10T10:44:59.438851+0000","last_undegraded":"2024-09-10T10:44:59.438851+0000","last_fullsized":"2024-09-10T10:44:59.438851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:57.911151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429557+0000","last_change":"2024-09-10T10:42:53.255688+0000","last_active":"2024-09-10T10:44:59.429557+0000","last_peered":"2024-09-10T10:44:59.429557+0000","last_clean":"2024-09-10T10:44:59.429557+0000","last_became_active":"2024-09-10T10:42:53.255127+0000","last_became_peered":"2024-09-10T10:42:53.255127+0000","last_unstale":"2024-09-10T10:44:59.429557+0000","last_undegraded":"2024-09-10T10:44:59.429557+0000","last_fullsized":"2024-09-10T10:44:59.429557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:15.656184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440551+0000","last_change":"2024-09-10T10:42:53.245226+0000","last_active":"2024-09-10T10:44:59.440551+0000","last_peered":"2024-09-10T10:44:59.440551+0000","last_clean":"2024-09-10T10:44:59.440551+0000","last_became_active":"2024-09-10T10:42:53.244884+0000","last_became_peered":"2024-09-10T10:42:53.244884+0000","last_unstale":"2024-09-10T10:44:59.440551+0000","last_undegraded":"2024-09-10T10:44:59.440551+0000","last_fullsized":"2024-09-10T10:44:59.440551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:49.821431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437183+0000","last_change":"2024-09-10T10:42:53.242435+0000","last_active":"2024-09-10T10:44:59.437183+0000","last_peered":"2024-09-10T10:44:59.437183+0000","last_clean":"2024-09-10T10:44:59.437183+0000","last_became_active":"2024-09-10T10:42:53.242103+0000","last_became_peered":"2024-09-10T10:42:53.242103+0000","last_unstale":"2024-09-10T10:44:59.437183+0000","last_undegraded":"2024-09-10T10:44:59.437183+0000","last_fullsized":"2024-09-10T10:44:59.437183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:48.314114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439780+0000","last_change":"2024-09-10T10:42:53.250367+0000","last_active":"2024-09-10T10:44:59.439780+0000","last_peered":"2024-09-10T10:44:59.439780+0000","last_clean":"2024-09-10T10:44:59.439780+0000","last_became_active":"2024-09-10T10:42:53.250051+0000","last_became_peered":"2024-09-10T10:42:53.250051+0000","last_unstale":"2024-09-10T10:44:59.439780+0000","last_undegraded":"2024-09-10T10:44:59.439780+0000","last_fullsized":"2024-09-10T10:44:59.439780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:47.624927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441856+0000","last_change":"2024-09-10T10:42:53.245700+0000","last_active":"2024-09-10T10:44:59.441856+0000","last_peered":"2024-09-10T10:44:59.441856+0000","last_clean":"2024-09-10T10:44:59.441856+0000","last_became_active":"2024-09-10T10:42:53.245405+0000","last_became_peered":"2024-09-10T10:42:53.245405+0000","last_unstale":"2024-09-10T10:44:59.441856+0000","last_undegraded":"2024-09-10T10:44:59.441856+0000","last_fullsized":"2024-09-10T10:44:59.441856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:16.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.437328+0000","last_change":"2024-09-10T10:42:53.252792+0000","last_active":"2024-09-10T10:44:59.437328+0000","last_peered":"2024-09-10T10:44:59.437328+0000","last_clean":"2024-09-10T10:44:59.437328+0000","last_became_active":"2024-09-10T10:42:53.252487+0000","last_became_peered":"2024-09-10T10:42:53.252487+0000","last_unstale":"2024-09-10T10:44:59.437328+0000","last_undegraded":"2024-09-10T10:44:59.437328+0000","last_fullsized":"2024-09-10T10:44:59.437328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:36.244699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.438618+0000","last_change":"2024-09-10T10:42:53.247601+0000","last_active":"2024-09-10T10:44:59.438618+0000","last_peered":"2024-09-10T10:44:59.438618+0000","last_clean":"2024-09-10T10:44:59.438618+0000","last_became_active":"2024-09-10T10:42:53.247325+0000","last_became_peered":"2024-09-10T10:42:53.247325+0000","last_unstale":"2024-09-10T10:44:59.438618+0000","last_undegraded":"2024-09-10T10:44:59.438618+0000","last_fullsized":"2024-09-10T10:44:59.438618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:30.849424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.444056+0000","last_change":"2024-09-10T10:42:53.253713+0000","last_active":"2024-09-10T10:44:59.444056+0000","last_peered":"2024-09-10T10:44:59.444056+0000","last_clean":"2024-09-10T10:44:59.444056+0000","last_became_active":"2024-09-10T10:42:53.253430+0000","last_became_peered":"2024-09-10T10:42:53.253430+0000","last_unstale":"2024-09-10T10:44:59.444056+0000","last_undegraded":"2024-09-10T10:44:59.444056+0000","last_fullsized":"2024-09-10T10:44:59.444056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:55.603459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.432375+0000","last_change":"2024-09-10T10:44:00.508081+0000","last_active":"2024-09-10T10:44:59.432375+0000","last_peered":"2024-09-10T10:44:59.432375+0000","last_clean":"2024-09-10T10:44:59.432375+0000","last_became_active":"2024-09-10T10:44:00.507718+0000","last_became_peered":"2024-09-10T10:44:00.507718+0000","last_unstale":"2024-09-10T10:44:59.432375+0000","last_undegraded":"2024-09-10T10:44:59.432375+0000","last_fullsized":"2024-09-10T10:44:59.432375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:39.777264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442679+0000","last_change":"2024-09-10T10:42:53.253090+0000","last_active":"2024-09-10T10:44:59.442679+0000","last_peered":"2024-09-10T10:44:59.442679+0000","last_clean":"2024-09-10T10:44:59.442679+0000","last_became_active":"2024-09-10T10:42:53.252770+0000","last_became_peered":"2024-09-10T10:42:53.252770+0000","last_unstale":"2024-09-10T10:44:59.442679+0000","last_undegraded":"2024-09-10T10:44:59.442679+0000","last_fullsized":"2024-09-10T10:44:59.442679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:47.755987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.433231+0000","last_change":"2024-09-10T10:42:53.253719+0000","last_active":"2024-09-10T10:44:59.433231+0000","last_peered":"2024-09-10T10:44:59.433231+0000","last_clean":"2024-09-10T10:44:59.433231+0000","last_became_active":"2024-09-10T10:42:53.252421+0000","last_became_peered":"2024-09-10T10:42:53.252421+0000","last_unstale":"2024-09-10T10:44:59.433231+0000","last_undegraded":"2024-09-10T10:44:59.433231+0000","last_fullsized":"2024-09-10T10:44:59.433231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:29.147057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.442732+0000","last_change":"2024-09-10T10:42:53.244922+0000","last_active":"2024-09-10T10:44:59.442732+0000","last_peered":"2024-09-10T10:44:59.442732+0000","last_clean":"2024-09-10T10:44:59.442732+0000","last_became_active":"2024-09-10T10:42:53.244595+0000","last_became_peered":"2024-09-10T10:42:53.244595+0000","last_unstale":"2024-09-10T10:44:59.442732+0000","last_undegraded":"2024-09-10T10:44:59.442732+0000","last_fullsized":"2024-09-10T10:44:59.442732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:21.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.440910+0000","last_change":"2024-09-10T10:42:53.240749+0000","last_active":"2024-09-10T10:44:59.440910+0000","last_peered":"2024-09-10T10:44:59.440910+0000","last_clean":"2024-09-10T10:44:59.440910+0000","last_became_active":"2024-09-10T10:42:53.240367+0000","last_became_peered":"2024-09-10T10:42:53.240367+0000","last_unstale":"2024-09-10T10:44:59.440910+0000","last_undegraded":"2024-09-10T10:44:59.440910+0000","last_fullsized":"2024-09-10T10:44:59.440910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:21.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.445105+0000","last_change":"2024-09-10T10:42:53.248954+0000","last_active":"2024-09-10T10:44:59.445105+0000","last_peered":"2024-09-10T10:44:59.445105+0000","last_clean":"2024-09-10T10:44:59.445105+0000","last_became_active":"2024-09-10T10:42:53.248640+0000","last_became_peered":"2024-09-10T10:42:53.248640+0000","last_unstale":"2024-09-10T10:44:59.445105+0000","last_undegraded":"2024-09-10T10:44:59.445105+0000","last_fullsized":"2024-09-10T10:44:59.445105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:13.476788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.428701+0000","last_change":"2024-09-10T10:42:53.245002+0000","last_active":"2024-09-10T10:44:59.428701+0000","last_peered":"2024-09-10T10:44:59.428701+0000","last_clean":"2024-09-10T10:44:59.428701+0000","last_became_active":"2024-09-10T10:42:53.244339+0000","last_became_peered":"2024-09-10T10:42:53.244339+0000","last_unstale":"2024-09-10T10:44:59.428701+0000","last_undegraded":"2024-09-10T10:44:59.428701+0000","last_fullsized":"2024-09-10T10:44:59.428701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:33.606653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.429050+0000","last_change":"2024-09-10T10:42:53.251104+0000","last_active":"2024-09-10T10:44:59.429050+0000","last_peered":"2024-09-10T10:44:59.429050+0000","last_clean":"2024-09-10T10:44:59.429050+0000","last_became_active":"2024-09-10T10:42:53.250136+0000","last_became_peered":"2024-09-10T10:42:53.250136+0000","last_unstale":"2024-09-10T10:44:59.429050+0000","last_undegraded":"2024-09-10T10:44:59.429050+0000","last_fullsized":"2024-09-10T10:44:59.429050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:39.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441024+0000","last_change":"2024-09-10T10:42:53.246779+0000","last_active":"2024-09-10T10:44:59.441024+0000","last_peered":"2024-09-10T10:44:59.441024+0000","last_clean":"2024-09-10T10:44:59.441024+0000","last_became_active":"2024-09-10T10:42:53.245844+0000","last_became_peered":"2024-09-10T10:42:53.245844+0000","last_unstale":"2024-09-10T10:44:59.441024+0000","last_undegraded":"2024-09-10T10:44:59.441024+0000","last_fullsized":"2024-09-10T10:44:59.441024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:53.438835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.441764+0000","last_change":"2024-09-10T10:42:53.245267+0000","last_active":"2024-09-10T10:44:59.441764+0000","last_peered":"2024-09-10T10:44:59.441764+0000","last_clean":"2024-09-10T10:44:59.441764+0000","last_became_active":"2024-09-10T10:42:53.244993+0000","last_became_peered":"2024-09-10T10:42:53.244993+0000","last_unstale":"2024-09-10T10:44:59.441764+0000","last_undegraded":"2024-09-10T10:44:59.441764+0000","last_fullsized":"2024-09-10T10:44:59.441764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:31.754774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439409+0000","last_change":"2024-09-10T10:42:53.245500+0000","last_active":"2024-09-10T10:44:59.439409+0000","last_peered":"2024-09-10T10:44:59.439409+0000","last_clean":"2024-09-10T10:44:59.439409+0000","last_became_active":"2024-09-10T10:42:53.244857+0000","last_became_peered":"2024-09-10T10:42:53.244857+0000","last_unstale":"2024-09-10T10:44:59.439409+0000","last_undegraded":"2024-09-10T10:44:59.439409+0000","last_fullsized":"2024-09-10T10:44:59.439409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:10.345283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-10T10:44:59.439118+0000","last_change":"2024-09-10T10:42:53.250071+0000","last_active":"2024-09-10T10:44:59.439118+0000","last_peered":"2024-09-10T10:44:59.439118+0000","last_clean":"2024-09-10T10:44:59.439118+0000","last_became_active":"2024-09-10T10:42:53.249547+0000","last_became_peered":"2024-09-10T10:42:53.249547+0000","last_unstale":"2024-09-10T10:44:59.439118+0000","last_undegraded":"2024-09-10T10:44:59.439118+0000","last_fullsized":"2024-09-10T10:44:59.439118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:42:52.218112+0000","last_clean_scrub_stamp":"2024-09-10T10:42:52.218112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:05.181765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79910,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1197,"num_read_kb":683,"num_write":280,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":62144,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":239730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547243,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":501134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776178,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1162025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":499967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168637,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":500345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397570,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1098863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561095,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":496359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790028,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1092528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":570621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215190,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":495179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1024000,"data_stored":498177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":567436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:32.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:32.946 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:32 smithi144 ceph-mon[34230]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:32.946 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:32 smithi178 ceph-mon[34813]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:33.194 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 -- 172.21.15.143:0/3367338306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 msgr2=0x7f01b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 --2- 172.21.15.143:0/3367338306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81021c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009900 tx=0x7f01b402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 -- 172.21.15.143:0/3367338306 shutdown_connections 2024-09-10T10:45:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 --2- 172.21.15.143:0/3367338306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 0x7f01b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 --2- 172.21.15.143:0/3367338306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01b8102790 0x7f01b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 --2- 172.21.15.143:0/3367338306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81021c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.685+0000 7f01c0269640 1 -- 172.21.15.143:0/3367338306 >> 172.21.15.143:0/3367338306 conn(0x7f01b80fd8c0 msgr2=0x7f01b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.686+0000 7f01c0269640 1 -- 172.21.15.143:0/3367338306 shutdown_connections 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.686+0000 7f01c0269640 1 -- 172.21.15.143:0/3367338306 wait complete. 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.686+0000 7f01c0269640 1 Processor -- start 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.686+0000 7f01c0269640 1 -- start start 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01b8102790 0x7f01b8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 0x7f01b819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8111de0 con 0x7f01b8101de0 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8111c60 con 0x7f01b8102790 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8111f60 con 0x7f01b8106f10 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50276/0 (socket says 172.21.15.143:50276) 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 -- 172.21.15.143:0/715431705 learned_addr learned my addr 172.21.15.143:0/715431705 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01bd804640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01b8102790 0x7f01b8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be806640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 0x7f01b819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 -- 172.21.15.143:0/715431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 msgr2=0x7f01b819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 0x7f01b819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 -- 172.21.15.143:0/715431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01b8102790 msgr2=0x7f01b8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01b8102790 0x7f01b8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 -- 172.21.15.143:0/715431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b819d370 con 0x7f01b8101de0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be806640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 0x7f01b819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01be005640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81983c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005940 tx=0x7f01b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b403f030 con 0x7f01b8101de0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01b4003aa0 con 0x7f01b8101de0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b819d600 con 0x7f01b8101de0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035070 con 0x7f01b8101de0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.687+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b81a4f40 con 0x7f01b8101de0 2024-09-10T10:45:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.688+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b81032c0 con 0x7f01b8101de0 2024-09-10T10:45:33.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.689+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f01b4042030 con 0x7f01b8101de0 2024-09-10T10:45:33.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.690+0000 7f01af7fe640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f01a40a2c10 0x7f01a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.690+0000 7f01bd804640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f01a40a2c10 0x7f01a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.690+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f01b40efd90 con 0x7f01b8101de0 2024-09-10T10:45:33.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.690+0000 7f01bd804640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f01a40a2c10 0x7f01a40a50d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005e10 tx=0x7f01a8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:33.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.694+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01b40f3030 con 0x7f01b8101de0 2024-09-10T10:45:33.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.895+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01b810a0f0 con 0x7f01b8101de0 2024-09-10T10:45:33.898 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:33 smithi143 ceph-mon[31057]: pgmap v366: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T10:45:33.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.896+0000 7f01af7fe640 1 -- 172.21.15.143:0/715431705 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f01b40bc830 con 0x7f01b8101de0 2024-09-10T10:45:33.899 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:33.899 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:45:33.899 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f01a40a2c10 msgr2=0x7f01a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f01a40a2c10 0x7f01a40a50d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005e10 tx=0x7f01a8005da0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 msgr2=0x7f01b81983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81983c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005940 tx=0x7f01b4004270 comp rx=0 tx=0).stop 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 shutdown_connections 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f01a40a2c10 0x7f01a40a50d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01b8106f10 0x7f01b819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01b8102790 0x7f01b8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 --2- 172.21.15.143:0/715431705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f01b8101de0 0x7f01b81983c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.898+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 >> 172.21.15.143:0/715431705 conn(0x7f01b80fd8c0 msgr2=0x7f01b80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.899+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 shutdown_connections 2024-09-10T10:45:33.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:33.899+0000 7f01c0269640 1 -- 172.21.15.143:0/715431705 wait complete. 2024-09-10T10:45:33.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:33 smithi178 ceph-mon[34813]: pgmap v366: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T10:45:33.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:33 smithi144 ceph-mon[34230]: pgmap v366: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T10:45:34.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:34 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/715431705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:34 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/715431705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:34.989 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:34 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/715431705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 -- 172.21.15.143:0/3197680082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0106f00 msgr2=0x7f3dc010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 --2- 172.21.15.143:0/3197680082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0106f00 0x7f3dc010ea80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009ca0 tx=0x7f3db802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 -- 172.21.15.143:0/3197680082 shutdown_connections 2024-09-10T10:45:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 --2- 172.21.15.143:0/3197680082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0106f00 0x7f3dc010ea80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 --2- 172.21.15.143:0/3197680082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc0102780 0x7f3dc01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 --2- 172.21.15.143:0/3197680082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc0101dd0 0x7f3dc01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 -- 172.21.15.143:0/3197680082 >> 172.21.15.143:0/3197680082 conn(0x7f3dc00fd890 msgr2=0x7f3dc00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 -- 172.21.15.143:0/3197680082 shutdown_connections 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.425+0000 7f3dc5599640 1 -- 172.21.15.143:0/3197680082 wait complete. 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 Processor -- start 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 -- start start 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc0101dd0 0x7f3dc0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 0x7f3dc0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc0106f00 0x7f3dc019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0111e30 con 0x7f3dc0102780 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0111cb0 con 0x7f3dc0106f00 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dc5599640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0111fb0 con 0x7f3dc0101dd0 2024-09-10T10:45:35.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 0x7f3dc0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 0x7f3dc0198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50304/0 (socket says 172.21.15.143:50304) 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 -- 172.21.15.143:0/2844539077 learned_addr learned my addr 172.21.15.143:0/2844539077 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dbeffd640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc0101dd0 0x7f3dc0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3dbf7fe640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc0106f00 0x7f3dc019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 -- 172.21.15.143:0/2844539077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc0101dd0 msgr2=0x7f3dc0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc0101dd0 0x7f3dc0198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 -- 172.21.15.143:0/2844539077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc0106f00 msgr2=0x7f3dc019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc0106f00 0x7f3dc019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.426+0000 7f3db67fc640 1 -- 172.21.15.143:0/2844539077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc019d2c0 con 0x7f3dc0102780 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.427+0000 7f3db67fc640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 0x7f3dc0198850 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b4a0 tx=0x7f3da800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.427+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8014070 con 0x7f3dc0102780 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.427+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da8010ac0 con 0x7f3dc0102780 2024-09-10T10:45:35.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.427+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc019d5b0 con 0x7f3dc0102780 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.427+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8017c30 con 0x7f3dc0102780 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.427+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc01a4e90 con 0x7f3dc0102780 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.428+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc01021b0 con 0x7f3dc0102780 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.429+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3da8017070 con 0x7f3dc0102780 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.430+0000 7f3dbcff9640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3dac0a2c10 0x7f3dac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.430+0000 7f3dbeffd640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3dac0a2c10 0x7f3dac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.430+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3da80cebe0 con 0x7f3dc0102780 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.430+0000 7f3dbeffd640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3dac0a2c10 0x7f3dac0a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005e10 tx=0x7f3db0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:35.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.433+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3da809b790 con 0x7f3dc0102780 2024-09-10T10:45:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.634+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc0199670 con 0x7f3dc0102780 2024-09-10T10:45:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.638+0000 7f3dbcff9640 1 -- 172.21.15.143:0/2844539077 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f3dc0199670 con 0x7f3dc0102780 2024-09-10T10:45:35.644 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:35.645 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3dac0a2c10 msgr2=0x7f3dac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3dac0a2c10 0x7f3dac0a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005e10 tx=0x7f3db0005da0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 msgr2=0x7f3dc0198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 0x7f3dc0198850 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b4a0 tx=0x7f3da800b960 comp rx=0 tx=0).stop 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 shutdown_connections 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3dac0a2c10 0x7f3dac0a50d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc0106f00 0x7f3dc019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dc0102780 0x7f3dc0198850 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 --2- 172.21.15.143:0/2844539077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3dc0101dd0 0x7f3dc0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 >> 172.21.15.143:0/2844539077 conn(0x7f3dc00fd890 msgr2=0x7f3dc00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 shutdown_connections 2024-09-10T10:45:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:35.641+0000 7f3dc5599640 1 -- 172.21.15.143:0/2844539077 wait complete. 2024-09-10T10:45:35.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:35 smithi178 ceph-mon[34813]: pgmap v367: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:45:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:35 smithi144 ceph-mon[34230]: pgmap v367: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:45:36.019 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:35 smithi143 ceph-mon[31057]: pgmap v367: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:45:36.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:45:36.621 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:36.887 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:36 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2844539077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:36.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:36 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2844539077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:36 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2844539077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 -- 172.21.15.143:0/2341350843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac100e20 msgr2=0x7f0eac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/2341350843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac100e20 0x7f0eac101200 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009900 tx=0x7f0ea802a480 comp rx=0 tx=0).stop 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 -- 172.21.15.143:0/2341350843 shutdown_connections 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/2341350843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eac102ef0 0x7f0eac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/2341350843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eac102570 0x7f0eac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/2341350843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac100e20 0x7f0eac101200 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 -- 172.21.15.143:0/2341350843 >> 172.21.15.143:0/2341350843 conn(0x7f0eac0fc7d0 msgr2=0x7f0eac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 -- 172.21.15.143:0/2341350843 shutdown_connections 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.117+0000 7f0eb42a5640 1 -- 172.21.15.143:0/2341350843 wait complete. 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 Processor -- start 2024-09-10T10:45:37.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 -- start start 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eac100e20 0x7f0eac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eac102570 0x7f0eac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 0x7f0eac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac110e30 con 0x7f0eac102ef0 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac110cb0 con 0x7f0eac102570 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb42a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac110fb0 con 0x7f0eac100e20 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb2842640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 0x7f0eac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb2842640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 0x7f0eac19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37028/0 (socket says 172.21.15.143:37028) 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.118+0000 7f0eb2842640 1 -- 172.21.15.143:0/3356507760 learned_addr learned my addr 172.21.15.143:0/3356507760 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb2842640 1 -- 172.21.15.143:0/3356507760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eac100e20 msgr2=0x7f0eac198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb2842640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eac100e20 0x7f0eac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb2842640 1 -- 172.21.15.143:0/3356507760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eac102570 msgr2=0x7f0eac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb2842640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eac102570 0x7f0eac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb2842640 1 -- 172.21.15.143:0/3356507760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19d3c0 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb2842640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 0x7f0eac19cce0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400eaa0 tx=0x7f0ea400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400cdb0 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ea4005020 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac19d6b0 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40116d0 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.119+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a4f90 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.120+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac101200 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.121+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f0ea40051d0 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.121+0000 7f0ea37fe640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0e980a2c90 0x7f0e980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.122+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0ea40cf780 con 0x7f0eac102ef0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.122+0000 7f0eb2041640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0e980a2c90 0x7f0e980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.122+0000 7f0eb2041640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0e980a2c90 0x7f0e980a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80096c0 tx=0x7f0ea8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.126+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ea4016030 con 0x7f0eac102ef0 2024-09-10T10:45:37.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.293+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eac1a51c0 con 0x7f0eac102ef0 2024-09-10T10:45:37.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.295+0000 7f0ea37fe640 1 -- 172.21.15.143:0/3356507760 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7f0ea409c220 con 0x7f0eac102ef0 2024-09-10T10:45:37.296 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:37.296 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0e980a2c90 msgr2=0x7f0e980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0e980a2c90 0x7f0e980a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80096c0 tx=0x7f0ea8000bc0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 msgr2=0x7f0eac19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 0x7f0eac19cce0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400eaa0 tx=0x7f0ea400ef60 comp rx=0 tx=0).stop 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 shutdown_connections 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0e980a2c90 0x7f0e980a5150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eac102ef0 0x7f0eac19cce0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eac102570 0x7f0eac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 --2- 172.21.15.143:0/3356507760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eac100e20 0x7f0eac198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 >> 172.21.15.143:0/3356507760 conn(0x7f0eac0fc7d0 msgr2=0x7f0eac06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:37.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 shutdown_connections 2024-09-10T10:45:37.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:37.297+0000 7f0eb42a5640 1 -- 172.21.15.143:0/3356507760 wait complete. 2024-09-10T10:45:37.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:37 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3356507760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:37.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:37 smithi178 ceph-mon[34813]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:45:37.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:37 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3356507760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:37.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:37 smithi144 ceph-mon[34230]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:45:37.937 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:37 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3356507760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:37.937 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:37 smithi143 ceph-mon[31057]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:45:37.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T10:45:38.215 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:39.412 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:39 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:45:39.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:39 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:45:39.440 INFO:teuthology.orchestra.run.smithi143.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T10:45:39.441 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T10:45:39.441 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T10:45:39.441 DEBUG:tasks.quiescer:Yielding 2024-09-10T10:45:39.441 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T10:45:39.455 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:45:39.456 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T10:45:39.456 INFO:tasks.workunit:timeout=3h 2024-09-10T10:45:39.456 INFO:tasks.workunit:cleanup=True 2024-09-10T10:45:39.456 DEBUG:teuthology.orchestra.run.smithi143:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:39.457 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:39.473 INFO:teuthology.orchestra.run.smithi143.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-09-10 10:43:36.147546490 +0000 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-09-10 10:43:36.147546490 +0000 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-09-10 10:45:16.497953220 +0000 2024-09-10T10:45:39.474 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2024-09-10T10:45:39.474 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:39.475 DEBUG:teuthology.orchestra.run.smithi143:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T10:45:39.545 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:45:39.601 INFO:tasks.workunit.client.0.smithi143.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T10:45:39.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:39 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:45:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 -- 172.21.15.143:0/912002883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0784103700 msgr2=0x7f078410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 --2- 172.21.15.143:0/912002883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0784103700 0x7f078410bbf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0778009900 tx=0x7f077802a480 comp rx=0 tx=0).stop 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 -- 172.21.15.143:0/912002883 shutdown_connections 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 --2- 172.21.15.143:0/912002883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f078410c180 0x7f078410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 --2- 172.21.15.143:0/912002883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0784103700 0x7f078410bbf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 --2- 172.21.15.143:0/912002883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 0x7f07841031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 -- 172.21.15.143:0/912002883 >> 172.21.15.143:0/912002883 conn(0x7f07840fc7a0 msgr2=0x7f07840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:40.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 -- 172.21.15.143:0/912002883 shutdown_connections 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.212+0000 7f078aa38640 1 -- 172.21.15.143:0/912002883 wait complete. 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 Processor -- start 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 -- start start 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 0x7f07841983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0784103700 0x7f07841988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 0x7f078419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0784111de0 con 0x7f078410c180 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0784111c60 con 0x7f0784102de0 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f078aa38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0784111f60 con 0x7f0784103700 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 0x7f078419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 0x7f078419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37050/0 (socket says 172.21.15.143:37050) 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 -- 172.21.15.143:0/3117214878 learned_addr learned my addr 172.21.15.143:0/3117214878 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f07837fe640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0784103700 0x7f07841988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 -- 172.21.15.143:0/3117214878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0784103700 msgr2=0x7f07841988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0784103700 0x7f07841988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0783fff640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 0x7f07841983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 -- 172.21.15.143:0/3117214878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 msgr2=0x7f07841983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 0x7f07841983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0788fd5640 1 -- 172.21.15.143:0/3117214878 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078419d350 con 0x7f078410c180 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.213+0000 7f0783fff640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 0x7f07841983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.214+0000 7f0788fd5640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 0x7f078419cc70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f077c00cad0 tx=0x7f077c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.214+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c00ae20 con 0x7f078410c180 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.214+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f077c005020 con 0x7f078410c180 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.214+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078419d640 con 0x7f078410c180 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.214+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c00f6b0 con 0x7f078410c180 2024-09-10T10:45:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.214+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07841a4f20 con 0x7f078410c180 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.215+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07841031c0 con 0x7f078410c180 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.216+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f077c0051d0 con 0x7f078410c180 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.217+0000 7f07817fa640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f076c0a2c10 0x7f076c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.217+0000 7f0783fff640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f076c0a2c10 0x7f076c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.217+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f077c0cf5d0 con 0x7f078410c180 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.217+0000 7f0783fff640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f076c0a2c10 0x7f076c0a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0774005d00 tx=0x7f0774009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.221+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f077c014030 con 0x7f078410c180 2024-09-10T10:45:40.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.425+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f078402d7c0 con 0x7f078410c180 2024-09-10T10:45:40.427 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:40 smithi143 ceph-mon[31057]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:45:40.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.426+0000 7f07817fa640 1 -- 172.21.15.143:0/3117214878 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f077c09c070 con 0x7f078410c180 2024-09-10T10:45:40.428 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:45:40.428 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:45:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f076c0a2c10 msgr2=0x7f076c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f076c0a2c10 0x7f076c0a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0774005d00 tx=0x7f0774009550 comp rx=0 tx=0).stop 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 msgr2=0x7f078419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 0x7f078419cc70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f077c00cad0 tx=0x7f077c00cf90 comp rx=0 tx=0).stop 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 shutdown_connections 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f076c0a2c10 0x7f076c0a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078410c180 0x7f078419cc70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0784103700 0x7f07841988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 --2- 172.21.15.143:0/3117214878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0784102de0 0x7f07841983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 >> 172.21.15.143:0/3117214878 conn(0x7f07840fc7a0 msgr2=0x7f07840fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 shutdown_connections 2024-09-10T10:45:40.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:40.430+0000 7f078aa38640 1 -- 172.21.15.143:0/3117214878 wait complete. 2024-09-10T10:45:40.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:40 smithi144 ceph-mon[34230]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:45:40.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:40 smithi178 ceph-mon[34813]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:45:41.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:42:53.956292+0000', 'modified': '2024-09-10T10:45:16.857221+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24502, 'mds_1': 24485, 'mds_2': 24481, 'mds_3': 24475, 'mds_4': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6839/2518540467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2518540467}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2518540467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6841/1083221625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6840', 'nonce': 1083221625}, {'type': 'v1', 'addr': '172.21.15.143:6841', 'nonce': 1083221625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3121359191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3121359191}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3121359191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1016684638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1016684638}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1016684638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3190468681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3190468681}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3190468681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.144:6837/3174266316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 3174266316}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 3174266316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.178:6835/261851614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 261851614}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 261851614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.178:6837/2330516637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2330516637}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2330516637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.144:6839/1994626034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 1994626034}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 1994626034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.178:6839/774166191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 774166191}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 774166191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24502, 'qdb_cluster': [24458, 24475, 24481, 24485, 24502]} 2024-09-10T10:45:41.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T10:45:41.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T10:45:41.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:41 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3117214878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:41.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:41 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3117214878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:41.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:41 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3117214878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.868+0000 7f0580fdb640 1 -- 172.21.15.143:0/3745976500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074590 msgr2=0x7f057c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.868+0000 7f0580fdb640 1 --2- 172.21.15.143:0/3745976500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074590 0x7f057c074970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f056c009900 tx=0x7f056c02a480 comp rx=0 tx=0).stop 2024-09-10T10:45:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 -- 172.21.15.143:0/3745976500 shutdown_connections 2024-09-10T10:45:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 --2- 172.21.15.143:0/3745976500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c112b70 0x7f057c114f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 --2- 172.21.15.143:0/3745976500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c074eb0 0x7f057c112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 --2- 172.21.15.143:0/3745976500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074590 0x7f057c074970 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 -- 172.21.15.143:0/3745976500 >> 172.21.15.143:0/3745976500 conn(0x7f057c0fd880 msgr2=0x7f057c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 -- 172.21.15.143:0/3745976500 shutdown_connections 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 -- 172.21.15.143:0/3745976500 wait complete. 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 Processor -- start 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.869+0000 7f0580fdb640 1 -- start start 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0580fdb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c074590 0x7f057c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0580fdb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 0x7f057c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0580fdb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c112b70 0x7f057c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0580fdb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f057c118400 con 0x7f057c074eb0 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0580fdb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f057c118280 con 0x7f057c074590 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0580fdb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f057c118580 con 0x7f057c112b70 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 0x7f057c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 0x7f057c1a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37060/0 (socket says 172.21.15.143:37060) 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 -- 172.21.15.143:0/2566461147 learned_addr learned my addr 172.21.15.143:0/2566461147 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f057a59c640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c074590 0x7f057c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f057ad9d640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c112b70 0x7f057c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 -- 172.21.15.143:0/2566461147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c112b70 msgr2=0x7f057c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c112b70 0x7f057c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 -- 172.21.15.143:0/2566461147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c074590 msgr2=0x7f057c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c074590 0x7f057c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 -- 172.21.15.143:0/2566461147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c19efd0 con 0x7f057c074eb0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0571d9b640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 0x7f057c1a4bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f056400c970 tx=0x7f056400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.870+0000 7f0573fff640 1 -- 172.21.15.143:0/2566461147 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564007bb0 con 0x7f057c074eb0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.871+0000 7f0573fff640 1 -- 172.21.15.143:0/2566461147 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0564007d50 con 0x7f057c074eb0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.871+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c19f2c0 con 0x7f057c074eb0 2024-09-10T10:45:41.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.871+0000 7f0573fff640 1 -- 172.21.15.143:0/2566461147 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056400f450 con 0x7f057c074eb0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.871+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c1ab510 con 0x7f057c074eb0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.873+0000 7f0573fff640 1 -- 172.21.15.143:0/2566461147 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f0564004f20 con 0x7f057c074eb0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.873+0000 7f05833b4740 1 Processor -- start 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.873+0000 7f05833b4740 1 -- start start 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f05833b4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f659d0 0x55e941f65db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f05833b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f05833b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f72fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f05833b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e941ecc090 con 0x55e941f70bb0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f05833b4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e941e73a70 con 0x55e941f659d0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f05833b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e941ec8050 con 0x55e941f662f0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f72fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f72fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37076/0 (socket says 172.21.15.143:37076) 2024-09-10T10:45:41.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 -- 172.21.15.143:0/4129942725 learned_addr learned my addr 172.21.15.143:0/4129942725 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f057ad9d640 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f659d0 0x55e941f65db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0573fff640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f05680a2c10 0x7f05680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f057a59c640 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f70670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f057ad9d640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f05680a2c10 0x7f05680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 -- 172.21.15.143:0/4129942725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 msgr2=0x55e941f70670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f70670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 -- 172.21.15.143:0/4129942725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f659d0 msgr2=0x55e941f65db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f659d0 0x55e941f65db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0571d9b640 1 -- 172.21.15.143:0/4129942725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e941e80130 con 0x55e941f70bb0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.874+0000 7f0573fff640 1 -- 172.21.15.143:0/2566461147 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f05640cf200 con 0x7f057c074eb0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f0571d9b640 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f72fa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f05640cc790 tx=0x7f056401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414f5bf87d0208ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f057ad9d640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f05680a2c10 0x7f05680a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f057400a2e0 tx=0x7f0574004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f0570d99640 1 -- 172.21.15.143:0/4129942725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640a4210 con 0x55e941f70bb0 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f0570d99640 1 -- 172.21.15.143:0/4129942725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f056409b650 con 0x55e941f70bb0 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f0570d99640 1 -- 172.21.15.143:0/4129942725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564099950 con 0x55e941f70bb0 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f05833b4740 1 -- 172.21.15.143:0/4129942725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 msgr2=0x55e941f72fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f05833b4740 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f72fa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f05640cc790 tx=0x7f056401f000 comp rx=0 tx=0).stop 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f05833b4740 1 -- 172.21.15.143:0/4129942725 shutdown_connections 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f05833b4740 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f72fa0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f05833b4740 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f70670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.875+0000 7f05833b4740 1 --2- 172.21.15.143:0/4129942725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f659d0 0x55e941f65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- 172.21.15.143:0/4129942725 >> 172.21.15.143:0/4129942725 conn(0x55e941f61f60 msgr2=0x55e941f62340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- 172.21.15.143:0/4129942725 shutdown_connections 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- 172.21.15.143:0/4129942725 wait complete. 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- start start 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 Processor -- start 2024-09-10T10:45:41.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 0x55e941f70410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e941f73780 con 0x55e941f659d0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e941e86f20 con 0x55e941f70bb0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e941f73900 con 0x55e941f662f0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f05833b4740 10 client.0 _ll_register_callbacks cb 0x55e941e50030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f057a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 0x55e941f70410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f057a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 0x55e941f70410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37080/0 (socket says 172.21.15.143:37080) 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.876+0000 7f057a59c640 1 -- 172.21.15.143:0/1598716323 learned_addr learned my addr 172.21.15.143:0/1598716323 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0571d9b640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057ad9d640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057a59c640 1 -- 172.21.15.143:0/1598716323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 msgr2=0x55e941f6d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057a59c640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f6d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057a59c640 1 -- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 msgr2=0x55e941f6dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057a59c640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057a59c640 1 -- 172.21.15.143:0/1598716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e941c3d2b0 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f057a59c640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 0x55e941f70410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f056c009900 tx=0x7f056c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd9bcdc62972523 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0561ffb640 1 -- 172.21.15.143:0/1598716323 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c035040 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0561ffb640 1 -- 172.21.15.143:0/1598716323 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f056c004040 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056409b650 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0561ffb640 1 -- 172.21.15.143:0/1598716323 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f056c004240 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f05833b4740 10 client.15465 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e941f7f070 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0561ffb640 1 -- 172.21.15.143:0/1598716323 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f056409b650 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f0561ffb640 1 -- 172.21.15.143:0/1598716323 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f056c046600 con 0x55e941f659d0 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.877+0000 7f05833b4740 10 client.15465 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:41.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f05833b4740 10 client.15465 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f05833b4740 20 client.15465 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f05833b4740 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941f82b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f05833b4740 4 client.15465 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e941ed9600 con 0x55e941f816d0 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f0571d9b640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941f82b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f0561ffb640 10 client.15465 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.878+0000 7f0571d9b640 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941f82b40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7c01f8eedb4e83 server_cookie=f04bdfa1496211ce in_seq=0 out_seq=0 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f0561ffb640 1 -- 172.21.15.143:0/1598716323 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e941ed9600 con 0x55e941f816d0 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f0561ffb640 10 client.15465 handle_command_reply: tid=0 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 client.15465 shutdown 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 msgr2=0x55e941f70410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 0x55e941f70410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f056c009900 tx=0x7f056c0316d0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 shutdown_connections 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941f82b40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f662f0 0x55e941f6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f0561ffb640 0 client.15465 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 --2- 172.21.15.143:0/1598716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f659d0 0x55e941f70410 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 >> 172.21.15.143:0/1598716323 conn(0x55e941f61f60 msgr2=0x55e941f67c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 shutdown_connections 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 1 -- 172.21.15.143:0/1598716323 wait complete. 2024-09-10T10:45:41.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.879+0000 7f05833b4740 20 client.15465 trim_cache size 0 max 16384 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.897+0000 7f05833b4740 1 Processor -- start 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.897+0000 7f05833b4740 1 -- start start 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f05833b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f82b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f05833b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 0x55e941f6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f05833b4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f05833b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e941f80e80 con 0x55e941f662f0 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f05833b4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e941f81550 con 0x55e941f70bb0 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f05833b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e941f81c20 con 0x55e941f659d0 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 0x55e941f6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 0x55e941f6a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37094/0 (socket says 172.21.15.143:37094) 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 -- 172.21.15.143:0/1777393115 learned_addr learned my addr 172.21.15.143:0/1777393115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f057a59c640 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f82b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 -- 172.21.15.143:0/1777393115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 msgr2=0x55e941f82b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f82b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 -- 172.21.15.143:0/1777393115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 msgr2=0x55e941f6af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 -- 172.21.15.143:0/1777393115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e941f70030 con 0x55e941f662f0 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0571d9b640 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 0x55e941f6a9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f056401e000 tx=0x7f05640861d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbccf69bcb743cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0563fff640 1 -- 172.21.15.143:0/1777393115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640cce60 con 0x55e941f662f0 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0563fff640 1 -- 172.21.15.143:0/1777393115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05640a4210 con 0x55e941f662f0 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.898+0000 7f0563fff640 1 -- 172.21.15.143:0/1777393115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640ccaa0 con 0x55e941f662f0 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 -- 172.21.15.143:0/1777393115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 msgr2=0x55e941f6a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 0x55e941f6a9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f056401e000 tx=0x7f05640861d0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 -- 172.21.15.143:0/1777393115 shutdown_connections 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f70bb0 0x55e941f6af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f662f0 0x55e941f6a9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 --2- 172.21.15.143:0/1777393115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f82b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 -- 172.21.15.143:0/1777393115 >> 172.21.15.143:0/1777393115 conn(0x55e941f61f60 msgr2=0x55e941f6c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 -- 172.21.15.143:0/1777393115 shutdown_connections 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 -- 172.21.15.143:0/1777393115 wait complete. 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.899+0000 7f05833b4740 1 -- start start 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 Processor -- start 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f662f0 0x55e941f84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f84890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e941f829c0 con 0x55e941f70bb0 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e941f822f0 con 0x55e941f662f0 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e941f83090 con 0x55e941f659d0 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f05833b4740 10 client.0 _ll_register_callbacks cb 0x7f056c031d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f84890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f84890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37104/0 (socket says 172.21.15.143:37104) 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 -- 172.21.15.143:0/3898967246 learned_addr learned my addr 172.21.15.143:0/3898967246 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057a59c640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f0571d9b640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f662f0 0x55e941f84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 -- 172.21.15.143:0/3898967246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 msgr2=0x55e941f83e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f83e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 -- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f662f0 msgr2=0x55e941f84350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f662f0 0x55e941f84350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 -- 172.21.15.143:0/3898967246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e941fe9f80 con 0x55e941f70bb0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f057ad9d640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f84890 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0574016040 tx=0x7f057400a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01d3ad1bd8fed14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.900+0000 7f0561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f0561ffb640 1 -- 172.21.15.143:0/3898967246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740119f0 con 0x55e941f70bb0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f0561ffb640 1 -- 172.21.15.143:0/3898967246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0574011b90 con 0x55e941f70bb0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05640a4210 con 0x55e941f70bb0 2024-09-10T10:45:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f0561ffb640 1 -- 172.21.15.143:0/3898967246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0574011d90 con 0x55e941f70bb0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e941feb2e0 con 0x55e941f70bb0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f0561ffb640 1 -- 172.21.15.143:0/3898967246 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05640a4210 con 0x55e941f70bb0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f0561ffb640 1 -- 172.21.15.143:0/3898967246 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f057401e520 con 0x55e941f70bb0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 10 client.15471 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 20 client.15471 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941feebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 4 client.15471 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "status"}) -- 0x55e941ed3b10 con 0x55e941f816d0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.901+0000 7f057a59c640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941feebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f057a59c640 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941feebf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e00e9c9a3e652d2 server_cookie=e087813dbe518cb9 in_seq=0 out_seq=0 2024-09-10T10:45:41.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f0561ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f0561ffb640 1 -- 172.21.15.143:0/3898967246 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e941ed3b10 con 0x55e941f816d0 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f0561ffb640 10 client.15471 handle_command_reply: tid=0 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 client.15471 shutdown 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 msgr2=0x55e941f84890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f84890 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0574016040 tx=0x7f057400a990 comp rx=0 tx=0).stop 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 shutdown_connections 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55e941f816d0 0x55e941feebf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e941f70bb0 0x55e941f84890 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e941f662f0 0x55e941f84350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 --2- 172.21.15.143:0/3898967246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e941f659d0 0x55e941f83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 >> 172.21.15.143:0/3898967246 conn(0x55e941f61f60 msgr2=0x55e941f68de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f0561ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:45:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 shutdown_connections 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 1 -- 172.21.15.143:0/3898967246 wait complete. 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.902+0000 7f05833b4740 20 client.15471 trim_cache size 0 max 16384 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 4, 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24458, 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:41.912 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 130.360699122, 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 29, 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 190.39687914300001 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:41.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f05680a2c10 msgr2=0x7f05680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f05680a2c10 0x7f05680a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f057400a2e0 tx=0x7f0574004ab0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 msgr2=0x7f057c1a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 0x7f057c1a4bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f056400c970 tx=0x7f056400ce30 comp rx=0 tx=0).stop 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 shutdown_connections 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f05680a2c10 0x7f05680a50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f057c112b70 0x7f057c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057c074eb0 0x7f057c1a4bf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 --2- 172.21.15.143:0/2566461147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c074590 0x7f057c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 >> 172.21.15.143:0/2566461147 conn(0x7f057c0fd880 msgr2=0x7f057c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 shutdown_connections 2024-09-10T10:45:41.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:41.903+0000 7f0580fdb640 1 -- 172.21.15.143:0/2566461147 wait complete. 2024-09-10T10:45:42.499 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:42 smithi143 ceph-mon[31057]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:45:42.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:42 smithi178 ceph-mon[34813]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:45:42.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:42 smithi144 ceph-mon[34230]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:45:42.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T10:45:42.967 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:43.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- 172.21.15.143:0/2322790893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc102790 msgr2=0x7fc9dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/2322790893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc102790 0x7fc9dc1067e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009900 tx=0x7fc9cc02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:43.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- 172.21.15.143:0/2322790893 shutdown_connections 2024-09-10T10:45:43.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/2322790893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc106f10 0x7fc9dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/2322790893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc102790 0x7fc9dc1067e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/2322790893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- 172.21.15.143:0/2322790893 >> 172.21.15.143:0/2322790893 conn(0x7fc9dc0fd8c0 msgr2=0x7fc9dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:43.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- 172.21.15.143:0/2322790893 shutdown_connections 2024-09-10T10:45:43.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- 172.21.15.143:0/2322790893 wait complete. 2024-09-10T10:45:43.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 Processor -- start 2024-09-10T10:45:43.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- start start 2024-09-10T10:45:43.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9dc102790 0x7fc9dc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc106f10 0x7fc9dc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc111e20 con 0x7fc9dc101de0 2024-09-10T10:45:43.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc111ca0 con 0x7fc9dc102790 2024-09-10T10:45:43.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.506+0000 7fc9e2eff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc111fa0 con 0x7fc9dc106f10 2024-09-10T10:45:43.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37130/0 (socket says 172.21.15.143:37130) 2024-09-10T10:45:43.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/1209724834 learned_addr learned my addr 172.21.15.143:0/1209724834 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:43.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e149c640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc106f10 0x7fc9dc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9dc102790 0x7fc9dc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/1209724834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc106f10 msgr2=0x7fc9dc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc106f10 0x7fc9dc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/1209724834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9dc102790 msgr2=0x7fc9dc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9dc102790 0x7fc9dc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/1209724834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc19d370 con 0x7fc9dc101de0 2024-09-10T10:45:43.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1983c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800e9f0 tx=0x7fc9d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:43.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9d1ffb640 1 -- 172.21.15.143:0/1209724834 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800cd80 con 0x7fc9dc101de0 2024-09-10T10:45:43.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9d1ffb640 1 -- 172.21.15.143:0/1209724834 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9d80108e0 con 0x7fc9dc101de0 2024-09-10T10:45:43.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc19d660 con 0x7fc9dc101de0 2024-09-10T10:45:43.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.507+0000 7fc9d1ffb640 1 -- 172.21.15.143:0/1209724834 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8018920 con 0x7fc9dc101de0 2024-09-10T10:45:43.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.508+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc1a4f40 con 0x7fc9dc101de0 2024-09-10T10:45:43.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9d1ffb640 1 -- 172.21.15.143:0/1209724834 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc9d8010a90 con 0x7fc9dc101de0 2024-09-10T10:45:43.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 Processor -- start 2024-09-10T10:45:43.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 -- start start 2024-09-10T10:45:43.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9d1ffb640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9c40a2c10 0x7fc9c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bbe8109090 con 0x55bbe81a2890 2024-09-10T10:45:43.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bbe80b0a70 con 0x55bbe81b1fc0 2024-09-10T10:45:43.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbe8105050 con 0x55bbe81a31b0 2024-09-10T10:45:43.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e149c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e149c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37136/0 (socket says 172.21.15.143:37136) 2024-09-10T10:45:43.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.510+0000 7fc9e149c640 1 -- 172.21.15.143:0/74960847 learned_addr learned my addr 172.21.15.143:0/74960847 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:43.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9c40a2c10 0x7fc9c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9d1ffb640 1 -- 172.21.15.143:0/1209724834 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc9d80cf800 con 0x7fc9dc101de0 2024-09-10T10:45:43.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 -- 172.21.15.143:0/74960847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 msgr2=0x55bbe81b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 -- 172.21.15.143:0/74960847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 msgr2=0x55bbe81b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 -- 172.21.15.143:0/74960847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe80bd130 con 0x55bbe81a2890 2024-09-10T10:45:43.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:43.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:43.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9e149c640 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81a2c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4014a60 tx=0x7fc9d4014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc01c2053ed2103 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:43.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9c2ffd640 1 -- 172.21.15.143:0/74960847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d401ec50 con 0x55bbe81a2890 2024-09-10T10:45:43.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9c2ffd640 1 -- 172.21.15.143:0/74960847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9d4023070 con 0x55bbe81a2890 2024-09-10T10:45:43.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.511+0000 7fc9c2ffd640 1 -- 172.21.15.143:0/74960847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4005670 con 0x55bbe81a2890 2024-09-10T10:45:43.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 -- 172.21.15.143:0/74960847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 msgr2=0x55bbe81a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81a2c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4014a60 tx=0x7fc9d4014f20 comp rx=0 tx=0).stop 2024-09-10T10:45:43.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e149c640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9c40a2c10 0x7fc9c40a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4022000 tx=0x7fc9d401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:43.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 -- 172.21.15.143:0/74960847 shutdown_connections 2024-09-10T10:45:43.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/74960847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81a2c70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 -- 172.21.15.143:0/74960847 >> 172.21.15.143:0/74960847 conn(0x55bbe819ef60 msgr2=0x55bbe819f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 -- 172.21.15.143:0/74960847 shutdown_connections 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 -- 172.21.15.143:0/74960847 wait complete. 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 -- start start 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.512+0000 7fc9e52d8740 1 Processor -- start 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 0x55bbe81af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81b4b90 con 0x55bbe81b1fc0 2024-09-10T10:45:43.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bbe80c3f20 con 0x55bbe81a2890 2024-09-10T10:45:43.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81b4d10 con 0x55bbe81a31b0 2024-09-10T10:45:43.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e52d8740 10 client.0 _ll_register_callbacks cb 0x55bbe808d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:43.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 0x55bbe81af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 0x55bbe81af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37146/0 (socket says 172.21.15.143:37146) 2024-09-10T10:45:43.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 -- 172.21.15.143:0/2992291625 learned_addr learned my addr 172.21.15.143:0/2992291625 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:43.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 -- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 msgr2=0x55bbe81aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 -- 172.21.15.143:0/2992291625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 msgr2=0x55bbe81b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 -- 172.21.15.143:0/2992291625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe7e7a2b0 con 0x55bbe81b1fc0 2024-09-10T10:45:43.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9e149c640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 0x55bbe81af050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40128d0 tx=0x7fc9d4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d2321e548ec5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:43.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.513+0000 7fc9bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:43.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9bbfff640 1 -- 172.21.15.143:0/2992291625 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4037060 con 0x55bbe81b1fc0 2024-09-10T10:45:43.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d4023070 con 0x55bbe81b1fc0 2024-09-10T10:45:43.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9bbfff640 1 -- 172.21.15.143:0/2992291625 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc9d401ec50 con 0x55bbe81b1fc0 2024-09-10T10:45:43.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9bbfff640 1 -- 172.21.15.143:0/2992291625 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9d403e050 con 0x55bbe81b1fc0 2024-09-10T10:45:43.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 10 client.15483 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:43.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bbe81c0420 con 0x55bbe81b1fc0 2024-09-10T10:45:43.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9bbfff640 1 -- 172.21.15.143:0/2992291625 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9d4023070 con 0x55bbe81b1fc0 2024-09-10T10:45:43.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9bbfff640 1 -- 172.21.15.143:0/2992291625 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc9d400f040 con 0x55bbe81b1fc0 2024-09-10T10:45:43.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 10 client.15483 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:43.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 10 client.15483 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T10:45:43.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 20 client.15483 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:43.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe81c26d0 0x55bbe81c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 4 client.15483 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:43.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.514+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe8116600 con 0x55bbe81c26d0 2024-09-10T10:45:43.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.515+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe81c26d0 0x55bbe81c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.515+0000 7fc9bbfff640 10 client.15483 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:43.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.515+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe81c26d0 0x55bbe81c3b40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c65151b2f06f44b server_cookie=94b12e92260e9044 in_seq=0 out_seq=0 2024-09-10T10:45:43.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9bbfff640 1 -- 172.21.15.143:0/2992291625 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbe8116600 con 0x55bbe81c26d0 2024-09-10T10:45:43.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9bbfff640 10 client.15483 handle_command_reply: tid=0 2024-09-10T10:45:43.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 client.15483 shutdown 2024-09-10T10:45:43.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 msgr2=0x55bbe81af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 0x55bbe81af050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40128d0 tx=0x7fc9d4005050 comp rx=0 tx=0).stop 2024-09-10T10:45:43.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 shutdown_connections 2024-09-10T10:45:43.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe81c26d0 0x55bbe81c3b40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81b1fc0 0x55bbe81af050 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/2992291625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 >> 172.21.15.143:0/2992291625 conn(0x55bbe819ef60 msgr2=0x55bbe81a4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:43.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9bbfff640 0 client.15483 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:43.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 shutdown_connections 2024-09-10T10:45:43.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 1 -- 172.21.15.143:0/2992291625 wait complete. 2024-09-10T10:45:43.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.516+0000 7fc9e52d8740 20 client.15483 trim_cache size 0 max 16384 2024-09-10T10:45:43.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.534+0000 7fc9e52d8740 1 Processor -- start 2024-09-10T10:45:43.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.534+0000 7fc9e52d8740 1 -- start start 2024-09-10T10:45:43.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81c1e80 con 0x55bbe81a2890 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81c2550 con 0x55bbe81b1fc0 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81c2c20 con 0x55bbe81a31b0 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81c26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37160/0 (socket says 172.21.15.143:37160) 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/4028663458 learned_addr learned my addr 172.21.15.143:0/4028663458 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e149c640 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/4028663458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 msgr2=0x55bbe81c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/4028663458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 msgr2=0x55bbe81c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 -- 172.21.15.143:0/4028663458 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe81ad060 con 0x55bbe81a2890 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e149c640 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81c26d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc9d809bbe0 tx=0x7fc9d80cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e4d163414dec8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9c27fc640 1 -- 172.21.15.143:0/4028663458 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80b3070 con 0x55bbe81a2890 2024-09-10T10:45:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9c27fc640 1 -- 172.21.15.143:0/4028663458 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9d809bc50 con 0x55bbe81a2890 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.535+0000 7fc9c27fc640 1 -- 172.21.15.143:0/4028663458 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d809a780 con 0x55bbe81a2890 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 -- 172.21.15.143:0/4028663458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 msgr2=0x55bbe81c26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81c26d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc9d809bbe0 tx=0x7fc9d80cf040 comp rx=0 tx=0).stop 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 -- 172.21.15.143:0/4028663458 shutdown_connections 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81a31b0 0x55bbe81c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/4028663458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a2890 0x55bbe81c26d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 -- 172.21.15.143:0/4028663458 >> 172.21.15.143:0/4028663458 conn(0x55bbe819ef60 msgr2=0x55bbe81a9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 -- 172.21.15.143:0/4028663458 shutdown_connections 2024-09-10T10:45:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.536+0000 7fc9e52d8740 1 -- 172.21.15.143:0/4028663458 wait complete. 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 -- start start 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 Processor -- start 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 0x55bbe81c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81c39c0 con 0x55bbe81a31b0 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81c32f0 con 0x55bbe81a2890 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbe81c4090 con 0x55bbe81b1fc0 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e52d8740 10 client.0 _ll_register_callbacks cb 0x7fc9d4014860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 0x55bbe81c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 0x55bbe81c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37172/0 (socket says 172.21.15.143:37172) 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9d3fff640 1 -- 172.21.15.143:0/327714896 learned_addr learned my addr 172.21.15.143:0/327714896 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.537+0000 7fc9e149c640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9d3fff640 1 -- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81b1fc0 msgr2=0x55bbe81c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9d3fff640 1 -- 172.21.15.143:0/327714896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 msgr2=0x55bbe81c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9d3fff640 1 -- 172.21.15.143:0/327714896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe822ae00 con 0x55bbe81a31b0 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9d3fff640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 0x55bbe81c5060 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc02a930 tx=0x7fc9cc0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13599911f13acab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9bbfff640 1 -- 172.21.15.143:0/327714896 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc035040 con 0x55bbe81a31b0 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9bbfff640 1 -- 172.21.15.143:0/327714896 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc9cc004450 con 0x55bbe81a31b0 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9d809bc50 con 0x55bbe81a31b0 2024-09-10T10:45:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9bbfff640 1 -- 172.21.15.143:0/327714896 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc050050 con 0x55bbe81a31b0 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9e52d8740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bbe822c110 con 0x55bbe81a31b0 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9bbfff640 1 -- 172.21.15.143:0/327714896 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9d809bc50 con 0x55bbe81a31b0 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.538+0000 7fc9bbfff640 1 -- 172.21.15.143:0/327714896 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc9cc004af0 con 0x55bbe81a31b0 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e52d8740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e52d8740 10 client.15489 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e52d8740 20 client.15489 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe8230d10 0x55bbe822fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e52d8740 4 client.15489 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "status"}) -- 0x55bbe8110b10 con 0x55bbe8230d10 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe8230d10 0x55bbe822fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9bbfff640 10 client.15489 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.539+0000 7fc9e0c9b640 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe8230d10 0x55bbe822fa20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5a7e662ed700924 server_cookie=1a1be82a53df3838 in_seq=0 out_seq=0 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9bbfff640 1 -- 172.21.15.143:0/327714896 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bbe8110b10 con 0x55bbe8230d10 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9bbfff640 10 client.15489 handle_command_reply: tid=0 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 client.15489 shutdown 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 msgr2=0x55bbe81c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 0x55bbe81c5060 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc02a930 tx=0x7fc9cc0317b0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 shutdown_connections 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55bbe8230d10 0x55bbe822fa20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9bbfff640 0 client.15489 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe81b1fc0 0x55bbe81c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe81a31b0 0x55bbe81c5060 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 --2- 172.21.15.143:0/327714896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe81a2890 0x55bbe81c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 >> 172.21.15.143:0/327714896 conn(0x55bbe819ef60 msgr2=0x55bbe81a4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 shutdown_connections 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 1 -- 172.21.15.143:0/327714896 wait complete. 2024-09-10T10:45:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.540+0000 7fc9e52d8740 20 client.15489 trim_cache size 0 max 16384 2024-09-10T10:45:43.576 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:43.576 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:43.576 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 3, 2024-09-10T10:45:43.576 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24475, 2024-09-10T10:45:43.576 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 133.00845501000001, 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 29, 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T10:45:43.577 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 197.30296115600001 2024-09-10T10:45:43.578 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9c40a2c10 msgr2=0x7fc9c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9c40a2c10 0x7fc9c40a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4022000 tx=0x7fc9d401e000 comp rx=0 tx=0).stop 2024-09-10T10:45:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 msgr2=0x7fc9dc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1983c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800e9f0 tx=0x7fc9d800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 shutdown_connections 2024-09-10T10:45:43.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9c40a2c10 0x7fc9c40a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc106f10 0x7fc9dc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9dc102790 0x7fc9dc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 --2- 172.21.15.143:0/1209724834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9dc101de0 0x7fc9dc1983c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:43.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 >> 172.21.15.143:0/1209724834 conn(0x7fc9dc0fd8c0 msgr2=0x7fc9dc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:43.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 shutdown_connections 2024-09-10T10:45:43.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:43.541+0000 7fc9e2eff640 1 -- 172.21.15.143:0/1209724834 wait complete. 2024-09-10T10:45:44.297 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T10:45:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:44.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:44 smithi143 ceph-mon[31057]: pgmap v371: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T10:45:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:44 smithi178 ceph-mon[34813]: pgmap v371: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T10:45:44.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:44 smithi144 ceph-mon[34230]: pgmap v371: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 -- 172.21.15.143:0/3216848177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c100a80 msgr2=0x7f415c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 --2- 172.21.15.143:0/3216848177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c100a80 0x7f415c100ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4144009930 tx=0x7f414402a420 comp rx=0 tx=0).stop 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 -- 172.21.15.143:0/3216848177 shutdown_connections 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 --2- 172.21.15.143:0/3216848177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 0x7f415c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 --2- 172.21.15.143:0/3216848177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c100a80 0x7f415c100ee0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 --2- 172.21.15.143:0/3216848177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c106a80 0x7f415c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 -- 172.21.15.143:0/3216848177 >> 172.21.15.143:0/3216848177 conn(0x7f415c0fc7a0 msgr2=0x7f415c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 -- 172.21.15.143:0/3216848177 shutdown_connections 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.033+0000 7f4162186640 1 -- 172.21.15.143:0/3216848177 wait complete. 2024-09-10T10:45:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 Processor -- start 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 -- start start 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 0x7f415c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 0x7f415c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c106a80 0x7f415c19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f415c110d30 con 0x7f415c106a80 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f415c110bb0 con 0x7f415c100a80 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4162186640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f415c110eb0 con 0x7f415c101420 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4153fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 0x7f415c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 0x7f415c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f415bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c106a80 0x7f415c19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 0x7f415c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:34590/0 (socket says 172.21.15.143:34590) 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f415b7fe640 1 -- 172.21.15.143:0/2078279508 learned_addr learned my addr 172.21.15.143:0/2078279508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.034+0000 7f4153fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 0x7f415c1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:39226/0 (socket says 172.21.15.143:39226) 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415bfff640 1 -- 172.21.15.143:0/2078279508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 msgr2=0x7f415c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415bfff640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 0x7f415c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415bfff640 1 -- 172.21.15.143:0/2078279508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 msgr2=0x7f415c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415bfff640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 0x7f415c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415bfff640 1 -- 172.21.15.143:0/2078279508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c19c220 con 0x7f415c106a80 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415b7fe640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 0x7f415c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f415bfff640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c106a80 0x7f415c19bb40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f415400e940 tx=0x7f415400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f41597fa640 1 -- 172.21.15.143:0/2078279508 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154004d20 con 0x7f415c106a80 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f41597fa640 1 -- 172.21.15.143:0/2078279508 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4154005020 con 0x7f415c106a80 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f41597fa640 1 -- 172.21.15.143:0/2078279508 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41540116d0 con 0x7f415c106a80 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c19c510 con 0x7f415c106a80 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.035+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c1a3df0 con 0x7f415c106a80 2024-09-10T10:45:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.037+0000 7f41597fa640 1 -- 172.21.15.143:0/2078279508 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f415400cb20 con 0x7f415c106a80 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.037+0000 7f416455f740 1 Processor -- start 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 -- start start 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef13fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef1e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562d6ee78090 con 0x562d6ef14500 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d6ee1fa70 con 0x562d6ef13be0 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f416455f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562d6ee74050 con 0x562d6ef1edc0 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef1e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef1e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37186/0 (socket says 172.21.15.143:37186) 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 -- 172.21.15.143:0/3105157481 learned_addr learned my addr 172.21.15.143:0/3105157481 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f4153fff640 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef13fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f41597fa640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41480a2c10 0x7f41480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f4153fff640 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 -- 172.21.15.143:0/3105157481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 msgr2=0x562d6ef211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 -- 172.21.15.143:0/3105157481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 msgr2=0x562d6ef13fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef13fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 -- 172.21.15.143:0/3105157481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6ee2c130 con 0x562d6ef14500 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f4153fff640 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef13fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415bfff640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41480a2c10 0x7f41480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f41597fa640 1 -- 172.21.15.143:0/2078279508 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f41540cf8f0 con 0x7f415c106a80 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.038+0000 7f415b7fe640 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef1e880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f414c010580 tx=0x7f414c010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479d163895c8a5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f4151ffb640 1 -- 172.21.15.143:0/3105157481 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c011820 con 0x562d6ef14500 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f4151ffb640 1 -- 172.21.15.143:0/3105157481 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f414c00e430 con 0x562d6ef14500 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f4151ffb640 1 -- 172.21.15.143:0/3105157481 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c01a5e0 con 0x562d6ef14500 2024-09-10T10:45:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 -- 172.21.15.143:0/3105157481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 msgr2=0x562d6ef1e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f415bfff640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41480a2c10 0x7f41480a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4154000f80 tx=0x7f415400ef30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef1e880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f414c010580 tx=0x7f414c010a40 comp rx=0 tx=0).stop 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 -- 172.21.15.143:0/3105157481 shutdown_connections 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef1e880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 --2- 172.21.15.143:0/3105157481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef13fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 -- 172.21.15.143:0/3105157481 >> 172.21.15.143:0/3105157481 conn(0x562d6ef10170 msgr2=0x562d6ef10550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 -- 172.21.15.143:0/3105157481 shutdown_connections 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.039+0000 7f416455f740 1 -- 172.21.15.143:0/3105157481 wait complete. 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 -- start start 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 Processor -- start 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef21990 con 0x562d6ef1edc0 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d6ee32f20 con 0x562d6ef13be0 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef21b10 con 0x562d6ef14500 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f416455f740 10 client.0 _ll_register_callbacks cb 0x562d6edfc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f4153fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f4153fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef1bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37202/0 (socket says 172.21.15.143:37202) 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f4153fff640 1 -- 172.21.15.143:0/1257704347 learned_addr learned my addr 172.21.15.143:0/1257704347 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f4153fff640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.040+0000 7f415b7fe640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f4153fff640 1 -- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 msgr2=0x562d6ef1b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f4153fff640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f4153fff640 1 -- 172.21.15.143:0/1257704347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 msgr2=0x562d6ef1e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f4153fff640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef1e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f4153fff640 1 -- 172.21.15.143:0/1257704347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6ebe92b0 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f4153fff640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef1bef0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f414402a940 tx=0x7f414402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8739029942d16179 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f413affd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f413affd640 1 -- 172.21.15.143:0/1257704347 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144035040 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f413affd640 1 -- 172.21.15.143:0/1257704347 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4144004230 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414c00e430 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f413affd640 1 -- 172.21.15.143:0/1257704347 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4144051050 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 10 client.15501 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d6ef2d2b0 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f413affd640 1 -- 172.21.15.143:0/1257704347 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f414c00e430 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f413affd640 1 -- 172.21.15.143:0/1257704347 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4144049020 con 0x562d6ef1edc0 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 10 client.15501 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:45.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 10 client.15501 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 20 client.15501 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef2f8c0 0x562d6ef30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 4 client.15501 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.041+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d6ee85600 con 0x562d6ef2f8c0 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.042+0000 7f415b7fe640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef2f8c0 0x562d6ef30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.042+0000 7f413affd640 10 client.15501 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.042+0000 7f415b7fe640 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef2f8c0 0x562d6ef30d30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=905398cc7f5d304 server_cookie=ab22858c2253fb33 in_seq=0 out_seq=0 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f413affd640 1 -- 172.21.15.143:0/1257704347 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d6ee85600 con 0x562d6ef2f8c0 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f413affd640 10 client.15501 handle_command_reply: tid=0 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 client.15501 shutdown 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 msgr2=0x562d6ef1bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef1bef0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f414402a940 tx=0x7f414402a420 comp rx=0 tx=0).stop 2024-09-10T10:45:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 shutdown_connections 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef2f8c0 0x562d6ef30d30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef1bef0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f413affd640 0 client.15501 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 --2- 172.21.15.143:0/1257704347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef1e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 >> 172.21.15.143:0/1257704347 conn(0x562d6ef10170 msgr2=0x562d6ef0d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 shutdown_connections 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 1 -- 172.21.15.143:0/1257704347 wait complete. 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.043+0000 7f416455f740 20 client.15501 trim_cache size 0 max 16384 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 Processor -- start 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 -- start start 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef16ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef2f070 con 0x562d6ef1edc0 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef2f740 con 0x562d6ef13be0 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f416455f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef2fe10 con 0x562d6ef14500 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef16ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef16ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37216/0 (socket says 172.21.15.143:37216) 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 -- 172.21.15.143:0/3657369948 learned_addr learned my addr 172.21.15.143:0/3657369948 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f415b7fe640 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 -- 172.21.15.143:0/3657369948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 msgr2=0x562d6ef1b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 -- 172.21.15.143:0/3657369948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 msgr2=0x562d6ef30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 -- 172.21.15.143:0/3657369948 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6ef1e280 con 0x562d6ef1edc0 2024-09-10T10:45:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.061+0000 7f4153fff640 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef16ee0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4144009890 tx=0x7f4144031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a568115a1105180 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f41517fa640 1 -- 172.21.15.143:0/3657369948 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144041070 con 0x562d6ef1edc0 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f41517fa640 1 -- 172.21.15.143:0/3657369948 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4144005850 con 0x562d6ef1edc0 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f41517fa640 1 -- 172.21.15.143:0/3657369948 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144035070 con 0x562d6ef1edc0 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 -- 172.21.15.143:0/3657369948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 msgr2=0x562d6ef16ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef16ee0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4144009890 tx=0x7f4144031f10 comp rx=0 tx=0).stop 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 -- 172.21.15.143:0/3657369948 shutdown_connections 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef1edc0 0x562d6ef16ee0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef14500 0x562d6ef1b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 --2- 172.21.15.143:0/3657369948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 -- 172.21.15.143:0/3657369948 >> 172.21.15.143:0/3657369948 conn(0x562d6ef10170 msgr2=0x562d6ef167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 -- 172.21.15.143:0/3657369948 shutdown_connections 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.062+0000 7f416455f740 1 -- 172.21.15.143:0/3657369948 wait complete. 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 -- start start 2024-09-10T10:45:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 Processor -- start 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef39560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef30bb0 con 0x562d6ef14500 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef304e0 con 0x562d6ef13be0 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562d6ef31280 con 0x562d6ef1edc0 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f416455f740 10 client.0 _ll_register_callbacks cb 0x7f4144009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f415b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37228/0 (socket says 172.21.15.143:37228) 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f415b7fe640 1 -- 172.21.15.143:0/1456328826 learned_addr learned my addr 172.21.15.143:0/1456328826 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.063+0000 7f4153fff640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef39560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f4153fff640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f415b7fe640 1 -- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 msgr2=0x562d6ef18a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f415b7fe640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef18a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f415b7fe640 1 -- 172.21.15.143:0/1456328826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 msgr2=0x562d6ef39560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f415b7fe640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef39560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f415b7fe640 1 -- 172.21.15.143:0/1456328826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6ef19070 con 0x562d6ef14500 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f4153fff640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef18a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f415b7fe640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef184f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f414c010580 tx=0x7f414c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465ea866478a2e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f413affd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f413affd640 1 -- 172.21.15.143:0/1456328826 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c005450 con 0x562d6ef14500 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f413affd640 1 -- 172.21.15.143:0/1456328826 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f414c0055f0 con 0x562d6ef14500 2024-09-10T10:45:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4144005850 con 0x562d6ef14500 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f413affd640 1 -- 172.21.15.143:0/1456328826 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f414c038050 con 0x562d6ef14500 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f416455f740 10 client.15507 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d6ef99fd0 con 0x562d6ef14500 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f413affd640 1 -- 172.21.15.143:0/1456328826 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4144005850 con 0x562d6ef14500 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.064+0000 7f413affd640 1 -- 172.21.15.143:0/1456328826 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f414c02e030 con 0x562d6ef14500 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 10 client.15507 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 20 client.15507 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef9e6b0 0x562d6ef9d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 4 client.15507 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "status"}) -- 0x562d6ee7fb10 con 0x562d6ef9e6b0 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f4153fff640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef9e6b0 0x562d6ef9d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f413affd640 10 client.15507 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f4153fff640 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef9e6b0 0x562d6ef9d3c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4d27bf2ba489809 server_cookie=fd3f6da8a97703c2 in_seq=0 out_seq=0 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f413affd640 1 -- 172.21.15.143:0/1456328826 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562d6ee7fb10 con 0x562d6ef9e6b0 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f413affd640 10 client.15507 handle_command_reply: tid=0 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.065+0000 7f416455f740 1 client.15507 shutdown 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 msgr2=0x562d6ef184f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef184f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f414c010580 tx=0x7f414c00ee00 comp rx=0 tx=0).stop 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 shutdown_connections 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562d6ef9e6b0 0x562d6ef9d3c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562d6ef1edc0 0x562d6ef18a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562d6ef14500 0x562d6ef184f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 --2- 172.21.15.143:0/1456328826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562d6ef13be0 0x562d6ef39560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 >> 172.21.15.143:0/1456328826 conn(0x562d6ef10170 msgr2=0x562d6ef16ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f413affd640 0 client.15507 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:45:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 shutdown_connections 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 1 -- 172.21.15.143:0/1456328826 wait complete. 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f416455f740 20 client.15507 trim_cache size 0 max 16384 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 2, 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24481, 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:45.083 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 135.54023870399999, 2024-09-10T10:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 29, 2024-09-10T10:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T10:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 196.22488958700001 2024-09-10T10:45:45.084 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41480a2c10 msgr2=0x7f41480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41480a2c10 0x7f41480a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4154000f80 tx=0x7f415400ef30 comp rx=0 tx=0).stop 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c106a80 msgr2=0x7f415c19bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c106a80 0x7f415c19bb40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f415400e940 tx=0x7f415400ee00 comp rx=0 tx=0).stop 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 shutdown_connections 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f41480a2c10 0x7f41480a50d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c106a80 0x7f415c19bb40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c101420 0x7f415c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.066+0000 7f4162186640 1 --2- 172.21.15.143:0/2078279508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f415c100a80 0x7f415c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.067+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 >> 172.21.15.143:0/2078279508 conn(0x7f415c0fc7a0 msgr2=0x7f415c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.067+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 shutdown_connections 2024-09-10T10:45:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:45.067+0000 7f4162186640 1 -- 172.21.15.143:0/2078279508 wait complete. 2024-09-10T10:45:46.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:45:46.249 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:46.654 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:46 smithi143 ceph-mon[31057]: pgmap v372: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.733+0000 7fd57f160640 1 -- 172.21.15.143:0/3643316016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 msgr2=0x7fd5781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.733+0000 7fd57f160640 1 --2- 172.21.15.143:0/3643316016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781067e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009900 tx=0x7fd56c02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- 172.21.15.143:0/3643316016 shutdown_connections 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 --2- 172.21.15.143:0/3643316016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578106f10 0x7fd57810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 --2- 172.21.15.143:0/3643316016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781067e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 --2- 172.21.15.143:0/3643316016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd578101de0 0x7fd5781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- 172.21.15.143:0/3643316016 >> 172.21.15.143:0/3643316016 conn(0x7fd5780fd8c0 msgr2=0x7fd5780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- 172.21.15.143:0/3643316016 shutdown_connections 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- 172.21.15.143:0/3643316016 wait complete. 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 Processor -- start 2024-09-10T10:45:46.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- start start 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578101de0 0x7fd5781983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd578106f10 0x7fd57819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd578111e20 con 0x7fd578102790 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd578111ca0 con 0x7fd578101de0 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd57f160640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd578111fa0 con 0x7fd578106f10 2024-09-10T10:45:46.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd577fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.734+0000 7fd577fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59988/0 (socket says 172.21.15.143:59988) 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 -- 172.21.15.143:0/2326073239 learned_addr learned my addr 172.21.15.143:0/2326073239 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd578106f10 0x7fd57819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd57cefc640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578101de0 0x7fd5781983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 -- 172.21.15.143:0/2326073239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd578106f10 msgr2=0x7fd57819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd578106f10 0x7fd57819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 -- 172.21.15.143:0/2326073239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578101de0 msgr2=0x7fd5781983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578101de0 0x7fd5781983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 -- 172.21.15.143:0/2326073239 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57819d350 con 0x7fd578102790 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd57cefc640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578101de0 0x7fd5781983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd577fff640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781988e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005940 tx=0x7fd56c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:46.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd575ffb640 1 -- 172.21.15.143:0/2326073239 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c03f030 con 0x7fd578102790 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd575ffb640 1 -- 172.21.15.143:0/2326073239 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd56c003aa0 con 0x7fd578102790 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd575ffb640 1 -- 172.21.15.143:0/2326073239 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c035070 con 0x7fd578102790 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57819d5e0 con 0x7fd578102790 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.735+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57819da70 con 0x7fd578102790 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.737+0000 7fd575ffb640 1 -- 172.21.15.143:0/2326073239 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd56c042030 con 0x7fd578102790 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 Processor -- start 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 -- start start 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd575ffb640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd5600a2c10 0x7fd5600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89a98d0 0x55c4b89a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 0x55c4b89bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c4b8910090 con 0x55c4b89b8fc0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4b88b7a70 con 0x55c4b89a98d0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd581539740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c4b890c050 con 0x55c4b89aa1f0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd5600a2c10 0x7fd5600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd575ffb640 1 -- 172.21.15.143:0/2326073239 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd56c0efcb0 con 0x7fd578102790 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd57d6fd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 0x55c4b89bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd57d6fd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 0x55c4b89bb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60004/0 (socket says 172.21.15.143:60004) 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.738+0000 7fd57d6fd640 1 -- 172.21.15.143:0/1409327365 learned_addr learned my addr 172.21.15.143:0/1409327365 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57cefc640 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89a98d0 0x55c4b89a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd577fff640 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 -- 172.21.15.143:0/1409327365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89aa1f0 msgr2=0x55c4b89b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 -- 172.21.15.143:0/1409327365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89a98d0 msgr2=0x55c4b89a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89a98d0 0x55c4b89a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 -- 172.21.15.143:0/1409327365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b88c4130 con 0x55c4b89b8fc0 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd5600a2c10 0x7fd5600a50d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd570011840 tx=0x7fd57000a9f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 0x55c4b89bb3b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd570005ac0 tx=0x7fd57001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277b279feee1000c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd55effd640 1 -- 172.21.15.143:0/1409327365 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570024070 con 0x55c4b89b8fc0 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd55effd640 1 -- 172.21.15.143:0/1409327365 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57000fe50 con 0x55c4b89b8fc0 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.739+0000 7fd55effd640 1 -- 172.21.15.143:0/1409327365 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700107d0 con 0x55c4b89b8fc0 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 -- 172.21.15.143:0/1409327365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 msgr2=0x55c4b89bb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 0x55c4b89bb3b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd570005ac0 tx=0x7fd57001b000 comp rx=0 tx=0).stop 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 -- 172.21.15.143:0/1409327365 shutdown_connections 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89b8fc0 0x55c4b89bb3b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 --2- 172.21.15.143:0/1409327365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89a98d0 0x55c4b89a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 -- 172.21.15.143:0/1409327365 >> 172.21.15.143:0/1409327365 conn(0x55c4b89a5f60 msgr2=0x55c4b89a6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 -- 172.21.15.143:0/1409327365 shutdown_connections 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 -- 172.21.15.143:0/1409327365 wait complete. 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 -- start start 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.740+0000 7fd581539740 1 Processor -- start 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89bbb90 con 0x55c4b89a98d0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4b88caf20 con 0x55c4b89aa1f0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89bbd10 con 0x55c4b89b8fc0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 10 client.0 _ll_register_callbacks cb 0x55c4b8894030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60008/0 (socket says 172.21.15.143:60008) 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 -- 172.21.15.143:0/4050654541 learned_addr learned my addr 172.21.15.143:0/4050654541 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd577fff640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 -- 172.21.15.143:0/4050654541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 msgr2=0x55c4b89b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 -- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 msgr2=0x55c4b89b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 -- 172.21.15.143:0/4050654541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b86812b0 con 0x55c4b89a98d0 2024-09-10T10:45:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd57cefc640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89b87d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd5680126b0 tx=0x7fd568012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb1420094070c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd557fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd557fff640 1 -- 172.21.15.143:0/4050654541 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568010dc0 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57000fe50 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.741+0000 7fd557fff640 1 -- 172.21.15.143:0/4050654541 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd56801a3e0 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd557fff640 1 -- 172.21.15.143:0/4050654541 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd56801a5e0 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4b89c74c0 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd557fff640 1 -- 172.21.15.143:0/4050654541 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57000fe50 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd557fff640 1 -- 172.21.15.143:0/4050654541 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd568019070 con 0x55c4b89a98d0 2024-09-10T10:45:46.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 10 client.15519 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 20 client.15519 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b89c9b30 0x55c4b89cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 4 client.15519 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4b891d600 con 0x55c4b89c9b30 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.742+0000 7fd577fff640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b89c9b30 0x55c4b89cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd577fff640 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b89c9b30 0x55c4b89cafa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8217218770dce93 server_cookie=e09e5f50f891774e in_seq=0 out_seq=0 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd557fff640 10 client.15519 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd557fff640 1 -- 172.21.15.143:0/4050654541 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4b891d600 con 0x55c4b89c9b30 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd557fff640 10 client.15519 handle_command_reply: tid=0 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd581539740 1 client.15519 shutdown 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 msgr2=0x55c4b89b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.743+0000 7fd581539740 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89b87d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd5680126b0 tx=0x7fd568012b70 comp rx=0 tx=0).stop 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 shutdown_connections 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b89c9b30 0x55c4b89cafa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 --2- 172.21.15.143:0/4050654541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89b87d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd557fff640 0 client.15519 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 >> 172.21.15.143:0/4050654541 conn(0x55c4b89a5f60 msgr2=0x55c4b89abcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 shutdown_connections 2024-09-10T10:45:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 1 -- 172.21.15.143:0/4050654541 wait complete. 2024-09-10T10:45:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.744+0000 7fd581539740 20 client.15519 trim_cache size 0 max 16384 2024-09-10T10:45:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 Processor -- start 2024-09-10T10:45:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 -- start start 2024-09-10T10:45:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89a98d0 0x55c4b89b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89c92e0 con 0x55c4b89aa1f0 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89c99b0 con 0x55c4b89b8fc0 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd581539740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89ca080 con 0x55c4b89a98d0 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60020/0 (socket says 172.21.15.143:60020) 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 -- 172.21.15.143:0/566596764 learned_addr learned my addr 172.21.15.143:0/566596764 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd57cefc640 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89a98d0 0x55c4b89b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 -- 172.21.15.143:0/566596764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89a98d0 msgr2=0x55c4b89b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89a98d0 0x55c4b89b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 -- 172.21.15.143:0/566596764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89b8fc0 msgr2=0x55c4b89b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 -- 172.21.15.143:0/566596764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b89a5d90 con 0x55c4b89aa1f0 2024-09-10T10:45:46.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd57cefc640 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89a98d0 0x55c4b89b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:46.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.762+0000 7fd577fff640 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b0f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0b84b0 tx=0x7fd56c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa82df2f62165d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:46.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd55e7fc640 1 -- 172.21.15.143:0/566596764 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c049040 con 0x55c4b89aa1f0 2024-09-10T10:45:46.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd55e7fc640 1 -- 172.21.15.143:0/566596764 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd56c0bc050 con 0x55c4b89aa1f0 2024-09-10T10:45:46.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd55e7fc640 1 -- 172.21.15.143:0/566596764 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c03bd60 con 0x55c4b89aa1f0 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 -- 172.21.15.143:0/566596764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 msgr2=0x55c4b89b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b0f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0b84b0 tx=0x7fd56c0dc040 comp rx=0 tx=0).stop 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 -- 172.21.15.143:0/566596764 shutdown_connections 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89b8fc0 0x55c4b89b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89aa1f0 0x55c4b89b0f60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 --2- 172.21.15.143:0/566596764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89a98d0 0x55c4b89b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 -- 172.21.15.143:0/566596764 >> 172.21.15.143:0/566596764 conn(0x55c4b89a5f60 msgr2=0x55c4b89b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 -- 172.21.15.143:0/566596764 shutdown_connections 2024-09-10T10:45:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.763+0000 7fd581539740 1 -- 172.21.15.143:0/566596764 wait complete. 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 -- start start 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 Processor -- start 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89cae20 con 0x55c4b89a98d0 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89ca750 con 0x55c4b89aa1f0 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c4b89cb4f0 con 0x55c4b89b8fc0 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd581539740 10 client.0 _ll_register_callbacks cb 0x7fd568009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.764+0000 7fd57cefc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60034/0 (socket says 172.21.15.143:60034) 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 -- 172.21.15.143:0/1848618314 learned_addr learned my addr 172.21.15.143:0/1848618314 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd577fff640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57d6fd640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 -- 172.21.15.143:0/1848618314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 msgr2=0x55c4b89ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 -- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 msgr2=0x55c4b89cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 -- 172.21.15.143:0/1848618314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b89bbd00 con 0x55c4b89a98d0 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd57cefc640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89cc2c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd568010e70 tx=0x7fd568010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bd4accce01f618 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd557fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd557fff640 1 -- 172.21.15.143:0/1848618314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56802d020 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd56c0bc050 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd557fff640 1 -- 172.21.15.143:0/1848618314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd56801b650 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd557fff640 1 -- 172.21.15.143:0/1848618314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd568038050 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd581539740 10 client.15525 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.765+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4b8a33450 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd557fff640 1 -- 172.21.15.143:0/1848618314 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd56c0bc050 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd557fff640 1 -- 172.21.15.143:0/1848618314 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd568019050 con 0x55c4b89a98d0 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd581539740 10 client.15525 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd581539740 10 client.15525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd581539740 20 client.15525 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd581539740 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b8a38070 0x55c4b8a36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd581539740 4 client.15525 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "status"}) -- 0x55c4b8917b10 con 0x55c4b8a38070 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.766+0000 7fd577fff640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b8a38070 0x55c4b8a36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd557fff640 10 client.15525 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd577fff640 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b8a38070 0x55c4b8a36d80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e1d8ebad4955399 server_cookie=1e7fb124a88f84a2 in_seq=0 out_seq=0 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd557fff640 1 -- 172.21.15.143:0/1848618314 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55c4b8917b10 con 0x55c4b8a38070 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd557fff640 10 client.15525 handle_command_reply: tid=0 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 client.15525 shutdown 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 msgr2=0x55c4b89cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89cc2c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd568010e70 tx=0x7fd568010ea0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 shutdown_connections 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4b89b8fc0 0x55c4b89ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55c4b8a38070 0x55c4b8a36d80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b89aa1f0 0x55c4b89cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 --2- 172.21.15.143:0/1848618314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c4b89a98d0 0x55c4b89cc2c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 >> 172.21.15.143:0/1848618314 conn(0x55c4b89a5f60 msgr2=0x55c4b89b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd557fff640 0 client.15525 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 shutdown_connections 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 1 -- 172.21.15.143:0/1848618314 wait complete. 2024-09-10T10:45:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.767+0000 7fd581539740 20 client.15525 trim_cache size 0 max 16384 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24485, 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 138.24980979, 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 29, 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:46.776 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T10:45:46.777 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 183.729809343 2024-09-10T10:45:46.777 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:46.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd5600a2c10 msgr2=0x7fd5600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd5600a2c10 0x7fd5600a50d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd570011840 tx=0x7fd57000a9f0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 msgr2=0x7fd5781988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781988e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005940 tx=0x7fd56c004270 comp rx=0 tx=0).stop 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 shutdown_connections 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd5600a2c10 0x7fd5600a50d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd578106f10 0x7fd57819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd578102790 0x7fd5781988e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 --2- 172.21.15.143:0/2326073239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd578101de0 0x7fd5781983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 >> 172.21.15.143:0/2326073239 conn(0x7fd5780fd8c0 msgr2=0x7fd5780fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 shutdown_connections 2024-09-10T10:45:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:46.768+0000 7fd57f160640 1 -- 172.21.15.143:0/2326073239 wait complete. 2024-09-10T10:45:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:46 smithi178 ceph-mon[34813]: pgmap v372: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T10:45:46.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:46 smithi144 ceph-mon[34230]: pgmap v372: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T10:45:47.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T10:45:47.775 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 -- 172.21.15.143:0/3125666028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44101420 msgr2=0x7ffa4410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 --2- 172.21.15.143:0/3125666028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44101420 0x7ffa4410d950 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009cd0 tx=0x7ffa3c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 -- 172.21.15.143:0/3125666028 shutdown_connections 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 --2- 172.21.15.143:0/3125666028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44101420 0x7ffa4410d950 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 --2- 172.21.15.143:0/3125666028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa44100a80 0x7ffa44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 --2- 172.21.15.143:0/3125666028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa44106a80 0x7ffa44106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 -- 172.21.15.143:0/3125666028 >> 172.21.15.143:0/3125666028 conn(0x7ffa440fc7a0 msgr2=0x7ffa440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 -- 172.21.15.143:0/3125666028 shutdown_connections 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 -- 172.21.15.143:0/3125666028 wait complete. 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 Processor -- start 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.310+0000 7ffa4a720640 1 -- start start 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa4a720640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa44100a80 0x7ffa441991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa4a720640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa44101420 0x7ffa44199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa4a720640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 0x7ffa441a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa4a720640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44110d30 con 0x7ffa44106a80 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa4a720640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44110bb0 con 0x7ffa44101420 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa4a720640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44110eb0 con 0x7ffa44100a80 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 0x7ffa441a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 0x7ffa441a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60054/0 (socket says 172.21.15.143:60054) 2024-09-10T10:45:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 -- 172.21.15.143:0/1250984899 learned_addr learned my addr 172.21.15.143:0/1250984899 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa43fff640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa44100a80 0x7ffa441991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa437fe640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa44101420 0x7ffa44199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 -- 172.21.15.143:0/1250984899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa44100a80 msgr2=0x7ffa441991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa44100a80 0x7ffa441991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 -- 172.21.15.143:0/1250984899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa44101420 msgr2=0x7ffa44199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa44101420 0x7ffa44199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 -- 172.21.15.143:0/1250984899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441a47a0 con 0x7ffa44106a80 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.311+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 0x7ffa441a4260 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005cc0 tx=0x7ffa3c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.312+0000 7ffa417fa640 1 -- 172.21.15.143:0/1250984899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004450 con 0x7ffa44106a80 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.312+0000 7ffa417fa640 1 -- 172.21.15.143:0/1250984899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c031c90 con 0x7ffa44106a80 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.312+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa441a4970 con 0x7ffa44106a80 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.312+0000 7ffa417fa640 1 -- 172.21.15.143:0/1250984899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c03b400 con 0x7ffa44106a80 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.312+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa441a4e20 con 0x7ffa44106a80 2024-09-10T10:45:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.314+0000 7ffa417fa640 1 -- 172.21.15.143:0/1250984899 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7ffa3c004680 con 0x7ffa44106a80 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.314+0000 7ffa4caf9740 1 Processor -- start 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.314+0000 7ffa4caf9740 1 -- start start 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c757854c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757863a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa417fa640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa2c0a2c10 0x7ffa2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757863fc0 0x55c7578663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c7577bb090 con 0x55c7578551b0 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c757762a70 con 0x55c757854890 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c7577b7050 con 0x55c757863fc0 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757863a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa2c0a2c10 0x7ffa2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c757854c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757863a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60060/0 (socket says 172.21.15.143:60060) 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa43fff640 1 -- 172.21.15.143:0/474568383 learned_addr learned my addr 172.21.15.143:0/474568383 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa417fa640 1 -- 172.21.15.143:0/1250984899 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffa3c02e030 con 0x7ffa44106a80 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757863fc0 0x55c7578663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 -- 172.21.15.143:0/474568383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757863fc0 msgr2=0x55c7578663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757863fc0 0x55c7578663b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 -- 172.21.15.143:0/474568383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 msgr2=0x55c757863a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757863a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 -- 172.21.15.143:0/474568383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c75776f130 con 0x55c757854890 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa43fff640 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757863a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa2c0a2c10 0x7ffa2c0a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800f730 tx=0x7ffa3801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.315+0000 7ffa437fe640 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c757854c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa38005fd0 tx=0x7ffa38012e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af7eb24b8fdeed87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa2a7fc640 1 -- 172.21.15.143:0/474568383 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38022030 con 0x55c757854890 2024-09-10T10:45:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa2a7fc640 1 -- 172.21.15.143:0/474568383 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa38012510 con 0x55c757854890 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa2a7fc640 1 -- 172.21.15.143:0/474568383 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004b40 con 0x55c757854890 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 -- 172.21.15.143:0/474568383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 msgr2=0x55c757854c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c757854c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa38005fd0 tx=0x7ffa38012e80 comp rx=0 tx=0).stop 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 -- 172.21.15.143:0/474568383 shutdown_connections 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757863fc0 0x55c7578663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757863a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/474568383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c757854c70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 -- 172.21.15.143:0/474568383 >> 172.21.15.143:0/474568383 conn(0x55c757850f60 msgr2=0x55c757851340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 -- 172.21.15.143:0/474568383 shutdown_connections 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.316+0000 7ffa4caf9740 1 -- 172.21.15.143:0/474568383 wait complete. 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 -- start start 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 Processor -- start 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757863820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757860b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c7578610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c757866ad0 con 0x55c7578551b0 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c757775f20 con 0x55c757863fc0 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c757866c50 con 0x55c757854890 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa4caf9740 10 client.0 _ll_register_callbacks cb 0x55c75773f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757860b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757860b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60062/0 (socket says 172.21.15.143:60062) 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa437fe640 1 -- 172.21.15.143:0/396609335 learned_addr learned my addr 172.21.15.143:0/396609335 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.317+0000 7ffa43fff640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757863820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c7578610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa437fe640 1 -- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 msgr2=0x55c757863820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa437fe640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757863820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa437fe640 1 -- 172.21.15.143:0/396609335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 msgr2=0x55c7578610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa437fe640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c7578610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa437fe640 1 -- 172.21.15.143:0/396609335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c75752c2b0 con 0x55c7578551b0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c7578610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa437fe640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757860b70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffa38010010 tx=0x7ffa3800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d1e03e0c6be68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa237fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa237fe640 1 -- 172.21.15.143:0/396609335 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380052d0 con 0x55c7578551b0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa237fe640 1 -- 172.21.15.143:0/396609335 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffa38005470 con 0x55c7578551b0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa38012510 con 0x55c7578551b0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa237fe640 1 -- 172.21.15.143:0/396609335 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa38004040 con 0x55c7578551b0 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa4caf9740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c7578723f0 con 0x55c7578551b0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.318+0000 7ffa237fe640 1 -- 172.21.15.143:0/396609335 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa38012510 con 0x55c7578551b0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa237fe640 1 -- 172.21.15.143:0/396609335 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ffa380035f0 con 0x55c7578551b0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa4caf9740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa4caf9740 10 client.15537 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa4caf9740 20 client.15537 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578748f0 0x55c757875d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa4caf9740 4 client.15537 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7577c8600 con 0x55c7578748f0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa43fff640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578748f0 0x55c757875d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa237fe640 10 client.15537 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.319+0000 7ffa43fff640 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578748f0 0x55c757875d60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c68d35dba774680 server_cookie=d0d7aa93baa42fd8 in_seq=0 out_seq=0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa237fe640 1 -- 172.21.15.143:0/396609335 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7577c8600 con 0x55c7578748f0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa237fe640 10 client.15537 handle_command_reply: tid=0 2024-09-10T10:45:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 client.15537 shutdown 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 msgr2=0x55c757860b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757860b70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffa38010010 tx=0x7ffa3800ba90 comp rx=0 tx=0).stop 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 shutdown_connections 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c7578610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757860b70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578748f0 0x55c757875d60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/396609335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757863820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa237fe640 0 client.15537 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 >> 172.21.15.143:0/396609335 conn(0x55c757850f60 msgr2=0x55c757856cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 shutdown_connections 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 1 -- 172.21.15.143:0/396609335 wait complete. 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.320+0000 7ffa4caf9740 20 client.15537 trim_cache size 0 max 16384 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 Processor -- start 2024-09-10T10:45:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 -- start start 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757874220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757875690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c75785ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c7578740a0 con 0x55c7578551b0 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c757874770 con 0x55c757863fc0 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c757874e40 con 0x55c757854890 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757875690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757875690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60074/0 (socket says 172.21.15.143:60074) 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 -- 172.21.15.143:0/1674518325 learned_addr learned my addr 172.21.15.143:0/1674518325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa43fff640 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757874220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c75785ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 -- 172.21.15.143:0/1674518325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 msgr2=0x55c757874220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757874220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 -- 172.21.15.143:0/1674518325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 msgr2=0x55c75785ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c75785ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 -- 172.21.15.143:0/1674518325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c757851bf0 con 0x55c7578551b0 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c75785ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.338+0000 7ffa437fe640 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757875690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffa380039e0 tx=0x7ffa38005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfea29bd8d3324a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa29ffb640 1 -- 172.21.15.143:0/1674518325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38012510 con 0x55c7578551b0 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa29ffb640 1 -- 172.21.15.143:0/1674518325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa380126c0 con 0x55c7578551b0 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa29ffb640 1 -- 172.21.15.143:0/1674518325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38005330 con 0x55c7578551b0 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 -- 172.21.15.143:0/1674518325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 msgr2=0x55c757875690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757875690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffa380039e0 tx=0x7ffa38005fd0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 -- 172.21.15.143:0/1674518325 shutdown_connections 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757863fc0 0x55c75785ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c7578551b0 0x55c757875690 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/1674518325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c757854890 0x55c757874220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 -- 172.21.15.143:0/1674518325 >> 172.21.15.143:0/1674518325 conn(0x55c757850f60 msgr2=0x55c7578738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 -- 172.21.15.143:0/1674518325 shutdown_connections 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.339+0000 7ffa4caf9740 1 -- 172.21.15.143:0/1674518325 wait complete. 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 -- start start 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 Processor -- start 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c75785cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7578551b0 0x55c75785d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 0x55c75785da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c757875be0 con 0x55c757863fc0 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c757875510 con 0x55c757854890 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c7578762b0 con 0x55c7578551b0 2024-09-10T10:45:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa4caf9740 10 client.0 _ll_register_callbacks cb 0x7ffa3400b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 0x55c75785da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 0x55c75785da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60084/0 (socket says 172.21.15.143:60084) 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 -- 172.21.15.143:0/3023659617 learned_addr learned my addr 172.21.15.143:0/3023659617 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa437fe640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7578551b0 0x55c75785d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 -- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7578551b0 msgr2=0x55c75785d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7578551b0 0x55c75785d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa43fff640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c75785cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 -- 172.21.15.143:0/3023659617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 msgr2=0x55c75785cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c75785cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.340+0000 7ffa48cbd640 1 -- 172.21.15.143:0/3023659617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c75787d290 con 0x55c757863fc0 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa437fe640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7578551b0 0x55c75785d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa48cbd640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 0x55c75785da20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0b9400 tx=0x7ffa3c0ca0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62291fb96b208368 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa237fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa237fe640 1 -- 172.21.15.143:0/3023659617 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0098b0 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa380126c0 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa237fe640 1 -- 172.21.15.143:0/3023659617 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ffa3c0305a0 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa237fe640 1 -- 172.21.15.143:0/3023659617 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa3c0bca30 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c75787e500 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa237fe640 1 -- 172.21.15.143:0/3023659617 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa380126c0 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa237fe640 1 -- 172.21.15.143:0/3023659617 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ffa3c04d020 con 0x55c757863fc0 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 10 client.15543 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 20 client.15543 populate_metadata read hostname 'smithi143' 2024-09-10T10:45:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578e32a0 0x55c7578e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 4 client.15543 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.341+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "status"}) -- 0x55c7577c2b10 con 0x55c7578e32a0 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa43fff640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578e32a0 0x55c7578e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa237fe640 10 client.15543 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa43fff640 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578e32a0 0x55c7578e1fb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e019837fd80fca server_cookie=d0ebd5841271aaf2 in_seq=0 out_seq=0 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa237fe640 1 -- 172.21.15.143:0/3023659617 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c7577c2b10 con 0x55c7578e32a0 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa237fe640 10 client.15543 handle_command_reply: tid=0 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa4caf9740 1 client.15543 shutdown 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 msgr2=0x55c75785da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 0x55c75785da20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0b9400 tx=0x7ffa3c0ca0b0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.342+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 shutdown_connections 2024-09-10T10:45:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c757863fc0 0x55c75785da20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c7578e32a0 0x55c7578e1fb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7578551b0 0x55c75785d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 --2- 172.21.15.143:0/3023659617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c757854890 0x55c75785cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 >> 172.21.15.143:0/3023659617 conn(0x55c757850f60 msgr2=0x55c75785ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa237fe640 0 client.15543 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 shutdown_connections 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 1 -- 172.21.15.143:0/3023659617 wait complete. 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4caf9740 20 client.15543 trim_cache size 0 max 16384 2024-09-10T10:45:48.361 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "34212b78-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 24502, 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 174.098129086, 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 29, 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 89, 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 194.246081051 2024-09-10T10:45:48.362 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa2c0a2c10 msgr2=0x7ffa2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa2c0a2c10 0x7ffa2c0a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800f730 tx=0x7ffa3801d000 comp rx=0 tx=0).stop 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 msgr2=0x7ffa441a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 0x7ffa441a4260 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005cc0 tx=0x7ffa3c003c20 comp rx=0 tx=0).stop 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 shutdown_connections 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ffa2c0a2c10 0x7ffa2c0a50d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa44106a80 0x7ffa441a4260 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa44101420 0x7ffa44199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 --2- 172.21.15.143:0/1250984899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa44100a80 0x7ffa441991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 >> 172.21.15.143:0/1250984899 conn(0x7ffa440fc7a0 msgr2=0x7ffa440fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 shutdown_connections 2024-09-10T10:45:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:45:48.343+0000 7ffa4a720640 1 -- 172.21.15.143:0/1250984899 wait complete. 2024-09-10T10:45:48.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:48 smithi143 ceph-mon[31057]: pgmap v373: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T10:45:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:48 smithi178 ceph-mon[34813]: pgmap v373: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T10:45:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:48 smithi144 ceph-mon[34230]: pgmap v373: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T10:45:49.115 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T10:45:50.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:50 smithi143 ceph-mon[31057]: pgmap v374: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T10:45:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:50 smithi178 ceph-mon[34813]: pgmap v374: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T10:45:50.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:50 smithi144 ceph-mon[34230]: pgmap v374: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T10:45:52.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:52 smithi143 ceph-mon[31057]: pgmap v375: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T10:45:52.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:52 smithi178 ceph-mon[34813]: pgmap v375: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T10:45:52.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:52 smithi144 ceph-mon[34230]: pgmap v375: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T10:45:54.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:54 smithi143 ceph-mon[31057]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:45:54.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:54 smithi178 ceph-mon[34813]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:45:54.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:54 smithi144 ceph-mon[34230]: pgmap v376: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:45:56.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:56 smithi143 ceph-mon[31057]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T10:45:56.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:56 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:56 smithi178 ceph-mon[34813]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T10:45:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:56 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:56 smithi144 ceph-mon[34230]: pgmap v377: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T10:45:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:56 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:45:57 smithi178 ceph-mon[34813]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T10:45:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:45:57 smithi144 ceph-mon[34230]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T10:45:58.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:45:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:45:57 smithi143 ceph-mon[31057]: pgmap v378: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T10:45:59.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:45:59.927 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 -- 172.21.15.143:0/3177738638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 msgr2=0x7f3cec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 --2- 172.21.15.143:0/3177738638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec101200 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009900 tx=0x7f3cdc02a480 comp rx=0 tx=0).stop 2024-09-10T10:46:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 -- 172.21.15.143:0/3177738638 shutdown_connections 2024-09-10T10:46:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 --2- 172.21.15.143:0/3177738638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cec102ef0 0x7f3cec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 --2- 172.21.15.143:0/3177738638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cec102570 0x7f3cec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 --2- 172.21.15.143:0/3177738638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec101200 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.433+0000 7f3cf360a640 1 -- 172.21.15.143:0/3177738638 >> 172.21.15.143:0/3177738638 conn(0x7f3cec0fc7d0 msgr2=0x7f3cec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.434+0000 7f3cf360a640 1 -- 172.21.15.143:0/3177738638 shutdown_connections 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.434+0000 7f3cf360a640 1 -- 172.21.15.143:0/3177738638 wait complete. 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 Processor -- start 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 -- start start 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cec102570 0x7f3cec1967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cec102ef0 0x7f3cec19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10fd30 con 0x7f3cec100e20 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10fbb0 con 0x7f3cec102ef0 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf360a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10feb0 con 0x7f3cec102570 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf13a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf13a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37590/0 (socket says 172.21.15.143:37590) 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf13a6640 1 -- 172.21.15.143:0/2899816307 learned_addr learned my addr 172.21.15.143:0/2899816307 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.435+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cec102570 0x7f3cec1967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cec102ef0 0x7f3cec19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf13a6640 1 -- 172.21.15.143:0/2899816307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cec102570 msgr2=0x7f3cec1967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf13a6640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cec102570 0x7f3cec1967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf13a6640 1 -- 172.21.15.143:0/2899816307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cec102ef0 msgr2=0x7f3cec19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf13a6640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cec102ef0 0x7f3cec19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf13a6640 1 -- 172.21.15.143:0/2899816307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19b220 con 0x7f3cec100e20 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf13a6640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec196270 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc006010 tx=0x7f3cdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3ce27fc640 1 -- 172.21.15.143:0/2899816307 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc03f070 con 0x7f3cec100e20 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3ce27fc640 1 -- 172.21.15.143:0/2899816307 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cdc003aa0 con 0x7f3cec100e20 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19b4b0 con 0x7f3cec100e20 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3ce27fc640 1 -- 172.21.15.143:0/2899816307 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc035030 con 0x7f3cec100e20 2024-09-10T10:46:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.436+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec1a2df0 con 0x7f3cec100e20 2024-09-10T10:46:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.439+0000 7f3cf59e3740 1 Processor -- start 2024-09-10T10:46:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.439+0000 7f3cf59e3740 1 -- start start 2024-09-10T10:46:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.439+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9ef0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.439+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9efb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.439+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9efda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.439+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5627b9e562e0 con 0x5627b9ef0dc0 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5627b9dfdce0 con 0x5627b9efb680 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5627b9e522a0 con 0x5627b9ef04a0 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9ef0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf0ba5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9efda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9ef0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:60190/0 (socket says 172.21.15.143:60190) 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/1169821403 learned_addr learned my addr 172.21.15.143:0/1169821403 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3ce27fc640 1 -- 172.21.15.143:0/2899816307 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3cdc042070 con 0x7f3cec100e20 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/1169821403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 msgr2=0x5627b9efda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9efda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/1169821403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 msgr2=0x5627b9efb140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9efb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.440+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/1169821403 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b9de2440 con 0x5627b9ef04a0 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9ef0880 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4008d40 tx=0x7f3ce400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bb722cc19c5ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3cd77fe640 1 -- 172.21.15.143:0/1169821403 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4014070 con 0x5627b9ef04a0 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3cd77fe640 1 -- 172.21.15.143:0/1169821403 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce4003b70 con 0x5627b9ef04a0 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3cd77fe640 1 -- 172.21.15.143:0/1169821403 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4010700 con 0x5627b9ef04a0 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3ce27fc640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3cd80a70a0 0x7f3cd80a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3cf13a6640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3cd80a70a0 0x7f3cd80a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.441+0000 7f3ce27fc640 1 -- 172.21.15.143:0/2899816307 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3cdc0efa70 con 0x7f3cec100e20 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 -- 172.21.15.143:0/1169821403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 msgr2=0x5627b9ef0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9ef0880 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4008d40 tx=0x7f3ce400e720 comp rx=0 tx=0).stop 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 -- 172.21.15.143:0/1169821403 shutdown_connections 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9efda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9efb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/1169821403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9ef0880 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 -- 172.21.15.143:0/1169821403 >> 172.21.15.143:0/1169821403 conn(0x5627b9eeca30 msgr2=0x5627b9eece10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf13a6640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3cd80a70a0 0x7f3cd80a9560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5627b9eed890 tx=0x7f3cdc08bcc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 -- 172.21.15.143:0/1169821403 shutdown_connections 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.442+0000 7f3cf59e3740 1 -- 172.21.15.143:0/1169821403 wait complete. 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 -- start start 2024-09-10T10:46:00.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 Processor -- start 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9efafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9ef8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9ef87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5627b9efe250 con 0x5627b9ef0dc0 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5627b9e11050 con 0x5627b9efb680 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5627b9efe3d0 con 0x5627b9ef04a0 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.443+0000 7f3cf59e3740 10 client.0 _ll_register_callbacks cb 0x5627b9e51160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf0ba5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9ef8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf0ba5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9ef8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37618/0 (socket says 172.21.15.143:37618) 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf0ba5640 1 -- 172.21.15.143:0/3892963227 learned_addr learned my addr 172.21.15.143:0/3892963227 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9efafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9ef87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/3892963227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 msgr2=0x5627b9ef87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9ef87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/3892963227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 msgr2=0x5627b9ef8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9ef8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/3892963227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b9bc73c0 con 0x5627b9ef04a0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9ef8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9efafe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4008ad0 tx=0x7f3ce40045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ea84e101cadb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.444+0000 7f3cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/3892963227 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4014070 con 0x5627b9ef04a0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/3892963227 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3ce4011d60 con 0x5627b9ef04a0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce4003b70 con 0x5627b9ef04a0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/3892963227 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ce402e050 con 0x5627b9ef04a0 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cf59e3740 10 client.25114 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627b9f09c50 con 0x5627b9ef04a0 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/3892963227 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce4003b70 con 0x5627b9ef04a0 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/3892963227 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3ce4003d20 con 0x5627b9ef04a0 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cf59e3740 10 client.25114 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cf59e3740 10 client.25114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.445+0000 7f3cf59e3740 20 client.25114 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.446+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f0b060 0x5627b9f0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.446+0000 7f3cf59e3740 4 client.25114 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.446+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b9e63850 con 0x5627b9f0b060 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.446+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f0b060 0x5627b9f0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.446+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f0b060 0x5627b9f0d450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd703c79b168011 server_cookie=2e2bb8923816c2dc in_seq=0 out_seq=0 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.446+0000 7f3cd4ff9640 10 client.25114 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/3892963227 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627b9e63850 con 0x5627b9f0b060 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cd4ff9640 10 client.25114 handle_command_reply: tid=0 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 client.25114 shutdown 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 msgr2=0x5627b9efafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9efafe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4008ad0 tx=0x7f3ce40045e0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 shutdown_connections 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9ef87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9ef8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f0b060 0x5627b9f0d450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/3892963227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9efafe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 >> 172.21.15.143:0/3892963227 conn(0x5627b9eeca30 msgr2=0x5627b9ef2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.447+0000 7f3cd4ff9640 0 client.25114 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.448+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 shutdown_connections 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.448+0000 7f3cf59e3740 1 -- 172.21.15.143:0/3892963227 wait complete. 2024-09-10T10:46:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.448+0000 7f3cf59e3740 20 client.25114 trim_cache size 0 max 16384 2024-09-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 Processor -- start 2024-09-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 -- start start 2024-09-10T10:46:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9f074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9f12b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9f130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1ac8e0 con 0x5627b9ef0dc0 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1ae420 con 0x5627b9efb680 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1acfb0 con 0x5627b9ef04a0 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf0ba5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9f12b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf0ba5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9f12b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37628/0 (socket says 172.21.15.143:37628) 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf0ba5640 1 -- 172.21.15.143:0/142985382 learned_addr learned my addr 172.21.15.143:0/142985382 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9f074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.465+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9f130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cf0ba5640 1 -- 172.21.15.143:0/142985382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 msgr2=0x5627b9f074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9f074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cf0ba5640 1 -- 172.21.15.143:0/142985382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 msgr2=0x5627b9f130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9f130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cf0ba5640 1 -- 172.21.15.143:0/142985382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b9ef67e0 con 0x5627b9ef0dc0 2024-09-10T10:46:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9f12b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800a5c0 tx=0x7f3ce800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35b26357ce93121 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cd6ffd640 1 -- 172.21.15.143:0/142985382 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8005220 con 0x5627b9ef0dc0 2024-09-10T10:46:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cd6ffd640 1 -- 172.21.15.143:0/142985382 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce8004c50 con 0x5627b9ef0dc0 2024-09-10T10:46:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.466+0000 7f3cd6ffd640 1 -- 172.21.15.143:0/142985382 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800fb20 con 0x5627b9ef0dc0 2024-09-10T10:46:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 -- 172.21.15.143:0/142985382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 msgr2=0x5627b9f12b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9f12b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800a5c0 tx=0x7f3ce800a2d0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 -- 172.21.15.143:0/142985382 shutdown_connections 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9efb680 0x5627b9f130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef0dc0 0x5627b9f12b90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/142985382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9ef04a0 0x5627b9f074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 -- 172.21.15.143:0/142985382 >> 172.21.15.143:0/142985382 conn(0x5627b9eeca30 msgr2=0x5627b9ef3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 -- 172.21.15.143:0/142985382 shutdown_connections 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 -- 172.21.15.143:0/142985382 wait complete. 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 -- start start 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.467+0000 7f3cf59e3740 1 Processor -- start 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 0x5627b9efebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9ef0dc0 0x5627b9eff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9efb680 0x5627b9f73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1add50 con 0x5627b9ef04a0 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1ad680 con 0x5627b9ef0dc0 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1aa8c0 con 0x5627b9efb680 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf59e3740 10 client.0 _ll_register_callbacks cb 0x7f3ce800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf1ba7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 0x5627b9efebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf1ba7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 0x5627b9efebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37630/0 (socket says 172.21.15.143:37630) 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/266458187 learned_addr learned my addr 172.21.15.143:0/266458187 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9ef0dc0 0x5627b9eff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.468+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9efb680 0x5627b9f73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9efb680 msgr2=0x5627b9f73b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9efb680 0x5627b9f73b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/266458187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9ef0dc0 msgr2=0x5627b9eff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9ef0dc0 0x5627b9eff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf1ba7640 1 -- 172.21.15.143:0/266458187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b9f74090 con 0x5627b9ef04a0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9ef0dc0 0x5627b9eff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf1ba7640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 0x5627b9efebb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4004260 tx=0x7f3ce4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d9fa79c8323f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/266458187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4014040 con 0x5627b9ef04a0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/266458187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3ce4004a00 con 0x5627b9ef04a0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce8004c50 con 0x5627b9ef04a0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/266458187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ce4021040 con 0x5627b9ef04a0 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf59e3740 10 client.15558 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.469+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627b9f753a0 con 0x5627b9ef04a0 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/266458187 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce8004c50 con 0x5627b9ef04a0 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/266458187 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3ce4003970 con 0x5627b9ef04a0 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf59e3740 10 client.15558 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf59e3740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf59e3740 20 client.15558 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f75a60 0x5627b9f77e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf59e3740 4 client.15558 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627b9e5dd60 con 0x5627b9f75a60 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f75a60 0x5627b9f77e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cf0ba5640 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f75a60 0x5627b9f77e80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4615f1737d1eda2 server_cookie=b70a43dea9b58468 in_seq=0 out_seq=0 2024-09-10T10:46:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.470+0000 7f3cd4ff9640 10 client.15558 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cd4ff9640 1 -- 172.21.15.143:0/266458187 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5627b9e5dd60 con 0x5627b9f75a60 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cd4ff9640 10 client.15558 handle_command_reply: tid=0 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 client.15558 shutdown 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 msgr2=0x5627b9efebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 0x5627b9efebb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4004260 tx=0x7f3ce4005d90 comp rx=0 tx=0).stop 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 shutdown_connections 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5627b9f75a60 0x5627b9f77e80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5627b9efb680 0x5627b9f73b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627b9ef0dc0 0x5627b9eff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 --2- 172.21.15.143:0/266458187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5627b9ef04a0 0x5627b9efebb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cd4ff9640 0 client.15558 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 >> 172.21.15.143:0/266458187 conn(0x5627b9eeca30 msgr2=0x5627b9eeb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 shutdown_connections 2024-09-10T10:46:00.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 1 -- 172.21.15.143:0/266458187 wait complete. 2024-09-10T10:46:00.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.471+0000 7f3cf59e3740 20 client.15558 trim_cache size 0 max 16384 2024-09-10T10:46:00.477 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:00.477 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:46:00.477 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:46:00.477 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:00.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3cd80a70a0 msgr2=0x7f3cd80a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3cd80a70a0 0x7f3cd80a9560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5627b9eed890 tx=0x7f3cdc08bcc0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 msgr2=0x7f3cec196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec196270 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc006010 tx=0x7f3cdc0042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 shutdown_connections 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3cd80a70a0 0x7f3cd80a9560 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cec102ef0 0x7f3cec19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cec102570 0x7f3cec1967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 --2- 172.21.15.143:0/2899816307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cec100e20 0x7f3cec196270 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 >> 172.21.15.143:0/2899816307 conn(0x7f3cec0fc7d0 msgr2=0x7f3cec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 shutdown_connections 2024-09-10T10:46:00.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:00.472+0000 7f3cf360a640 1 -- 172.21.15.143:0/2899816307 wait complete. 2024-09-10T10:46:00.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:00 smithi143 ceph-mon[31057]: pgmap v379: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-10T10:46:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:00 smithi178 ceph-mon[34813]: pgmap v379: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-10T10:46:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:00 smithi144 ceph-mon[34230]: pgmap v379: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-10T10:46:01.340 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:46:01.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:46:01.603 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 -- 172.21.15.143:0/548374545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 msgr2=0x7f1558101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 --2- 172.21.15.143:0/548374545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f1558101200 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1548009900 tx=0x7f154802a480 comp rx=0 tx=0).stop 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 -- 172.21.15.143:0/548374545 shutdown_connections 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 --2- 172.21.15.143:0/548374545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1558102ef0 0x7f155810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 --2- 172.21.15.143:0/548374545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1558102570 0x7f15581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 --2- 172.21.15.143:0/548374545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f1558101200 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 -- 172.21.15.143:0/548374545 >> 172.21.15.143:0/548374545 conn(0x7f15580fc7d0 msgr2=0x7f15580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.108+0000 7f155da40640 1 -- 172.21.15.143:0/548374545 shutdown_connections 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.109+0000 7f155da40640 1 -- 172.21.15.143:0/548374545 wait complete. 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.109+0000 7f155da40640 1 Processor -- start 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.109+0000 7f155da40640 1 -- start start 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f155da40640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f15581973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f155da40640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1558102570 0x7f15581978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f155da40640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1558102ef0 0x7f155819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f155da40640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1558110e50 con 0x7f1558100e20 2024-09-10T10:46:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f155da40640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1558110cd0 con 0x7f1558102ef0 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f155da40640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1558110fd0 con 0x7f1558102570 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f15581973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f15581973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37656/0 (socket says 172.21.15.143:37656) 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 -- 172.21.15.143:0/2153226942 learned_addr learned my addr 172.21.15.143:0/2153226942 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f15577fe640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1558102ef0 0x7f155819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f15567fc640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1558102570 0x7f15581978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 -- 172.21.15.143:0/2153226942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1558102570 msgr2=0x7f15581978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1558102570 0x7f15581978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 -- 172.21.15.143:0/2153226942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1558102ef0 msgr2=0x7f155819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1558102ef0 0x7f155819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 -- 172.21.15.143:0/2153226942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155819c360 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.110+0000 7f1556ffd640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f15581973b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1548006010 tx=0x7f15480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.111+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154803f070 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.111+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155819c5f0 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.111+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1548003aa0 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.111+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548035030 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.111+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1558069f70 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.112+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15581021b0 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.114+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1548042070 con 0x7f1558100e20 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.115+0000 7f155ca3e640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15440a2c10 0x7f15440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.115+0000 7f15567fc640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15440a2c10 0x7f15440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.115+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15480f00d0 con 0x7f1558100e20 2024-09-10T10:46:02.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.115+0000 7f15567fc640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15440a2c10 0x7f15440a50d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15580ffe50 tx=0x7f1540008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:02.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.117+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15480bcb70 con 0x7f1558100e20 2024-09-10T10:46:02.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.332+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1558101200 con 0x7f1558100e20 2024-09-10T10:46:02.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.333+0000 7f155ca3e640 1 -- 172.21.15.143:0/2153226942 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f1558101200 con 0x7f1558100e20 2024-09-10T10:46:02.334 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:46:02.335 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:46:02.335 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15440a2c10 msgr2=0x7f15440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15440a2c10 0x7f15440a50d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15580ffe50 tx=0x7f1540008040 comp rx=0 tx=0).stop 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 msgr2=0x7f15581973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f15581973b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1548006010 tx=0x7f15480042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 shutdown_connections 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15440a2c10 0x7f15440a50d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1558102ef0 0x7f155819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1558102570 0x7f15581978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 --2- 172.21.15.143:0/2153226942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1558100e20 0x7f15581973b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.335+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 >> 172.21.15.143:0/2153226942 conn(0x7f15580fc7d0 msgr2=0x7f15580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:02.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.336+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 shutdown_connections 2024-09-10T10:46:02.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:02.336+0000 7f155da40640 1 -- 172.21.15.143:0/2153226942 wait complete. 2024-09-10T10:46:02.683 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:02 smithi143 ceph-mon[31057]: pgmap v380: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-10T10:46:02.684 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:02 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2153226942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:02 smithi178 ceph-mon[34813]: pgmap v380: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-10T10:46:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:02 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2153226942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:02 smithi144 ceph-mon[34230]: pgmap v380: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-10T10:46:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:02 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2153226942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:03.159 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:46:03.416 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 -- 172.21.15.143:0/1489041757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c102ef0 msgr2=0x7f6d5c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 --2- 172.21.15.143:0/1489041757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c10da40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009ca0 tx=0x7f6d5402a950 comp rx=0 tx=0).stop 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 -- 172.21.15.143:0/1489041757 shutdown_connections 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 --2- 172.21.15.143:0/1489041757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c10da40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 --2- 172.21.15.143:0/1489041757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d5c102570 0x7f6d5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 --2- 172.21.15.143:0/1489041757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c100e20 0x7f6d5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 -- 172.21.15.143:0/1489041757 >> 172.21.15.143:0/1489041757 conn(0x7f6d5c0fc7d0 msgr2=0x7f6d5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 -- 172.21.15.143:0/1489041757 shutdown_connections 2024-09-10T10:46:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.934+0000 7f6d627fc640 1 -- 172.21.15.143:0/1489041757 wait complete. 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 Processor -- start 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 -- start start 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 0x7f6d5c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d5c102570 0x7f6d5c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c110ed0 con 0x7f6d5c100e20 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c110d50 con 0x7f6d5c102570 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.935+0000 7f6d627fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c111050 con 0x7f6d5c102ef0 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 0x7f6d5c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 0x7f6d5c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37678/0 (socket says 172.21.15.143:37678) 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2632307161 learned_addr learned my addr 172.21.15.143:0/2632307161 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d5c102570 0x7f6d5c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2632307161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c102ef0 msgr2=0x7f6d5c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2632307161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d5c102570 msgr2=0x7f6d5c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d5c102570 0x7f6d5c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2632307161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c19d4f0 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 0x7f6d5c198540 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00b4a0 tx=0x7f6d4c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d597fa640 1 -- 172.21.15.143:0/2632307161 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c015020 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d597fa640 1 -- 172.21.15.143:0/2632307161 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d4c011ce0 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c19d7e0 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d597fa640 1 -- 172.21.15.143:0/2632307161 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c010dc0 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.936+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1a50c0 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.939+0000 7f6d597fa640 1 -- 172.21.15.143:0/2632307161 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6d4c018020 con 0x7f6d5c100e20 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.939+0000 7f6d64bd5740 1 Processor -- start 2024-09-10T10:46:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.939+0000 7f6d64bd5740 1 -- start start 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.939+0000 7f6d597fa640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d440a2c10 0x7f6d440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.939+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.939+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5581d87412f0 con 0x5581d87dc840 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581d86e8cf0 con 0x5581d87dbb80 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5581d873d2b0 con 0x5581d87db120 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87dcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37694/0 (socket says 172.21.15.143:37694) 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 -- 172.21.15.143:0/129014362 learned_addr learned my addr 172.21.15.143:0/129014362 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d60d99640 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d440a2c10 0x7f6d440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 -- 172.21.15.143:0/129014362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 msgr2=0x5581d87db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 -- 172.21.15.143:0/129014362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 msgr2=0x5581d87f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d597fa640 1 -- 172.21.15.143:0/2632307161 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d4c0ced60 con 0x7f6d5c100e20 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 -- 172.21.15.143:0/129014362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d86cd3a0 con 0x5581d87dc840 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87dcc20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c097be0 tx=0x7f6d4c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767a44b7d083c6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d427fc640 1 -- 172.21.15.143:0/129014362 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0151c0 con 0x5581d87dc840 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.940+0000 7f6d427fc640 1 -- 172.21.15.143:0/129014362 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d4c010480 con 0x5581d87dc840 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.941+0000 7f6d427fc640 1 -- 172.21.15.143:0/129014362 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0995c0 con 0x5581d87dc840 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.941+0000 7f6d64bd5740 1 -- 172.21.15.143:0/129014362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 msgr2=0x5581d87dcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.941+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d440a2c10 0x7f6d440a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d5402ae70 tx=0x7f6d5402a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.941+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87dcc20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c097be0 tx=0x7f6d4c0047d0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- 172.21.15.143:0/129014362 shutdown_connections 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/129014362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87dcc20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- 172.21.15.143:0/129014362 >> 172.21.15.143:0/129014362 conn(0x5581d87a4310 msgr2=0x5581d87d7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- 172.21.15.143:0/129014362 shutdown_connections 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- 172.21.15.143:0/129014362 wait complete. 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- start start 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 Processor -- start 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5581d87f1930 con 0x5581d87dc840 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581d86fc060 con 0x5581d87dbb80 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.942+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5581d87f1ab0 con 0x5581d87db120 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d64bd5740 10 client.0 _ll_register_callbacks cb 0x5581d844f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d5b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87ee690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37704/0 (socket says 172.21.15.143:37704) 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 -- 172.21.15.143:0/2657994165 learned_addr learned my addr 172.21.15.143:0/2657994165 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 -- 172.21.15.143:0/2657994165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 msgr2=0x5581d87f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 -- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 msgr2=0x5581d87ee0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87ee0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 -- 172.21.15.143:0/2657994165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d84b23c0 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87ee0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87ee690 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6d54003c20 tx=0x7f6d540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcda664399aa87f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.943+0000 7f6d3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2657994165 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5404c030 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2657994165 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d540045c0 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581d874e860 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2657994165 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d54034040 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 10 client.15576 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581d87fd380 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2657994165 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5581d874e860 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2657994165 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6d54047040 con 0x5581d87dc840 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 10 client.15576 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 10 client.15576 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 20 client.15576 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d87fe830 0x5581d8800c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 4 client.15576 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.944+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d8748d70 con 0x5581d87fe830 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.945+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d87fe830 0x5581d8800c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.945+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d87fe830 0x5581d8800c70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5c99f8e18f3942 server_cookie=ce4267a59a3f0a66 in_seq=0 out_seq=0 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.945+0000 7f6d3b7fe640 10 client.15576 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2657994165 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581d8748d70 con 0x5581d87fe830 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d3b7fe640 10 client.15576 handle_command_reply: tid=0 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 client.15576 shutdown 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 msgr2=0x5581d87ee690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87ee690 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6d54003c20 tx=0x7f6d540095c0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 shutdown_connections 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d87fe830 0x5581d8800c70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87ee690 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dbb80 0x5581d87ee0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d3b7fe640 0 client.15576 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2657994165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87db120 0x5581d87f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 >> 172.21.15.143:0/2657994165 conn(0x5581d87a4310 msgr2=0x5581d87d6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 shutdown_connections 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2657994165 wait complete. 2024-09-10T10:46:03.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.946+0000 7f6d64bd5740 20 client.15576 trim_cache size 0 max 16384 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 Processor -- start 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 -- start start 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87db120 0x5581d87fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1aca50 con 0x5581d87dc840 2024-09-10T10:46:03.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1ae590 con 0x5581d87db120 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1ad120 con 0x5581d87dbb80 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d60d99640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87db120 0x5581d87fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d60d99640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87d6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37708/0 (socket says 172.21.15.143:37708) 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d60d99640 1 -- 172.21.15.143:0/2465538992 learned_addr learned my addr 172.21.15.143:0/2465538992 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.960+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2465538992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 msgr2=0x5581d87e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87e20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2465538992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 msgr2=0x5581d87d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87d6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2465538992 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d87ec640 con 0x5581d87db120 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87db120 0x5581d87fabd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cc540 tx=0x7f6d4c0ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921167cf21295035 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d41ffb640 1 -- 172.21.15.143:0/2465538992 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c09b210 con 0x5581d87db120 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d41ffb640 1 -- 172.21.15.143:0/2465538992 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d4c099510 con 0x5581d87db120 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.961+0000 7f6d41ffb640 1 -- 172.21.15.143:0/2465538992 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c09f180 con 0x5581d87db120 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2465538992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87db120 msgr2=0x5581d87fabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87db120 0x5581d87fabd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cc540 tx=0x7f6d4c0ce040 comp rx=0 tx=0).stop 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2465538992 shutdown_connections 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87dc840 0x5581d87d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2465538992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87db120 0x5581d87fabd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2465538992 >> 172.21.15.143:0/2465538992 conn(0x5581d87a4310 msgr2=0x5581d87d64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2465538992 shutdown_connections 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2465538992 wait complete. 2024-09-10T10:46:03.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.962+0000 7f6d64bd5740 1 -- start start 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 Processor -- start 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 0x5581d87f7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dc840 0x5581d87f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1adec0 con 0x5581d87db120 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1ad7f0 con 0x5581d87dc840 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1aaa30 con 0x5581d87dbb80 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d64bd5740 10 client.0 _ll_register_callbacks cb 0x7f6d54009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 0x5581d87f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 0x5581d87f7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37710/0 (socket says 172.21.15.143:37710) 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2363300648 learned_addr learned my addr 172.21.15.143:0/2363300648 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dc840 0x5581d87f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2363300648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 msgr2=0x5581d87f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dc840 msgr2=0x5581d87f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dc840 0x5581d87f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5bfff640 1 -- 172.21.15.143:0/2363300648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d87f3240 con 0x5581d87db120 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d60d99640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dc840 0x5581d87f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.963+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d5bfff640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 0x5581d87f7330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00b470 tx=0x7f6d4c0107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8360f9a399c92611 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2363300648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c09b210 con 0x5581d87db120 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2363300648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d4c0cc570 con 0x5581d87db120 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d4c099510 con 0x5581d87db120 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2363300648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d4c09f180 con 0x5581d87db120 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 10 client.15582 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581d88693a0 con 0x5581d87db120 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2363300648 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d4c099510 con 0x5581d87db120 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2363300648 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6d4c09a660 con 0x5581d87db120 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 10 client.15582 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 10 client.15582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 20 client.15582 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d88695d0 0x5581d886b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 4 client.15582 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.964+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581d86c5170 con 0x5581d88695d0 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.965+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d88695d0 0x5581d886b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.965+0000 7f6d5b7fe640 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d88695d0 0x5581d886b9c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b2bf604199b3164 server_cookie=9aabb4ba131fd5f6 in_seq=0 out_seq=0 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.965+0000 7f6d3b7fe640 10 client.15582 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.965+0000 7f6d3b7fe640 1 -- 172.21.15.143:0/2363300648 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581d86c5170 con 0x5581d88695d0 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.965+0000 7f6d3b7fe640 10 client.15582 handle_command_reply: tid=0 2024-09-10T10:46:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.965+0000 7f6d64bd5740 1 client.15582 shutdown 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 msgr2=0x5581d87f7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 0x5581d87f7330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00b470 tx=0x7f6d4c0107f0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 shutdown_connections 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5581d88695d0 0x5581d886b9c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581d87dc840 0x5581d87f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d87dbb80 0x5581d87f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d3b7fe640 0 client.15582 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 --2- 172.21.15.143:0/2363300648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5581d87db120 0x5581d87f7330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 >> 172.21.15.143:0/2363300648 conn(0x5581d87a4310 msgr2=0x5581d87d6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 shutdown_connections 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 1 -- 172.21.15.143:0/2363300648 wait complete. 2024-09-10T10:46:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d64bd5740 20 client.15582 trim_cache size 0 max 16384 2024-09-10T10:46:03.981 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d440a2c10 msgr2=0x7f6d440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d627fc640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d440a2c10 0x7f6d440a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d5402ae70 tx=0x7f6d5402a950 comp rx=0 tx=0).stop 2024-09-10T10:46:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 msgr2=0x7f6d5c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.966+0000 7f6d627fc640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 0x7f6d5c198540 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00b4a0 tx=0x7f6d4c00b8a0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 shutdown_connections 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d440a2c10 0x7f6d440a50d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d5c102ef0 0x7f6d5c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d5c102570 0x7f6d5c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 --2- 172.21.15.143:0/2632307161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d5c100e20 0x7f6d5c198540 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 >> 172.21.15.143:0/2632307161 conn(0x7f6d5c0fc7d0 msgr2=0x7f6d5c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 shutdown_connections 2024-09-10T10:46:03.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:03.967+0000 7f6d627fc640 1 -- 172.21.15.143:0/2632307161 wait complete. 2024-09-10T10:46:04.115 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T10:46:04.116 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T10:46:04.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:46:04.793 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:04 smithi143 ceph-mon[31057]: pgmap v381: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T10:46:04.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:46:04.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:04 smithi178 ceph-mon[34813]: pgmap v381: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T10:46:04.933 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:04 smithi144 ceph-mon[34230]: pgmap v381: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-10T10:46:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 -- 172.21.15.143:0/744158497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd4101df0 msgr2=0x7f8cd41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/744158497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd4101df0 0x7f8cd41021d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009900 tx=0x7f8cc402a410 comp rx=0 tx=0).stop 2024-09-10T10:46:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 -- 172.21.15.143:0/744158497 shutdown_connections 2024-09-10T10:46:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/744158497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd4106f20 0x7f8cd410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/744158497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cd41027a0 0x7f8cd41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/744158497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd4101df0 0x7f8cd41021d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 -- 172.21.15.143:0/744158497 >> 172.21.15.143:0/744158497 conn(0x7f8cd40fd8f0 msgr2=0x7f8cd40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 -- 172.21.15.143:0/744158497 shutdown_connections 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.417+0000 7f8cd9e21640 1 -- 172.21.15.143:0/744158497 wait complete. 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 Processor -- start 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 -- start start 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd4101df0 0x7f8cd41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 0x7f8cd4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cd4106f20 0x7f8cd419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111e50 con 0x7f8cd41027a0 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111cd0 con 0x7f8cd4106f20 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd9e21640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111fd0 con 0x7f8cd4101df0 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 0x7f8cd4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 0x7f8cd4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37728/0 (socket says 172.21.15.143:37728) 2024-09-10T10:46:05.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 -- 172.21.15.143:0/192003876 learned_addr learned my addr 172.21.15.143:0/192003876 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd37fe640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd4101df0 0x7f8cd41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 -- 172.21.15.143:0/192003876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd4101df0 msgr2=0x7f8cd41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd4101df0 0x7f8cd41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 -- 172.21.15.143:0/192003876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cd4106f20 msgr2=0x7f8cd419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cd4106f20 0x7f8cd419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.418+0000 7f8cd2ffd640 1 -- 172.21.15.143:0/192003876 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd419d390 con 0x7f8cd41027a0 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.419+0000 7f8cd2ffd640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 0x7f8cd4198920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800c920 tx=0x7f8cc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.419+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8007c70 con 0x7f8cd41027a0 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.419+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd419d680 con 0x7f8cd41027a0 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.419+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc8007e10 con 0x7f8cd41027a0 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.419+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8018820 con 0x7f8cd41027a0 2024-09-10T10:46:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.419+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd4074420 con 0x7f8cd41027a0 2024-09-10T10:46:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.420+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd41021d0 con 0x7f8cd41027a0 2024-09-10T10:46:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.421+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f8cc80189c0 con 0x7f8cd41027a0 2024-09-10T10:46:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.422+0000 7f8cd0ff9640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.422+0000 7f8cd37fe640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.422+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8cc80cf750 con 0x7f8cd41027a0 2024-09-10T10:46:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.422+0000 7f8cd37fe640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4031440 tx=0x7f8cc40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.426+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cc8014030 con 0x7f8cd41027a0 2024-09-10T10:46:05.536 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:05.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.653+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cd410a130 con 0x7f8cd41027a0 2024-09-10T10:46:05.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.654+0000 7f8cd0ff9640 1 -- 172.21.15.143:0/192003876 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f8cc809c1f0 con 0x7f8cd41027a0 2024-09-10T10:46:05.657 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:46:05.657 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:46:05.657 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:46:05.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cbc0a2c10 msgr2=0x7f8cbc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4031440 tx=0x7f8cc40095f0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 msgr2=0x7f8cd4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 0x7f8cd4198920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800c920 tx=0x7f8cc800cde0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 shutdown_connections 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8cbc0a2c10 0x7f8cbc0a50d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cd4106f20 0x7f8cd419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cd41027a0 0x7f8cd4198920 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 --2- 172.21.15.143:0/192003876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd4101df0 0x7f8cd41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 >> 172.21.15.143:0/192003876 conn(0x7f8cd40fd8f0 msgr2=0x7f8cd40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 shutdown_connections 2024-09-10T10:46:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:05.657+0000 7f8cd9e21640 1 -- 172.21.15.143:0/192003876 wait complete. 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.246+0000 7f92aeae3640 1 -- 172.21.15.143:0/835806413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 msgr2=0x7f92a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.246+0000 7f92aeae3640 1 --2- 172.21.15.143:0/835806413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8107b90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9294009930 tx=0x7f929402a420 comp rx=0 tx=0).stop 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 -- 172.21.15.143:0/835806413 shutdown_connections 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 --2- 172.21.15.143:0/835806413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92a81082a0 0x7f92a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 --2- 172.21.15.143:0/835806413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8107b90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 --2- 172.21.15.143:0/835806413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a8102d30 0x7f92a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 -- 172.21.15.143:0/835806413 >> 172.21.15.143:0/835806413 conn(0x7f92a80fc7d0 msgr2=0x7f92a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 -- 172.21.15.143:0/835806413 shutdown_connections 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 -- 172.21.15.143:0/835806413 wait complete. 2024-09-10T10:46:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 Processor -- start 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.247+0000 7f92aeae3640 1 -- start start 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a8102d30 0x7f92a81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92a81082a0 0x7f92a819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111f10 con 0x7f92a81036e0 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8111d90 con 0x7f92a81082a0 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8112090 con 0x7f92a8102d30 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37742/0 (socket says 172.21.15.143:37742) 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 -- 172.21.15.143:0/2645559603 learned_addr learned my addr 172.21.15.143:0/2645559603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92ac87f640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a8102d30 0x7f92a81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92ad080640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92a81082a0 0x7f92a819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 -- 172.21.15.143:0/2645559603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a8102d30 msgr2=0x7f92a81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a8102d30 0x7f92a81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 -- 172.21.15.143:0/2645559603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92a81082a0 msgr2=0x7f92a819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92a81082a0 0x7f92a819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 -- 172.21.15.143:0/2645559603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a819d4a0 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a7fff640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8198a30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9294031440 tx=0x7f9294004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a5ffb640 1 -- 172.21.15.143:0/2645559603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929403f030 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a5ffb640 1 -- 172.21.15.143:0/2645559603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9294003aa0 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92a5ffb640 1 -- 172.21.15.143:0/2645559603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294035070 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.248+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a819d730 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.249+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81a5070 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.251+0000 7f92a5ffb640 1 -- 172.21.15.143:0/2645559603 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f9294042030 con 0x7f92a81036e0 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.251+0000 7f92b0ebc740 1 Processor -- start 2024-09-10T10:46:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 -- start start 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92a5ffb640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f92980a2c90 0x7f92980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd19780 0x55ff6fd1bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fc702f0 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fc17cf0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fc6c2b0 con 0x55ff6fd19780 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ad080640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f92980a2c90 0x7f92980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92a5ffb640 1 -- 172.21.15.143:0/2645559603 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f92940efae0 con 0x7f92a81036e0 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd0a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:55728/0 (socket says 172.21.15.143:55728) 2024-09-10T10:46:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 -- 172.21.15.143:0/3166585866 learned_addr learned my addr 172.21.15.143:0/3166585866 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ad080640 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd19780 0x55ff6fd1bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 -- 172.21.15.143:0/3166585866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd19780 msgr2=0x55ff6fd1bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd19780 0x55ff6fd1bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 -- 172.21.15.143:0/3166585866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 msgr2=0x55ff6fd0aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd0aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.252+0000 7f92ac87f640 1 -- 172.21.15.143:0/3166585866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6fbfc3a0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.253+0000 7f92ad080640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f92980a2c90 0x7f92980a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92a000a780 tx=0x7f92a0013a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.253+0000 7f92ac87f640 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd0a090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bcd0 tx=0x7f929c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de45a7f9e5219ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.253+0000 7f9292ffd640 1 -- 172.21.15.143:0/3166585866 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c005020 con 0x55ff6fd0bb40 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.253+0000 7f9292ffd640 1 -- 172.21.15.143:0/3166585866 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929c0047a0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.253+0000 7f9292ffd640 1 -- 172.21.15.143:0/3166585866 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0116a0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3166585866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 msgr2=0x55ff6fd0a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd0a090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bcd0 tx=0x7f929c0107a0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3166585866 shutdown_connections 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd19780 0x55ff6fd1bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3166585866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd0a090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3166585866 >> 172.21.15.143:0/3166585866 conn(0x55ff6fcd3310 msgr2=0x55ff6fd06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3166585866 shutdown_connections 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3166585866 wait complete. 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 -- start start 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.254+0000 7f92b0ebc740 1 Processor -- start 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd19120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd16970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fd1c480 con 0x55ff6fd0bb40 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fc2b060 con 0x55ff6fd19780 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fd1c600 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 10 client.0 _ll_register_callbacks cb 0x55ff6f97e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd163d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37766/0 (socket says 172.21.15.143:37766) 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 -- 172.21.15.143:0/4206262916 learned_addr learned my addr 172.21.15.143:0/4206262916 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92ac87f640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd19120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92ad080640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd16970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 -- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0a5d0 msgr2=0x55ff6fd19120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd19120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 -- 172.21.15.143:0/4206262916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 msgr2=0x55ff6fd16970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd16970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 -- 172.21.15.143:0/4206262916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6f9e13c0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92ad080640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd16970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92a7fff640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd163d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f92940bbec0 tx=0x7f92940bb950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ab5f45cb4cd0b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f9287fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f9287fff640 1 -- 172.21.15.143:0/4206262916 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294049040 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f9287fff640 1 -- 172.21.15.143:0/4206262916 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f92940ed380 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.255+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c0047a0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f9287fff640 1 -- 172.21.15.143:0/4206262916 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92940c4af0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 10 client.15600 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff6fd27ed0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f9287fff640 1 -- 172.21.15.143:0/4206262916 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92940e6e50 con 0x55ff6fd0bb40 2024-09-10T10:46:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f9287fff640 1 -- 172.21.15.143:0/4206262916 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f92940b85a0 con 0x55ff6fd0bb40 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 10 client.15600 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 10 client.15600 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 20 client.15600 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd29330 0x55ff6fd2b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 4 client.15600 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff6fc7d860 con 0x55ff6fd29330 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.256+0000 7f92ac87f640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd29330 0x55ff6fd2b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.257+0000 7f92ac87f640 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd29330 0x55ff6fd2b740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97e7ab3a7598d5c5 server_cookie=cc1c15b45f3d0588 in_seq=0 out_seq=0 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.257+0000 7f9287fff640 10 client.15600 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.257+0000 7f9287fff640 1 -- 172.21.15.143:0/4206262916 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff6fc7d860 con 0x55ff6fd29330 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.257+0000 7f9287fff640 10 client.15600 handle_command_reply: tid=0 2024-09-10T10:46:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 client.15600 shutdown 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 msgr2=0x55ff6fd163d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd163d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f92940bbec0 tx=0x7f92940bb950 comp rx=0 tx=0).stop 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 shutdown_connections 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd29330 0x55ff6fd2b740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd16970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd163d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/4206262916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd19120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f9287fff640 0 client.15600 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 >> 172.21.15.143:0/4206262916 conn(0x55ff6fcd3310 msgr2=0x55ff6fd0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 shutdown_connections 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 1 -- 172.21.15.143:0/4206262916 wait complete. 2024-09-10T10:46:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.258+0000 7f92b0ebc740 20 client.15600 trim_cache size 0 max 16384 2024-09-10T10:46:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.271+0000 7f92b0ebc740 1 Processor -- start 2024-09-10T10:46:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.271+0000 7f92b0ebc740 1 -- start start 2024-09-10T10:46:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd25720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd31000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 0x55ff6fd31540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81ac980 con 0x55ff6fd19780 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81ae4c0 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81ad050 con 0x55ff6fd0bb40 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 0x55ff6fd31540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 0x55ff6fd31540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37774/0 (socket says 172.21.15.143:37774) 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 -- 172.21.15.143:0/3418713935 learned_addr learned my addr 172.21.15.143:0/3418713935 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ac87f640 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd25720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92a7fff640 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd31000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 -- 172.21.15.143:0/3418713935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 msgr2=0x55ff6fd31000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd31000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 -- 172.21.15.143:0/3418713935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0a5d0 msgr2=0x55ff6fd25720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd25720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 -- 172.21.15.143:0/3418713935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6fd0ad50 con 0x55ff6fd19780 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92a7fff640 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd31000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92ad080640 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 0x55ff6fd31540 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f92a00132b0 tx=0x7f92a001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c74a6090ab3a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.272+0000 7f92927fc640 1 -- 172.21.15.143:0/3418713935 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0024070 con 0x55ff6fd19780 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92927fc640 1 -- 172.21.15.143:0/3418713935 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92a0003b70 con 0x55ff6fd19780 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92927fc640 1 -- 172.21.15.143:0/3418713935 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a00126b0 con 0x55ff6fd19780 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3418713935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 msgr2=0x55ff6fd31540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 0x55ff6fd31540 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f92a00132b0 tx=0x7f92a001d000 comp rx=0 tx=0).stop 2024-09-10T10:46:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3418713935 shutdown_connections 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd19780 0x55ff6fd31540 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd31000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/3418713935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd25720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3418713935 >> 172.21.15.143:0/3418713935 conn(0x55ff6fcd3310 msgr2=0x55ff6fd11720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3418713935 shutdown_connections 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.273+0000 7f92b0ebc740 1 -- 172.21.15.143:0/3418713935 wait complete. 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 -- start start 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 Processor -- start 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd21a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd21fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd1e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81addf0 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81ad720 con 0x55ff6fd19780 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81aa960 con 0x55ff6fd0bb40 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92b0ebc740 10 client.0 _ll_register_callbacks cb 0x7f92940b81d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92ac87f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd21a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92ac87f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd21a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37788/0 (socket says 172.21.15.143:37788) 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.274+0000 7f92ac87f640 1 -- 172.21.15.143:0/78740373 learned_addr learned my addr 172.21.15.143:0/78740373 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92a7fff640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd21fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ad080640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd1e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ac87f640 1 -- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 msgr2=0x55ff6fd21fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ac87f640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd21fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ac87f640 1 -- 172.21.15.143:0/78740373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 msgr2=0x55ff6fd1e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ac87f640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd1e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ac87f640 1 -- 172.21.15.143:0/78740373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6fd1f090 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ad080640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd1e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92ac87f640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd21a70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f929c011010 tx=0x7f929c00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111c742d81ad30db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f9287fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f9287fff640 1 -- 172.21.15.143:0/78740373 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c004220 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f9287fff640 1 -- 172.21.15.143:0/78740373 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f929c0043c0 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a0003b70 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f9287fff640 1 -- 172.21.15.143:0/78740373 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f929c037050 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92b0ebc740 10 client.15606 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.275+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff6fd94a70 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f9287fff640 1 -- 172.21.15.143:0/78740373 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92a0003b70 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f9287fff640 1 -- 172.21.15.143:0/78740373 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f929c003930 con 0x55ff6fd0a5d0 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92b0ebc740 10 client.15606 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92b0ebc740 10 client.15606 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92b0ebc740 20 client.15606 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd94f10 0x55ff6fd97300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92b0ebc740 4 client.15606 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff6fc77d70 con 0x55ff6fd94f10 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92a7fff640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd94f10 0x55ff6fd97300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f9287fff640 10 client.15606 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.276+0000 7f92a7fff640 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd94f10 0x55ff6fd97300 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85eb6be22e442a3d server_cookie=943b804bcbdc5d90 in_seq=0 out_seq=0 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f9287fff640 1 -- 172.21.15.143:0/78740373 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff6fc77d70 con 0x55ff6fd94f10 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f9287fff640 10 client.15606 handle_command_reply: tid=0 2024-09-10T10:46:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 client.15606 shutdown 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 msgr2=0x55ff6fd21a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd21a70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f929c011010 tx=0x7f929c00b9c0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 shutdown_connections 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55ff6fd94f10 0x55ff6fd97300 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff6fd19780 0x55ff6fd1e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fd0bb40 0x55ff6fd21fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f9287fff640 0 client.15606 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 --2- 172.21.15.143:0/78740373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ff6fd0a5d0 0x55ff6fd21a70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 >> 172.21.15.143:0/78740373 conn(0x55ff6fcd3310 msgr2=0x55ff6fd17c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 shutdown_connections 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 1 -- 172.21.15.143:0/78740373 wait complete. 2024-09-10T10:46:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.277+0000 7f92b0ebc740 20 client.15606 trim_cache size 0 max 16384 2024-09-10T10:46:06.290 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f92980a2c90 msgr2=0x7f92980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f92980a2c90 0x7f92980a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92a000a780 tx=0x7f92a0013a50 comp rx=0 tx=0).stop 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 msgr2=0x7f92a8198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8198a30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9294031440 tx=0x7f9294004270 comp rx=0 tx=0).stop 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 shutdown_connections 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f92980a2c90 0x7f92980a5150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92a81082a0 0x7f92a819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a81036e0 0x7f92a8198a30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 --2- 172.21.15.143:0/2645559603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92a8102d30 0x7f92a81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 >> 172.21.15.143:0/2645559603 conn(0x7f92a80fc7d0 msgr2=0x7f92a8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 shutdown_connections 2024-09-10T10:46:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:06.278+0000 7f92aeae3640 1 -- 172.21.15.143:0/2645559603 wait complete. 2024-09-10T10:46:06.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:06 smithi143 ceph-mon[31057]: pgmap v382: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:06.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:06 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/192003876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:06 smithi178 ceph-mon[34813]: pgmap v382: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:06 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/192003876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:06.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:06 smithi144 ceph-mon[34230]: pgmap v382: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:06 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/192003876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:07.202 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T10:46:07.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T10:46:07.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:46:07.793 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.289+0000 7f601a587640 1 -- 172.21.15.143:0/2091204688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601410e410 msgr2=0x7f6014110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.289+0000 7f601a587640 1 --2- 172.21.15.143:0/2091204688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601410e410 0x7f6014110800 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f600c009ca0 tx=0x7f600c02a950 comp rx=0 tx=0).stop 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.289+0000 7f601a587640 1 -- 172.21.15.143:0/2091204688 shutdown_connections 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.289+0000 7f601a587640 1 --2- 172.21.15.143:0/2091204688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f601410e410 0x7f6014110800 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.289+0000 7f601a587640 1 --2- 172.21.15.143:0/2091204688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6014074ee0 0x7f601410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.290+0000 7f601a587640 1 --2- 172.21.15.143:0/2091204688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60140745c0 0x7f60140749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.290+0000 7f601a587640 1 -- 172.21.15.143:0/2091204688 >> 172.21.15.143:0/2091204688 conn(0x7f60140fd8b0 msgr2=0x7f60140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.290+0000 7f601a587640 1 -- 172.21.15.143:0/2091204688 shutdown_connections 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.290+0000 7f601a587640 1 -- 172.21.15.143:0/2091204688 wait complete. 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.290+0000 7f601a587640 1 Processor -- start 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.290+0000 7f601a587640 1 -- start start 2024-09-10T10:46:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f601a587640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 0x7f601419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f601a587640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6014074ee0 0x7f601419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f601a587640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f601410e410 0x7f601419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f601a587640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6014114060 con 0x7f60140745c0 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f601a587640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6014113ee0 con 0x7f6014074ee0 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f601a587640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60141141e0 con 0x7f601410e410 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 0x7f601419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 0x7f601419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40300/0 (socket says 172.21.15.143:40300) 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 -- 172.21.15.143:0/1016896705 learned_addr learned my addr 172.21.15.143:0/1016896705 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f60137fe640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6014074ee0 0x7f601419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6018b24640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f601410e410 0x7f601419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 -- 172.21.15.143:0/1016896705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f601410e410 msgr2=0x7f601419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f601410e410 0x7f601419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 -- 172.21.15.143:0/1016896705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6014074ee0 msgr2=0x7f601419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6014074ee0 0x7f601419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 -- 172.21.15.143:0/1016896705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601419f510 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f60137fe640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6014074ee0 0x7f601419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.291+0000 7f6013fff640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 0x7f601419a560 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f600400b4a0 tx=0x7f600400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.292+0000 7f60117fa640 1 -- 172.21.15.143:0/1016896705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004014070 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.292+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601419f800 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.292+0000 7f60117fa640 1 -- 172.21.15.143:0/1016896705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6004010ac0 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.292+0000 7f60117fa640 1 -- 172.21.15.143:0/1016896705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004017c30 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.292+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141a70e0 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.294+0000 7f60117fa640 1 -- 172.21.15.143:0/1016896705 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6004017070 con 0x7f60140745c0 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.294+0000 7f601c960740 1 Processor -- start 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.294+0000 7f601c960740 1 -- start start 2024-09-10T10:46:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f601c960740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea83760 0x558c4ea83b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f601c960740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 0x558c4ea8e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f601c960740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea8e940 0x558c4ea90d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f601c960740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c4e9e92a0 con 0x558c4ea84080 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f601c960740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c4e990b10 con 0x558c4ea83760 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f601c960740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558c4e9e5260 con 0x558c4ea8e940 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 0x558c4ea8e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 0x558c4ea8e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40308/0 (socket says 172.21.15.143:40308) 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 -- 172.21.15.143:0/3590273115 learned_addr learned my addr 172.21.15.143:0/3590273115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6018b24640 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea83760 0x558c4ea83b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f60137fe640 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea8e940 0x558c4ea90d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f60117fa640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5ffc0a2c10 0x7f5ffc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6018b24640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5ffc0a2c10 0x7f5ffc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 -- 172.21.15.143:0/3590273115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea8e940 msgr2=0x558c4ea90d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea8e940 0x558c4ea90d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 -- 172.21.15.143:0/3590273115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea83760 msgr2=0x558c4ea83b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea83760 0x558c4ea83b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f6013fff640 1 -- 172.21.15.143:0/3590273115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c4e75ab90 con 0x558c4ea84080 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f60137fe640 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea8e940 0x558c4ea90d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.295+0000 7f60117fa640 1 -- 172.21.15.143:0/1016896705 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f60040cec10 con 0x7f60140745c0 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.296+0000 7f6013fff640 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 0x558c4ea8e400 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f60040cc250 tx=0x7f6004021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3085f07090ed1db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.296+0000 7f6018b24640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5ffc0a2c10 0x7f5ffc0a50d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f600c009430 tx=0x7f600c02a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.296+0000 7f5ffa7fc640 1 -- 172.21.15.143:0/3590273115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600409ae30 con 0x558c4ea84080 2024-09-10T10:46:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.296+0000 7f5ffa7fc640 1 -- 172.21.15.143:0/3590273115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600409b760 con 0x558c4ea84080 2024-09-10T10:46:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.296+0000 7f5ffa7fc640 1 -- 172.21.15.143:0/3590273115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600409a9c0 con 0x558c4ea84080 2024-09-10T10:46:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 -- 172.21.15.143:0/3590273115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 msgr2=0x558c4ea8e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 0x558c4ea8e400 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f60040cc250 tx=0x7f6004021000 comp rx=0 tx=0).stop 2024-09-10T10:46:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 -- 172.21.15.143:0/3590273115 shutdown_connections 2024-09-10T10:46:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea8e940 0x558c4ea90d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea84080 0x558c4ea8e400 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 --2- 172.21.15.143:0/3590273115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea83760 0x558c4ea83b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 -- 172.21.15.143:0/3590273115 >> 172.21.15.143:0/3590273115 conn(0x558c4ea7fcf0 msgr2=0x558c4ea800d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 -- 172.21.15.143:0/3590273115 shutdown_connections 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 -- 172.21.15.143:0/3590273115 wait complete. 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.297+0000 7f601c960740 1 -- start start 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 Processor -- start 2024-09-10T10:46:08.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea8b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4ea8bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4ea8c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558c4ea91510 con 0x558c4ea8e940 2024-09-10T10:46:08.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c4e9716e0 con 0x558c4ea84080 2024-09-10T10:46:08.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558c4ea91690 con 0x558c4ea83760 2024-09-10T10:46:08.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f601c960740 10 client.0 _ll_register_callbacks cb 0x558c4e995a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:08.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f6018b24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4ea8c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f6018b24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4ea8c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40316/0 (socket says 172.21.15.143:40316) 2024-09-10T10:46:08.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f6018b24640 1 -- 172.21.15.143:0/2653918749 learned_addr learned my addr 172.21.15.143:0/2653918749 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:08.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f6013fff640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea8b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.298+0000 7f60137fe640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4ea8bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f6018b24640 1 -- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 msgr2=0x558c4ea8b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f6018b24640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea8b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f6018b24640 1 -- 172.21.15.143:0/2653918749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 msgr2=0x558c4ea8bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f6018b24640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4ea8bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f6018b24640 1 -- 172.21.15.143:0/2653918749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c4ea9a9b0 con 0x558c4ea8e940 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f60137fe640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4ea8bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f6018b24640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4ea8c250 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f600c009930 tx=0x7f600c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9ebb81ddd72242 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f5ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f5ff37fe640 1 -- 172.21.15.143:0/2653918749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c034040 con 0x558c4ea8e940 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f5ff37fe640 1 -- 172.21.15.143:0/2653918749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f600c004450 con 0x558c4ea8e940 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600409b760 con 0x558c4ea8e940 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f5ff37fe640 1 -- 172.21.15.143:0/2653918749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f600c004650 con 0x558c4ea8e940 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f601c960740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:08.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.299+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c4ea9d320 con 0x558c4ea8e940 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f5ff37fe640 1 -- 172.21.15.143:0/2653918749 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600409b760 con 0x558c4ea8e940 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f5ff37fe640 1 -- 172.21.15.143:0/2653918749 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f600c03a430 con 0x558c4ea8e940 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f601c960740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f601c960740 10 client.15618 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f601c960740 20 client.15618 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f601c960740 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4ea9e880 0x558c4eaa0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f601c960740 4 client.15618 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c4e9f69d0 con 0x558c4ea9e880 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f6013fff640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4ea9e880 0x558c4eaa0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f6013fff640 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4ea9e880 0x558c4eaa0cc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa18234f250fb627 server_cookie=fee36a278d12ff27 in_seq=0 out_seq=0 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.300+0000 7f5ff37fe640 10 client.15618 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.301+0000 7f5ff37fe640 1 -- 172.21.15.143:0/2653918749 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c4e9f69d0 con 0x558c4ea9e880 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.301+0000 7f5ff37fe640 10 client.15618 handle_command_reply: tid=0 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.301+0000 7f601c960740 1 client.15618 shutdown 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.301+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 msgr2=0x558c4ea8c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.301+0000 7f601c960740 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4ea8c250 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f600c009930 tx=0x7f600c0095c0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 shutdown_connections 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4ea8c250 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4ea8bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4ea9e880 0x558c4eaa0cc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 --2- 172.21.15.143:0/2653918749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea8b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f5ff37fe640 0 client.15618 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 >> 172.21.15.143:0/2653918749 conn(0x558c4ea7fcf0 msgr2=0x558c4ea859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 shutdown_connections 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 1 -- 172.21.15.143:0/2653918749 wait complete. 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.302+0000 7f601c960740 20 client.15618 trim_cache size 0 max 16384 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 Processor -- start 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 -- start start 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea7ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4eaa6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4eaa6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b0de0 con 0x558c4ea8e940 2024-09-10T10:46:08.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b2920 con 0x558c4ea84080 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.334+0000 7f601c960740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b14b0 con 0x558c4ea83760 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4eaa6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f60137fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4eaa6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4eaa6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40332/0 (socket says 172.21.15.143:40332) 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 -- 172.21.15.143:0/3997608025 learned_addr learned my addr 172.21.15.143:0/3997608025 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6013fff640 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea7ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 -- 172.21.15.143:0/3997608025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 msgr2=0x558c4ea7ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea7ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 -- 172.21.15.143:0/3997608025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 msgr2=0x558c4eaa6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4eaa6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 -- 172.21.15.143:0/3997608025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c4ea7fb20 con 0x558c4ea8e940 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f60137fe640 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4eaa6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f6018b24640 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4eaa6ad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f600c005f30 tx=0x7f600c02fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b8befed927121e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f5ff9ffb640 1 -- 172.21.15.143:0/3997608025 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c03ab10 con 0x558c4ea8e940 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f5ff9ffb640 1 -- 172.21.15.143:0/3997608025 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600c063070 con 0x558c4ea8e940 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.335+0000 7f5ff9ffb640 1 -- 172.21.15.143:0/3997608025 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c03a690 con 0x558c4ea8e940 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 -- 172.21.15.143:0/3997608025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 msgr2=0x558c4eaa6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4eaa6ad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f600c005f30 tx=0x7f600c02fa20 comp rx=0 tx=0).stop 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 -- 172.21.15.143:0/3997608025 shutdown_connections 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea8e940 0x558c4eaa6ad0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea84080 0x558c4eaa6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 --2- 172.21.15.143:0/3997608025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea83760 0x558c4ea7ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 -- 172.21.15.143:0/3997608025 >> 172.21.15.143:0/3997608025 conn(0x558c4ea7fcf0 msgr2=0x558c4ea7e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 -- 172.21.15.143:0/3997608025 shutdown_connections 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.336+0000 7f601c960740 1 -- 172.21.15.143:0/3997608025 wait complete. 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 -- start start 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 Processor -- start 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 0x558c4ea96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea84080 0x558c4ea93120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea8e940 0x558c4ea93660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b2250 con 0x558c4ea83760 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b1b80 con 0x558c4ea8e940 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60141aedc0 con 0x558c4ea84080 2024-09-10T10:46:08.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f601c960740 10 client.0 _ll_register_callbacks cb 0x7f600c02f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f6013fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 0x558c4ea96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f6013fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 0x558c4ea96de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40342/0 (socket says 172.21.15.143:40342) 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.337+0000 7f6013fff640 1 -- 172.21.15.143:0/1443778120 learned_addr learned my addr 172.21.15.143:0/1443778120 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6018b24640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea8e940 0x558c4ea93660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f60137fe640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea84080 0x558c4ea93120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6013fff640 1 -- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea84080 msgr2=0x558c4ea93120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6013fff640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea84080 0x558c4ea93120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6013fff640 1 -- 172.21.15.143:0/1443778120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea8e940 msgr2=0x558c4ea93660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6013fff640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea8e940 0x558c4ea93660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6013fff640 1 -- 172.21.15.143:0/1443778120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c4ea7b260 con 0x558c4ea83760 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f6013fff640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 0x558c4ea96de0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f600400bfd0 tx=0x7f60040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64603825408cd8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f5ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f5ff37fe640 1 -- 172.21.15.143:0/1443778120 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600409aba0 con 0x558c4ea83760 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f5ff37fe640 1 -- 172.21.15.143:0/1443778120 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6004099100 con 0x558c4ea83760 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600c063070 con 0x558c4ea83760 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f5ff37fe640 1 -- 172.21.15.143:0/1443778120 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f600409ae30 con 0x558c4ea83760 2024-09-10T10:46:08.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f601c960740 10 client.15624 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c4eb0a130 con 0x558c4ea83760 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.338+0000 7f5ff37fe640 1 -- 172.21.15.143:0/1443778120 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f600c063070 con 0x558c4ea83760 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f5ff37fe640 1 -- 172.21.15.143:0/1443778120 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f60040d7040 con 0x558c4ea83760 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f601c960740 10 client.15624 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f601c960740 10 client.15624 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f601c960740 20 client.15624 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f601c960740 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4eb0a410 0x558c4eb0c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f601c960740 4 client.15624 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x558c4e9f0d20 con 0x558c4eb0a410 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f60137fe640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4eb0a410 0x558c4eb0c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f5ff37fe640 10 client.15624 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:08.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.339+0000 7f60137fe640 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4eb0a410 0x558c4eb0c800 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3229650daee5fe3 server_cookie=70e3c27bae5d263d in_seq=0 out_seq=0 2024-09-10T10:46:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.349+0000 7f5ff37fe640 1 -- 172.21.15.143:0/1443778120 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558c4e9f0d20 con 0x558c4eb0a410 2024-09-10T10:46:08.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.349+0000 7f5ff37fe640 10 client.15624 handle_command_reply: tid=0 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.349+0000 7f601c960740 1 client.15624 shutdown 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.349+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 msgr2=0x558c4ea96de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.349+0000 7f601c960740 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 0x558c4ea96de0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f600400bfd0 tx=0x7f60040046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 shutdown_connections 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c4ea8e940 0x558c4ea93660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558c4eb0a410 0x558c4eb0c800 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558c4ea84080 0x558c4ea93120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 --2- 172.21.15.143:0/1443778120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558c4ea83760 0x558c4ea96de0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 >> 172.21.15.143:0/1443778120 conn(0x558c4ea7fcf0 msgr2=0x558c4ea7ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f5ff37fe640 0 client.15624 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 shutdown_connections 2024-09-10T10:46:08.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 1 -- 172.21.15.143:0/1443778120 wait complete. 2024-09-10T10:46:08.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.350+0000 7f601c960740 20 client.15624 trim_cache size 0 max 16384 2024-09-10T10:46:08.368 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:08.368 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 29, 2024-09-10T10:46:08.368 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:46:08.368 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 2, 2024-09-10T10:46:08.368 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "9c3f80de": { 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 2, 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:46:08.369 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 150.0, 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:08.370 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:08.371 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:08.371 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5ffc0a2c10 msgr2=0x7f5ffc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5ffc0a2c10 0x7f5ffc0a50d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f600c009430 tx=0x7f600c02a950 comp rx=0 tx=0).stop 2024-09-10T10:46:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 msgr2=0x7f601419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 0x7f601419a560 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f600400b4a0 tx=0x7f600400b960 comp rx=0 tx=0).stop 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 shutdown_connections 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5ffc0a2c10 0x7f5ffc0a50d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f601410e410 0x7f601419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6014074ee0 0x7f601419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 --2- 172.21.15.143:0/1016896705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60140745c0 0x7f601419a560 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 >> 172.21.15.143:0/1016896705 conn(0x7f60140fd8b0 msgr2=0x7f60140fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 shutdown_connections 2024-09-10T10:46:08.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:08.351+0000 7f601a587640 1 -- 172.21.15.143:0/1016896705 wait complete. 2024-09-10T10:46:08.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:08 smithi143 ceph-mon[31057]: pgmap v383: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:46:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:08 smithi178 ceph-mon[34813]: pgmap v383: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:46:08.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:08 smithi144 ceph-mon[34230]: pgmap v383: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:46:10.219 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:10.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:10 smithi143 ceph-mon[31057]: pgmap v384: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:46:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:10 smithi178 ceph-mon[34813]: pgmap v384: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:46:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:10 smithi144 ceph-mon[34230]: pgmap v384: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:46:12.408 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9c3f80de', quiesce took 8.3 seconds. Will release after: 21.7 2024-09-10T10:46:12.622 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:12 smithi143 ceph-mon[31057]: pgmap v385: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efc9b7fe640 1 -- 172.21.15.143:0/1123056698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90031880 con 0x7efc9c106a60 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 -- 172.21.15.143:0/1123056698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c106a60 msgr2=0x7efc9c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 --2- 172.21.15.143:0/1123056698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c106a60 0x7efc9c106e40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efc90009930 tx=0x7efc9002a420 comp rx=0 tx=0).stop 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 -- 172.21.15.143:0/1123056698 shutdown_connections 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 --2- 172.21.15.143:0/1123056698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c101400 0x7efc9c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 --2- 172.21.15.143:0/1123056698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc9c100a60 0x7efc9c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 --2- 172.21.15.143:0/1123056698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c106a60 0x7efc9c106e40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 -- 172.21.15.143:0/1123056698 >> 172.21.15.143:0/1123056698 conn(0x7efc9c0fc780 msgr2=0x7efc9c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 -- 172.21.15.143:0/1123056698 shutdown_connections 2024-09-10T10:46:12.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.861+0000 7efca2f95640 1 -- 172.21.15.143:0/1123056698 wait complete. 2024-09-10T10:46:12.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 Processor -- start 2024-09-10T10:46:12.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 -- start start 2024-09-10T10:46:12.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 0x7efc9c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc9c101400 0x7efc9c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c106a60 0x7efc9c1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c110d60 con 0x7efc9c100a60 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c110be0 con 0x7efc9c106a60 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.862+0000 7efca2f95640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c110ee0 con 0x7efc9c101400 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 0x7efc9c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 0x7efc9c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40362/0 (socket says 172.21.15.143:40362) 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 -- 172.21.15.143:0/4277683296 learned_addr learned my addr 172.21.15.143:0/4277683296 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efc9bfff640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc9c101400 0x7efc9c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca1532640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c106a60 0x7efc9c1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 -- 172.21.15.143:0/4277683296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc9c101400 msgr2=0x7efc9c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc9c101400 0x7efc9c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 -- 172.21.15.143:0/4277683296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c106a60 msgr2=0x7efc9c1a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c106a60 0x7efc9c1a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 -- 172.21.15.143:0/4277683296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c197020 con 0x7efc9c100a60 2024-09-10T10:46:12.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.863+0000 7efca0d31640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 0x7efc9c199230 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc90004770 tx=0x7efc900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.864+0000 7efc99ffb640 1 -- 172.21.15.143:0/4277683296 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9003f070 con 0x7efc9c100a60 2024-09-10T10:46:12.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.864+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c1972b0 con 0x7efc9c100a60 2024-09-10T10:46:12.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.864+0000 7efc99ffb640 1 -- 172.21.15.143:0/4277683296 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc9003b460 con 0x7efc9c100a60 2024-09-10T10:46:12.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.864+0000 7efc99ffb640 1 -- 172.21.15.143:0/4277683296 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90035070 con 0x7efc9c100a60 2024-09-10T10:46:12.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.864+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c197790 con 0x7efc9c100a60 2024-09-10T10:46:12.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.867+0000 7efc99ffb640 1 -- 172.21.15.143:0/4277683296 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7efc9003b610 con 0x7efc9c100a60 2024-09-10T10:46:12.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.868+0000 7efc99ffb640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.868+0000 7efc99ffb640 1 -- 172.21.15.143:0/4277683296 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc900ef7e0 con 0x7efc9c100a60 2024-09-10T10:46:12.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.868+0000 7efca536e740 1 Processor -- start 2024-09-10T10:46:12.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.868+0000 7efc9bfff640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.868+0000 7efca536e740 1 -- start start 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.868+0000 7efca536e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c88b40 0x562984c87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca536e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 0x562984c87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca536e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca536e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562984bed2f0 con 0x562984c875d0 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca536e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562984b94cf0 con 0x562984c88b40 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca536e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562984be92b0 con 0x562984c92370 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 0x562984c87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 0x562984c87a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40374/0 (socket says 172.21.15.143:40374) 2024-09-10T10:46:12.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 -- 172.21.15.143:0/2502947705 learned_addr learned my addr 172.21.15.143:0/2502947705 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc9c06b2e0 tx=0x7efc8800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca1532640 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c94760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efca0d31640 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c88b40 0x562984c87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 -- 172.21.15.143:0/2502947705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 msgr2=0x562984c94760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c94760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 -- 172.21.15.143:0/2502947705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c88b40 msgr2=0x562984c87090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c88b40 0x562984c87090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.869+0000 7efc9bfff640 1 -- 172.21.15.143:0/2502947705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562984b793a0 con 0x562984c875d0 2024-09-10T10:46:12.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.870+0000 7efc9bfff640 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 0x562984c87a10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ff30 tx=0x7efc88012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1ecc320d66f8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.870+0000 7efc86ffd640 1 -- 172.21.15.143:0/2502947705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88016070 con 0x562984c875d0 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.870+0000 7efc86ffd640 1 -- 172.21.15.143:0/2502947705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc88025b20 con 0x562984c875d0 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.870+0000 7efc86ffd640 1 -- 172.21.15.143:0/2502947705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8801cb90 con 0x562984c875d0 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 -- 172.21.15.143:0/2502947705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 msgr2=0x562984c87a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 0x562984c87a10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efc8800ff30 tx=0x7efc88012000 comp rx=0 tx=0).stop 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 -- 172.21.15.143:0/2502947705 shutdown_connections 2024-09-10T10:46:12.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c875d0 0x562984c87a10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 --2- 172.21.15.143:0/2502947705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c88b40 0x562984c87090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 -- 172.21.15.143:0/2502947705 >> 172.21.15.143:0/2502947705 conn(0x562984c50310 msgr2=0x562984c83c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 -- 172.21.15.143:0/2502947705 shutdown_connections 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.871+0000 7efca536e740 1 -- 172.21.15.143:0/2502947705 wait complete. 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 -- start start 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 Processor -- start 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562984c95070 con 0x562984c88b40 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562984ba8060 con 0x562984c875d0 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562984c951f0 con 0x562984c92370 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.872+0000 7efca536e740 10 client.0 _ll_register_callbacks cb 0x5629848fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c8f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40390/0 (socket says 172.21.15.143:40390) 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 -- 172.21.15.143:0/3307580190 learned_addr learned my addr 172.21.15.143:0/3307580190 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efca0d31640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efca1532640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 -- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 msgr2=0x562984c8f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 -- 172.21.15.143:0/3307580190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 msgr2=0x562984c91d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c91d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 -- 172.21.15.143:0/3307580190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298495e3c0 con 0x562984c88b40 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc9bfff640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c8f040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efc8801c460 tx=0x7efc8800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6533e012d05502 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.873+0000 7efc7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efc7bfff640 1 -- 172.21.15.143:0/3307580190 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88016040 con 0x562984c88b40 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efc7bfff640 1 -- 172.21.15.143:0/3307580190 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc8801cce0 con 0x562984c88b40 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc88025b20 con 0x562984c88b40 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efc7bfff640 1 -- 172.21.15.143:0/3307580190 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc8803b050 con 0x562984c88b40 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 10 client.15636 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562984ca0b40 con 0x562984c88b40 2024-09-10T10:46:12.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efc7bfff640 1 -- 172.21.15.143:0/3307580190 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc88025b20 con 0x562984c88b40 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efc7bfff640 1 -- 172.21.15.143:0/3307580190 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efc88026ce0 con 0x562984c88b40 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 10 client.15636 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 10 client.15636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 20 client.15636 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.874+0000 7efca536e740 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984ca4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.875+0000 7efca536e740 4 client.15636 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.875+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562984bfa860 con 0x562984ca1f40 2024-09-10T10:46:12.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.875+0000 7efca0d31640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984ca4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.875+0000 7efca0d31640 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984ca4330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b84cab2101be24c6 server_cookie=1f8dafd71ff07cd0 in_seq=0 out_seq=0 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.875+0000 7efc7bfff640 10 client.15636 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efc7bfff640 1 -- 172.21.15.143:0/3307580190 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562984bfa860 con 0x562984ca1f40 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efc7bfff640 10 client.15636 handle_command_reply: tid=0 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efca536e740 1 client.15636 shutdown 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 msgr2=0x562984c8f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efca536e740 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c8f040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efc8801c460 tx=0x7efc8800a5b0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 shutdown_connections 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efca536e740 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984ca4330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.876+0000 7efc7bfff640 0 client.15636 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c8f040 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 1 --2- 172.21.15.143:0/3307580190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c91d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 >> 172.21.15.143:0/3307580190 conn(0x562984c50310 msgr2=0x562984c89930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 shutdown_connections 2024-09-10T10:46:12.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 1 -- 172.21.15.143:0/3307580190 wait complete. 2024-09-10T10:46:12.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.877+0000 7efca536e740 20 client.15636 trim_cache size 0 max 16384 2024-09-10T10:46:12.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.901+0000 7efca536e740 1 Processor -- start 2024-09-10T10:46:12.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.901+0000 7efca536e740 1 -- start start 2024-09-10T10:46:12.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.901+0000 7efca536e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c875d0 0x562984c9e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.901+0000 7efca536e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984ca9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efca536e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c92370 0x562984ca9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efca536e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b0df0 con 0x562984c88b40 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efca536e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b2930 con 0x562984c92370 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efca536e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b14c0 con 0x562984c875d0 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984ca9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984ca9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40392/0 (socket says 172.21.15.143:40392) 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 -- 172.21.15.143:0/1277336152 learned_addr learned my addr 172.21.15.143:0/1277336152 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efca0d31640 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c875d0 0x562984c9e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 -- 172.21.15.143:0/1277336152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c875d0 msgr2=0x562984c9e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c875d0 0x562984c9e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 -- 172.21.15.143:0/1277336152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c92370 msgr2=0x562984ca9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c92370 0x562984ca9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 -- 172.21.15.143:0/1277336152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562984c8a120 con 0x562984c88b40 2024-09-10T10:46:12.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.902+0000 7efc9bfff640 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984ca9aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efc880046d0 tx=0x7efc88005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af55067fb0e9e064 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.903+0000 7efc867fc640 1 -- 172.21.15.143:0/1277336152 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88010c90 con 0x562984c88b40 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.903+0000 7efc867fc640 1 -- 172.21.15.143:0/1277336152 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc88005020 con 0x562984c88b40 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.903+0000 7efc867fc640 1 -- 172.21.15.143:0/1277336152 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880049c0 con 0x562984c88b40 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 -- 172.21.15.143:0/1277336152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 msgr2=0x562984ca9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984ca9aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efc880046d0 tx=0x7efc88005a90 comp rx=0 tx=0).stop 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 -- 172.21.15.143:0/1277336152 shutdown_connections 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c92370 0x562984ca9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984ca9aa0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 --2- 172.21.15.143:0/1277336152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c875d0 0x562984c9e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 -- 172.21.15.143:0/1277336152 >> 172.21.15.143:0/1277336152 conn(0x562984c50310 msgr2=0x562984c8e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 -- 172.21.15.143:0/1277336152 shutdown_connections 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 -- 172.21.15.143:0/1277336152 wait complete. 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 -- start start 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.904+0000 7efca536e740 1 Processor -- start 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c9a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b2260 con 0x562984c88b40 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b1b90 con 0x562984c875d0 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1aedd0 con 0x562984c92370 2024-09-10T10:46:12.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca536e740 10 client.0 _ll_register_callbacks cb 0x7efc94009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efc9bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c9aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40398/0 (socket says 172.21.15.143:40398) 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca1532640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efc9bfff640 1 -- 172.21.15.143:0/1992260563 learned_addr learned my addr 172.21.15.143:0/1992260563 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca0d31640 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c9a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca1532640 1 -- 172.21.15.143:0/1992260563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 msgr2=0x562984c9a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca1532640 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c9a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca1532640 1 -- 172.21.15.143:0/1992260563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 msgr2=0x562984c9aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca1532640 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c9aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.905+0000 7efca1532640 1 -- 172.21.15.143:0/1992260563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562984c8bfe0 con 0x562984c92370 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efca1532640 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8b8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x562984c88070 tx=0x7efc94012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e04b757b72f784b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efc7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:12.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efc7bfff640 1 -- 172.21.15.143:0/1992260563 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94003b80 con 0x562984c92370 2024-09-10T10:46:12.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efc7bfff640 1 -- 172.21.15.143:0/1992260563 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc94003d20 con 0x562984c92370 2024-09-10T10:46:12.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc88005020 con 0x562984c92370 2024-09-10T10:46:12.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efc7bfff640 1 -- 172.21.15.143:0/1992260563 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc94025050 con 0x562984c92370 2024-09-10T10:46:12.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efca536e740 10 client.25180 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:12.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562984d0c510 con 0x562984c92370 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.906+0000 7efc7bfff640 1 -- 172.21.15.143:0/1992260563 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc88005020 con 0x562984c92370 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efc7bfff640 1 -- 172.21.15.143:0/1992260563 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efc94004580 con 0x562984c92370 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca536e740 10 client.25180 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca536e740 10 client.25180 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca536e740 20 client.25180 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca536e740 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984d0ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca536e740 4 client.25180 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562984bf4d70 con 0x562984ca1f40 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.907+0000 7efca0d31640 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984d0ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efc7bfff640 10 client.25180 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca0d31640 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984d0ecd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcdbdaed1897a011 server_cookie=7c7eb923d560b85a in_seq=0 out_seq=0 2024-09-10T10:46:12.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efc7bfff640 1 -- 172.21.15.143:0/1992260563 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562984bf4d70 con 0x562984ca1f40 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efc7bfff640 10 client.25180 handle_command_reply: tid=0 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 client.25180 shutdown 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 msgr2=0x562984c8b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8b8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x562984c88070 tx=0x7efc94012b60 comp rx=0 tx=0).stop 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 shutdown_connections 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562984ca1f40 0x562984d0ecd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562984c92370 0x562984c8b8a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562984c88b40 0x562984c9aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 --2- 172.21.15.143:0/1992260563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562984c875d0 0x562984c9a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 >> 172.21.15.143:0/1992260563 conn(0x562984c50310 msgr2=0x562984c89110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efc7bfff640 0 client.25180 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:12.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 shutdown_connections 2024-09-10T10:46:12.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.908+0000 7efca536e740 1 -- 172.21.15.143:0/1992260563 wait complete. 2024-09-10T10:46:12.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca536e740 20 client.25180 trim_cache size 0 max 16384 2024-09-10T10:46:12.915 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc8c0a2c10 msgr2=0x7efc8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc9c06b2e0 tx=0x7efc8800a2d0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 msgr2=0x7efc9c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 0x7efc9c199230 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc90004770 tx=0x7efc900042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 shutdown_connections 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c106a60 0x7efc9c1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc9c101400 0x7efc9c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 --2- 172.21.15.143:0/4277683296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9c100a60 0x7efc9c199230 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.909+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 >> 172.21.15.143:0/4277683296 conn(0x7efc9c0fc780 msgr2=0x7efc9c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.910+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 shutdown_connections 2024-09-10T10:46:12.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:12.910+0000 7efca2f95640 1 -- 172.21.15.143:0/4277683296 wait complete. 2024-09-10T10:46:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:12 smithi178 ceph-mon[34813]: pgmap v385: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:12 smithi144 ceph-mon[34230]: pgmap v385: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:13.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:46:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 -- 172.21.15.143:0/1714447457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 msgr2=0x7f2ad0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/1714447457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad0101200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009900 tx=0x7f2ac002a4a0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 -- 172.21.15.143:0/1714447457 shutdown_connections 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/1714447457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0102ef0 0x7f2ad010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/1714447457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ad0102570 0x7f2ad01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/1714447457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad0101200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 -- 172.21.15.143:0/1714447457 >> 172.21.15.143:0/1714447457 conn(0x7f2ad00fc7d0 msgr2=0x7f2ad00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 -- 172.21.15.143:0/1714447457 shutdown_connections 2024-09-10T10:46:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.554+0000 7f2ad62f7640 1 -- 172.21.15.143:0/1714447457 wait complete. 2024-09-10T10:46:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.555+0000 7f2ad62f7640 1 Processor -- start 2024-09-10T10:46:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.555+0000 7f2ad62f7640 1 -- start start 2024-09-10T10:46:14.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.555+0000 7f2ad62f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.555+0000 7f2ad62f7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ad0102570 0x7f2ad010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.555+0000 7f2ad62f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0102ef0 0x7f2ad010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.555+0000 7f2acffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad010e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40416/0 (socket says 172.21.15.143:40416) 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0111f10 con 0x7f2ad0100e20 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 -- 172.21.15.143:0/619923362 learned_addr learned my addr 172.21.15.143:0/619923362 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0111d90 con 0x7f2ad0102ef0 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0112090 con 0x7f2ad0102570 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ad0102570 0x7f2ad010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2ad4894640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0102ef0 0x7f2ad010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 -- 172.21.15.143:0/619923362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ad0102570 msgr2=0x7f2ad010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ad0102570 0x7f2ad010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 -- 172.21.15.143:0/619923362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0102ef0 msgr2=0x7f2ad010ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0102ef0 0x7f2ad010ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 -- 172.21.15.143:0/619923362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad01a6970 con 0x7f2ad0100e20 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acffff640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad010e4d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0000c00 tx=0x7f2ac0003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.556+0000 7f2acd7fa640 1 -- 172.21.15.143:0/619923362 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0040030 con 0x7f2ad0100e20 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.557+0000 7f2acd7fa640 1 -- 172.21.15.143:0/619923362 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac0004200 con 0x7f2ad0100e20 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.557+0000 7f2acd7fa640 1 -- 172.21.15.143:0/619923362 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00317a0 con 0x7f2ad0100e20 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.557+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad01a6c00 con 0x7f2ad0100e20 2024-09-10T10:46:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.557+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad01a7090 con 0x7f2ad0100e20 2024-09-10T10:46:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.559+0000 7f2acd7fa640 1 -- 172.21.15.143:0/619923362 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2ac0043030 con 0x7f2ad0100e20 2024-09-10T10:46:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 Processor -- start 2024-09-10T10:46:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 -- start start 2024-09-10T10:46:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2acd7fa640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2ab80a2c90 0x7f2ab80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b49fb40 0x55ac0b49e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b49ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2acd7fa640 1 -- 172.21.15.143:0/619923362 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ac00efad0 con 0x7f2ad0100e20 2024-09-10T10:46:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 0x55ac0b4ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b4042f0 con 0x55ac0b4a9370 2024-09-10T10:46:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b3abcf0 con 0x55ac0b49e5d0 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b4002b0 con 0x55ac0b49fb40 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad4894640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2ab80a2c90 0x7f2ab80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad4894640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 0x55ac0b4ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad4894640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 0x55ac0b4ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40422/0 (socket says 172.21.15.143:40422) 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.560+0000 7f2ad4894640 1 -- 172.21.15.143:0/1636814844 learned_addr learned my addr 172.21.15.143:0/1636814844 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2acffff640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b49fb40 0x55ac0b49e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b49ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 -- 172.21.15.143:0/1636814844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b49fb40 msgr2=0x55ac0b49e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b49fb40 0x55ac0b49e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 -- 172.21.15.143:0/1636814844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 msgr2=0x55ac0b49ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b49ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 -- 172.21.15.143:0/1636814844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0b3903a0 con 0x55ac0b4a9370 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2acffff640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b49fb40 0x55ac0b49e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b49ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2ab80a2c90 0x7f2ab80a5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8012c00 tx=0x7f2ac8009e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ad4894640 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 0x55ac0b4ab760 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8010800 tx=0x7f2ac801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df202423f1d7190 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ab67fc640 1 -- 172.21.15.143:0/1636814844 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8024070 con 0x55ac0b4a9370 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ab67fc640 1 -- 172.21.15.143:0/1636814844 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac800dcc0 con 0x55ac0b4a9370 2024-09-10T10:46:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.561+0000 7f2ab67fc640 1 -- 172.21.15.143:0/1636814844 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8020b70 con 0x55ac0b4a9370 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 -- 172.21.15.143:0/1636814844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 msgr2=0x55ac0b4ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 0x55ac0b4ab760 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8010800 tx=0x7f2ac801c000 comp rx=0 tx=0).stop 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 -- 172.21.15.143:0/1636814844 shutdown_connections 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b4a9370 0x55ac0b4ab760 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b49ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/1636814844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b49fb40 0x55ac0b49e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 -- 172.21.15.143:0/1636814844 >> 172.21.15.143:0/1636814844 conn(0x55ac0b467310 msgr2=0x55ac0b49ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 -- 172.21.15.143:0/1636814844 shutdown_connections 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.562+0000 7f2ad86d0740 1 -- 172.21.15.143:0/1636814844 wait complete. 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 -- start start 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 Processor -- start 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b4ac010 con 0x55ac0b49fb40 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b3bf060 con 0x55ac0b49e5d0 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac0b4ac190 con 0x55ac0b4a9370 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.563+0000 7f2ad86d0740 10 client.0 _ll_register_callbacks cb 0x55ac0b1128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40426/0 (socket says 172.21.15.143:40426) 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 -- 172.21.15.143:0/39040887 learned_addr learned my addr 172.21.15.143:0/39040887 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acffff640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2ad4894640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 -- 172.21.15.143:0/39040887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 msgr2=0x55ac0b4a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 -- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 msgr2=0x55ac0b4a8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4a8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 -- 172.21.15.143:0/39040887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0b1753c0 con 0x55ac0b49fb40 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a5f40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4013710 tx=0x7f2ac4013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272e2440efe46947 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/39040887 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4004460 con 0x55ac0b49fb40 2024-09-10T10:46:14.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/39040887 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ac40119c0 con 0x55ac0b49fb40 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.564+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac800dcc0 con 0x55ac0b49fb40 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.565+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/39040887 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ac4011bc0 con 0x55ac0b49fb40 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.565+0000 7f2ad86d0740 10 client.15654 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.565+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac0b4b7a20 con 0x55ac0b49fb40 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.565+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/39040887 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac800dcc0 con 0x55ac0b49fb40 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/39040887 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2ac401a070 con 0x55ac0b49fb40 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2ad86d0740 10 client.15654 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2ad86d0740 10 client.15654 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2ad86d0740 20 client.15654 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b4b8f30 0x55ac0b4bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2ad86d0740 4 client.15654 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:14.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac0b411860 con 0x55ac0b4b8f30 2024-09-10T10:46:14.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2acffff640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b4b8f30 0x55ac0b4bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2acffff640 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b4b8f30 0x55ac0b4bb380 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bc71fc22cd17cef server_cookie=6e1c6360db11163b in_seq=0 out_seq=0 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.566+0000 7f2aaf7fe640 10 client.15654 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.567+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/39040887 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac0b411860 con 0x55ac0b4b8f30 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.567+0000 7f2aaf7fe640 10 client.15654 handle_command_reply: tid=0 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.567+0000 7f2ad86d0740 1 client.15654 shutdown 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.567+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 msgr2=0x55ac0b4a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a5f40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4013710 tx=0x7f2ac4013bd0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 shutdown_connections 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a5f40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b4b8f30 0x55ac0b4bb380 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/39040887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 >> 172.21.15.143:0/39040887 conn(0x55ac0b467310 msgr2=0x55ac0b4a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2aaf7fe640 0 client.15654 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:14.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 shutdown_connections 2024-09-10T10:46:14.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 1 -- 172.21.15.143:0/39040887 wait complete. 2024-09-10T10:46:14.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.568+0000 7f2ad86d0740 20 client.15654 trim_cache size 0 max 16384 2024-09-10T10:46:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 Processor -- start 2024-09-10T10:46:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 -- start start 2024-09-10T10:46:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01b0d30 con 0x55ac0b49e5d0 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01b2870 con 0x55ac0b49fb40 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01b1400 con 0x55ac0b4a9370 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40434/0 (socket says 172.21.15.143:40434) 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 -- 172.21.15.143:0/477575194 learned_addr learned my addr 172.21.15.143:0/477575194 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acf7fe640 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2ad4894640 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 -- 172.21.15.143:0/477575194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 msgr2=0x55ac0b4a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 -- 172.21.15.143:0/477575194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49fb40 msgr2=0x55ac0b4a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.592+0000 7f2acffff640 1 -- 172.21.15.143:0/477575194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0b49f300 con 0x55ac0b49e5d0 2024-09-10T10:46:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.593+0000 7f2acffff640 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4b5330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00b88f0 tx=0x7f2ac00bbd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa56c3bc23408442 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.593+0000 7f2ab5ffb640 1 -- 172.21.15.143:0/477575194 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00d8020 con 0x55ac0b49e5d0 2024-09-10T10:46:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.593+0000 7f2ab5ffb640 1 -- 172.21.15.143:0/477575194 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac00ed370 con 0x55ac0b49e5d0 2024-09-10T10:46:14.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.593+0000 7f2ab5ffb640 1 -- 172.21.15.143:0/477575194 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac004a040 con 0x55ac0b49e5d0 2024-09-10T10:46:14.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 -- 172.21.15.143:0/477575194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 msgr2=0x55ac0b4b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4b5330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00b88f0 tx=0x7f2ac00bbd80 comp rx=0 tx=0).stop 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 -- 172.21.15.143:0/477575194 shutdown_connections 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49fb40 0x55ac0b4a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/477575194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4b5330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 -- 172.21.15.143:0/477575194 >> 172.21.15.143:0/477575194 conn(0x55ac0b467310 msgr2=0x55ac0b4a46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 -- 172.21.15.143:0/477575194 shutdown_connections 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 -- 172.21.15.143:0/477575194 wait complete. 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.594+0000 7f2ad86d0740 1 -- start start 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 Processor -- start 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01b21a0 con 0x55ac0b49fb40 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01b1ad0 con 0x55ac0b49e5d0 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad01aed10 con 0x55ac0b4a9370 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad86d0740 10 client.0 _ll_register_callbacks cb 0x7f2ac40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40436/0 (socket says 172.21.15.143:40436) 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2acf7fe640 1 -- 172.21.15.143:0/2802937431 learned_addr learned my addr 172.21.15.143:0/2802937431 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2acffff640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.595+0000 7f2ad4894640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad4894640 1 -- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 msgr2=0x55ac0b4ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad4894640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad4894640 1 -- 172.21.15.143:0/2802937431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 msgr2=0x55ac0b4aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad4894640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad4894640 1 -- 172.21.15.143:0/2802937431 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0b521ee0 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2acffff640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad4894640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4acf80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800a010 tx=0x7f2ac80206e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f824816c68cbbf88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/2802937431 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8024040 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac00ed370 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/2802937431 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2ac8004d30 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/2802937431 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ac800d7e0 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad86d0740 10 client.25192 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.596+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac0b5231e0 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/2802937431 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac00ed370 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/2802937431 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2ac801f070 con 0x55ac0b4a9370 2024-09-10T10:46:14.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2ad86d0740 10 client.25192 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2ad86d0740 10 client.25192 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2ad86d0740 20 client.25192 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b5236b0 0x55ac0b525aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2ad86d0740 4 client.25192 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac0b40bd70 con 0x55ac0b5236b0 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.597+0000 7f2acffff640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b5236b0 0x55ac0b525aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2aaf7fe640 10 client.25192 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:14.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2acffff640 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b5236b0 0x55ac0b525aa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d5ecfa101a7c751 server_cookie=d82ae522efaa9357 in_seq=0 out_seq=0 2024-09-10T10:46:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2aaf7fe640 1 -- 172.21.15.143:0/2802937431 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac0b40bd70 con 0x55ac0b5236b0 2024-09-10T10:46:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2aaf7fe640 10 client.25192 handle_command_reply: tid=0 2024-09-10T10:46:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 client.25192 shutdown 2024-09-10T10:46:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 msgr2=0x55ac0b4acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4acf80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800a010 tx=0x7f2ac80206e0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 shutdown_connections 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0b4a9370 0x55ac0b4acf80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ac0b49fb40 0x55ac0b4aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55ac0b5236b0 0x55ac0b525aa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 --2- 172.21.15.143:0/2802937431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac0b49e5d0 0x55ac0b4ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 >> 172.21.15.143:0/2802937431 conn(0x55ac0b467310 msgr2=0x55ac0b4a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.598+0000 7f2aaf7fe640 0 client.25192 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 shutdown_connections 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad86d0740 1 -- 172.21.15.143:0/2802937431 wait complete. 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad86d0740 20 client.25192 trim_cache size 0 max 16384 2024-09-10T10:46:14.604 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2ab80a2c90 msgr2=0x7f2ab80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2ab80a2c90 0x7f2ab80a5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8012c00 tx=0x7f2ac8009e00 comp rx=0 tx=0).stop 2024-09-10T10:46:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 msgr2=0x7f2ad010e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.599+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad010e4d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0000c00 tx=0x7f2ac0003e00 comp rx=0 tx=0).stop 2024-09-10T10:46:14.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 shutdown_connections 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2ab80a2c90 0x7f2ab80a5150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0102ef0 0x7f2ad010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ad0102570 0x7f2ad010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 --2- 172.21.15.143:0/619923362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ad0100e20 0x7f2ad010e4d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 >> 172.21.15.143:0/619923362 conn(0x7f2ad00fc7d0 msgr2=0x7f2ad00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 shutdown_connections 2024-09-10T10:46:14.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:14.600+0000 7f2ad62f7640 1 -- 172.21.15.143:0/619923362 wait complete. 2024-09-10T10:46:14.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:14 smithi143 ceph-mon[31057]: pgmap v386: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T10:46:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:14 smithi178 ceph-mon[34813]: pgmap v386: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T10:46:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:14 smithi144 ceph-mon[34230]: pgmap v386: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T10:46:15.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:46:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:16.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 -- 172.21.15.143:0/2280544902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 msgr2=0x7f2bf4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2280544902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4107b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009930 tx=0x7f2be802a420 comp rx=0 tx=0).stop 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 -- 172.21.15.143:0/2280544902 shutdown_connections 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2280544902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 0x7f2bf410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2280544902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4107b90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2280544902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4102d30 0x7f2bf4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 -- 172.21.15.143:0/2280544902 >> 172.21.15.143:0/2280544902 conn(0x7f2bf40fc7d0 msgr2=0x7f2bf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 -- 172.21.15.143:0/2280544902 shutdown_connections 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.217+0000 7f2bfa531640 1 -- 172.21.15.143:0/2280544902 wait complete. 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 Processor -- start 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 -- start start 2024-09-10T10:46:16.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4102d30 0x7f2bf4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 0x7f2bf4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111f10 con 0x7f2bf41036e0 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4111d90 con 0x7f2bf41082a0 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.218+0000 7f2bfa531640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4112090 con 0x7f2bf4102d30 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf8ace640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 0x7f2bf4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40464/0 (socket says 172.21.15.143:40464) 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 -- 172.21.15.143:0/2989780808 learned_addr learned my addr 172.21.15.143:0/2989780808 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4102d30 0x7f2bf4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 -- 172.21.15.143:0/2989780808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4102d30 msgr2=0x7f2bf4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4102d30 0x7f2bf4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 -- 172.21.15.143:0/2989780808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 msgr2=0x7f2bf4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 0x7f2bf4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 -- 172.21.15.143:0/2989780808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf41aae00 con 0x7f2bf41036e0 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 0x7f2bf4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4100ed0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2be8031440 tx=0x7f2be8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf17fa640 1 -- 172.21.15.143:0/2989780808 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be803f030 con 0x7f2bf41036e0 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf17fa640 1 -- 172.21.15.143:0/2989780808 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2be8003aa0 con 0x7f2bf41036e0 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.219+0000 7f2bf17fa640 1 -- 172.21.15.143:0/2989780808 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8035070 con 0x7f2bf41036e0 2024-09-10T10:46:16.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.220+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41aafd0 con 0x7f2bf41036e0 2024-09-10T10:46:16.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.220+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41ab370 con 0x7f2bf41036e0 2024-09-10T10:46:16.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bf17fa640 1 -- 172.21.15.143:0/2989780808 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2be8042030 con 0x7f2bf41036e0 2024-09-10T10:46:16.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 Processor -- start 2024-09-10T10:46:16.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 -- start start 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2f03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c8b2e5c2f0 con 0x55c8b2f01370 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8b2e03cf0 con 0x55c8b2ef65d0 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c8b2e582b0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2f03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2f03760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40476/0 (socket says 172.21.15.143:40476) 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.223+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4003854615 learned_addr learned my addr 172.21.15.143:0/4003854615 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:16.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf17fa640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bdc0a2c90 0x7f2bdc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4003854615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 msgr2=0x55c8b2ef6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4003854615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 msgr2=0x55c8b2ef6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4003854615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b2de83a0 con 0x55c8b2f01370 2024-09-10T10:46:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bdc0a2c90 0x7f2bdc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf17fa640 1 -- 172.21.15.143:0/2989780808 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2be80efac0 con 0x7f2bf41036e0 2024-09-10T10:46:16.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2f03760 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ed050 tx=0x7f2be804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecd89a5f4db136a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.224+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bdc0a2c90 0x7f2bdc0a5150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005fd0 tx=0x7f2bec004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bda7fc640 1 -- 172.21.15.143:0/4003854615 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80bb9a0 con 0x55c8b2f01370 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bda7fc640 1 -- 172.21.15.143:0/4003854615 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2be802d1c0 con 0x55c8b2f01370 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4003854615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 msgr2=0x55c8b2f03760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2f03760 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ed050 tx=0x7f2be804b000 comp rx=0 tx=0).stop 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4003854615 shutdown_connections 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2f03760 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4003854615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4003854615 >> 172.21.15.143:0/4003854615 conn(0x55c8b2ebf310 msgr2=0x55c8b2ef2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4003854615 shutdown_connections 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.225+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4003854615 wait complete. 2024-09-10T10:46:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.226+0000 7f2bfc90a740 1 -- start start 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.226+0000 7f2bfc90a740 1 Processor -- start 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.226+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2f00cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c8b2f03e30 con 0x55c8b2f01370 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8b2e17060 con 0x55c8b2ef7b40 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c8b2f03fb0 con 0x55c8b2ef65d0 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bfc90a740 10 client.0 _ll_register_callbacks cb 0x55c8b2b6a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf8ace640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:51626/0 (socket says 172.21.15.143:51626) 2024-09-10T10:46:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf8ace640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40484/0 (socket says 172.21.15.143:40484) 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 -- 172.21.15.143:0/739385367 learned_addr learned my addr 172.21.15.143:0/739385367 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2f00cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 -- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 msgr2=0x55c8b2f00cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2f00cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 -- 172.21.15.143:0/739385367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 msgr2=0x55c8b2efe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 -- 172.21.15.143:0/739385367 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b2bcd3c0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efdf80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2be8032fd0 tx=0x7f2be8009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42fdc9ae3b91a19c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.227+0000 7f2bd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bd37fe640 1 -- 172.21.15.143:0/739385367 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8048370 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be802d1c0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bd37fe640 1 -- 172.21.15.143:0/739385367 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2be80d0040 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bd37fe640 1 -- 172.21.15.143:0/739385367 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2be8048570 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bfc90a740 10 client.25169 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8b2f03e30 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bd37fe640 1 -- 172.21.15.143:0/739385367 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2be80bbe60 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.228+0000 7f2bd37fe640 1 -- 172.21.15.143:0/739385367 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2be80b84d0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bfc90a740 10 client.25169 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bfc90a740 10 client.25169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bfc90a740 20 client.25169 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f10bf0 0x55c8b2f13010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bfc90a740 4 client.25169 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2be802d1c0 con 0x55c8b2f10bf0 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f10bf0 0x55c8b2f13010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f10bf0 0x55c8b2f13010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2a7dc80ca70990 server_cookie=951b54ba16441041 in_seq=0 out_seq=0 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.229+0000 7f2bd37fe640 10 client.25169 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bd37fe640 1 -- 172.21.15.143:0/739385367 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2be802d1c0 con 0x55c8b2f10bf0 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bd37fe640 10 client.25169 handle_command_reply: tid=0 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 client.25169 shutdown 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 msgr2=0x55c8b2efdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efdf80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2be8032fd0 tx=0x7f2be8009680 comp rx=0 tx=0).stop 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 shutdown_connections 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efdf80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f10bf0 0x55c8b2f13010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/739385367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2f00cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 >> 172.21.15.143:0/739385367 conn(0x55c8b2ebf310 msgr2=0x55c8b2ef8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bd37fe640 0 client.25169 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 shutdown_connections 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 1 -- 172.21.15.143:0/739385367 wait complete. 2024-09-10T10:46:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.230+0000 7f2bfc90a740 20 client.25169 trim_cache size 0 max 16384 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 Processor -- start 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 -- start start 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2f01370 0x55c8b2f188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41b50d0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41b6c10 con 0x55c8b2f01370 2024-09-10T10:46:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.243+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41b57a0 con 0x55c8b2ef65d0 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40492/0 (socket says 172.21.15.143:40492) 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4133253101 learned_addr learned my addr 172.21.15.143:0/4133253101 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2f01370 0x55c8b2f188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4133253101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 msgr2=0x55c8b2ef9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4133253101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2f01370 msgr2=0x55c8b2f188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2f01370 0x55c8b2f188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 -- 172.21.15.143:0/4133253101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b2f008f0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef9660 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2be8006010 tx=0x7f2be80f3610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd08bfd4bef053d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bd9ffb640 1 -- 172.21.15.143:0/4133253101 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80bb7c0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bd9ffb640 1 -- 172.21.15.143:0/4133253101 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2be80bc560 con 0x55c8b2ef7b40 2024-09-10T10:46:16.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.244+0000 7f2bd9ffb640 1 -- 172.21.15.143:0/4133253101 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80c4ad0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4133253101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 msgr2=0x55c8b2ef9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef9660 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2be8006010 tx=0x7f2be80f3610 comp rx=0 tx=0).stop 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4133253101 shutdown_connections 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2f01370 0x55c8b2f188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2ef7b40 0x55c8b2ef9660 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/4133253101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef65d0 0x55c8b2ef9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.245+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4133253101 >> 172.21.15.143:0/4133253101 conn(0x55c8b2ebf310 msgr2=0x55c8b2f00000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4133253101 shutdown_connections 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 -- 172.21.15.143:0/4133253101 wait complete. 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 -- start start 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 Processor -- start 2024-09-10T10:46:16.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2efb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41b6540 con 0x55c8b2f01370 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41b5e70 con 0x55c8b2ef65d0 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41b30b0 con 0x55c8b2ef7b40 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.246+0000 7f2bfc90a740 10 client.0 _ll_register_callbacks cb 0x7f2bec009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40506/0 (socket says 172.21.15.143:40506) 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 -- 172.21.15.143:0/1614574418 learned_addr learned my addr 172.21.15.143:0/1614574418 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf37fe640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2efb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 -- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 msgr2=0x55c8b2efb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 -- 172.21.15.143:0/1614574418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 msgr2=0x55c8b2efb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2efb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 -- 172.21.15.143:0/1614574418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b2f7a960 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bf8ace640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efbb90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2bec01f700 tx=0x7f2bec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b02129b1e217ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bd37fe640 1 -- 172.21.15.143:0/1614574418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0119f0 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bd37fe640 1 -- 172.21.15.143:0/1614574418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2bec011b90 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.247+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be80bc560 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bd37fe640 1 -- 172.21.15.143:0/1614574418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2bec011d90 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 10 client.15678 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8b2f7bcc0 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bd37fe640 1 -- 172.21.15.143:0/1614574418 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2be80bc560 con 0x55c8b2f01370 2024-09-10T10:46:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bd37fe640 1 -- 172.21.15.143:0/1614574418 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2bec00f040 con 0x55c8b2f01370 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 20 client.15678 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f7c4c0 0x55c8b2f7e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 4 client.15678 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8b2e69860 con 0x55c8b2f7c4c0 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.248+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f7c4c0 0x55c8b2f7e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bd37fe640 10 client.15678 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bf3fff640 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f7c4c0 0x55c8b2f7e8e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a988f65a1a2f33 server_cookie=553b1c9a62bab22f in_seq=0 out_seq=0 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bd37fe640 1 -- 172.21.15.143:0/1614574418 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8b2e69860 con 0x55c8b2f7c4c0 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bd37fe640 10 client.15678 handle_command_reply: tid=0 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 client.15678 shutdown 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 msgr2=0x55c8b2efbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efbb90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2bec01f700 tx=0x7f2bec004ab0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 shutdown_connections 2024-09-10T10:46:16.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c8b2f01370 0x55c8b2efbb90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c8b2f7c4c0 0x55c8b2f7e8e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bd37fe640 0 client.15678 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8b2ef7b40 0x55c8b2efb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 --2- 172.21.15.143:0/1614574418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8b2ef65d0 0x55c8b2efb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 >> 172.21.15.143:0/1614574418 conn(0x55c8b2ebf310 msgr2=0x55c8b2ef14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 shutdown_connections 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.249+0000 7f2bfc90a740 1 -- 172.21.15.143:0/1614574418 wait complete. 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfc90a740 20 client.15678 trim_cache size 0 max 16384 2024-09-10T10:46:16.255 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:16.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bdc0a2c90 msgr2=0x7f2bdc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bdc0a2c90 0x7f2bdc0a5150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005fd0 tx=0x7f2bec004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:16.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 msgr2=0x7f2bf4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4100ed0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2be8031440 tx=0x7f2be8004270 comp rx=0 tx=0).stop 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 shutdown_connections 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bdc0a2c90 0x7f2bdc0a5150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf41082a0 0x7f2bf4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf41036e0 0x7f2bf4100ed0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 --2- 172.21.15.143:0/2989780808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4102d30 0x7f2bf4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 >> 172.21.15.143:0/2989780808 conn(0x7f2bf40fc7d0 msgr2=0x7f2bf4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 shutdown_connections 2024-09-10T10:46:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:16.250+0000 7f2bfa531640 1 -- 172.21.15.143:0/2989780808 wait complete. 2024-09-10T10:46:16.820 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:16 smithi143 ceph-mon[31057]: pgmap v387: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T10:46:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:16 smithi178 ceph-mon[34813]: pgmap v387: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T10:46:16.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:16 smithi144 ceph-mon[34230]: pgmap v387: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T10:46:18.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:46:18.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:46:18.294 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:18.737 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:18 smithi143 ceph-mon[31057]: pgmap v388: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T10:46:18.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.855+0000 7f51d9d9b640 1 -- 172.21.15.143:0/177181292 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc031880 con 0x7f51dc100910 2024-09-10T10:46:18.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 -- 172.21.15.143:0/177181292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 msgr2=0x7f51dc100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 --2- 172.21.15.143:0/177181292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc100cf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009900 tx=0x7f51cc02a410 comp rx=0 tx=0).stop 2024-09-10T10:46:18.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 -- 172.21.15.143:0/177181292 shutdown_connections 2024-09-10T10:46:18.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 --2- 172.21.15.143:0/177181292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 0x7f51dc1108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 --2- 172.21.15.143:0/177181292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51dc1012c0 0x7f51dc10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 --2- 172.21.15.143:0/177181292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc100cf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 -- 172.21.15.143:0/177181292 >> 172.21.15.143:0/177181292 conn(0x7f51dc0fc780 msgr2=0x7f51dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:18.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 -- 172.21.15.143:0/177181292 shutdown_connections 2024-09-10T10:46:18.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.856+0000 7f51e1181640 1 -- 172.21.15.143:0/177181292 wait complete. 2024-09-10T10:46:18.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.857+0000 7f51e1181640 1 Processor -- start 2024-09-10T10:46:18.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51e1181640 1 -- start start 2024-09-10T10:46:18.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51e1181640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51dad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51dad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57178/0 (socket says 172.21.15.143:57178) 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51dad9d640 1 -- 172.21.15.143:0/3857144116 learned_addr learned my addr 172.21.15.143:0/3857144116 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51dc1012c0 0x7f51dc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.858+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 0x7f51dc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc114000 con 0x7f51dc100910 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc113e80 con 0x7f51dc10e4c0 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc114180 con 0x7f51dc1012c0 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51da59c640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51dc1012c0 0x7f51dc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51db59e640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 0x7f51dc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51db59e640 1 -- 172.21.15.143:0/3857144116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51dc1012c0 msgr2=0x7f51dc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51db59e640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51dc1012c0 0x7f51dc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51db59e640 1 -- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 msgr2=0x7f51dc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51db59e640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51db59e640 1 -- 172.21.15.143:0/3857144116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc19f5b0 con 0x7f51dc10e4c0 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.859+0000 7f51dad9d640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:18.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.860+0000 7f51db59e640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 0x7f51dc19eed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005fd0 tx=0x7f51d400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:18.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.860+0000 7f51c3fff640 1 -- 172.21.15.143:0/3857144116 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4004d20 con 0x7f51dc10e4c0 2024-09-10T10:46:18.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.860+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc19f8a0 con 0x7f51dc10e4c0 2024-09-10T10:46:18.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.860+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc1a7180 con 0x7f51dc10e4c0 2024-09-10T10:46:18.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.860+0000 7f51c3fff640 1 -- 172.21.15.143:0/3857144116 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d4005020 con 0x7f51dc10e4c0 2024-09-10T10:46:18.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.860+0000 7f51c3fff640 1 -- 172.21.15.143:0/3857144116 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d40116b0 con 0x7f51dc10e4c0 2024-09-10T10:46:18.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.863+0000 7f51e355a740 1 Processor -- start 2024-09-10T10:46:18.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51c3fff640 1 -- 172.21.15.143:0/3857144116 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f51d4011850 con 0x7f51dc10e4c0 2024-09-10T10:46:18.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 -- start start 2024-09-10T10:46:18.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e2796f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2797480 0x55e0e27a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e0e26fc280 con 0x55e0e27a1d40 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0e26a3b10 con 0x55e0e2796b60 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51e355a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0e26f8240 con 0x55e0e2797480 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.864+0000 7f51da59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e2796f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e2796f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:37734/0 (socket says 172.21.15.143:37734) 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 -- 172.21.15.143:0/184257297 learned_addr learned my addr 172.21.15.143:0/184257297 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51dad9d640 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2797480 0x55e0e27a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51c3fff640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f51c40a2c90 0x7f51c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51c3fff640 1 -- 172.21.15.143:0/3857144116 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f51d40cf350 con 0x7f51dc10e4c0 2024-09-10T10:46:18.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51db59e640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f51c40a2c90 0x7f51c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 -- 172.21.15.143:0/184257297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2797480 msgr2=0x55e0e27a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:18.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2797480 0x55e0e27a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 -- 172.21.15.143:0/184257297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 msgr2=0x55e0e2796f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:18.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e2796f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51da59c640 1 -- 172.21.15.143:0/184257297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e26b01f0 con 0x55e0e27a1d40 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.865+0000 7f51dad9d640 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2797480 0x55e0e27a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.866+0000 7f51da59c640 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27a4130 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f51d00127d0 tx=0x7f51d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd8464b027e1883 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.866+0000 7f51c17fa640 1 -- 172.21.15.143:0/184257297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0012080 con 0x55e0e27a1d40 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.866+0000 7f51c17fa640 1 -- 172.21.15.143:0/184257297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d0012220 con 0x55e0e27a1d40 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.866+0000 7f51c17fa640 1 -- 172.21.15.143:0/184257297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d00035f0 con 0x55e0e27a1d40 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 -- 172.21.15.143:0/184257297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 msgr2=0x55e0e27a4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27a4130 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f51d00127d0 tx=0x7f51d0004730 comp rx=0 tx=0).stop 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51db59e640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f51c40a2c90 0x7f51c40a5150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51d4004ab0 tx=0x7f51d4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 -- 172.21.15.143:0/184257297 shutdown_connections 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27a4130 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2797480 0x55e0e27a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 --2- 172.21.15.143:0/184257297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e2796f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 -- 172.21.15.143:0/184257297 >> 172.21.15.143:0/184257297 conn(0x55e0e27930f0 msgr2=0x55e0e27934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 -- 172.21.15.143:0/184257297 shutdown_connections 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.867+0000 7f51e355a740 1 -- 172.21.15.143:0/184257297 wait complete. 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 -- start start 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 Processor -- start 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e279ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 0x55e0e279f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e27a1d40 0x55e0e279f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e0e27a4910 con 0x55e0e2797480 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0e26b6f10 con 0x55e0e2796b60 2024-09-10T10:46:18.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0e27a4a90 con 0x55e0e27a1d40 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51e355a740 10 client.0 _ll_register_callbacks cb 0x55e0e267fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.868+0000 7f51dad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 0x55e0e279f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 0x55e0e279f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57196/0 (socket says 172.21.15.143:57196) 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 -- 172.21.15.143:0/1525610416 learned_addr learned my addr 172.21.15.143:0/1525610416 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 -- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e27a1d40 msgr2=0x55e0e279f6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e27a1d40 0x55e0e279f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 -- 172.21.15.143:0/1525610416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 msgr2=0x55e0e279ec70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e279ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 -- 172.21.15.143:0/1525610416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e246d6a0 con 0x55e0e2797480 2024-09-10T10:46:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51dad9d640 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 0x55e0e279f1b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006010 tx=0x7f51cc031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57da00d473725ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51beff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.869+0000 7f51beff5640 1 -- 172.21.15.143:0/1525610416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc035040 con 0x55e0e2797480 2024-09-10T10:46:18.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d0012220 con 0x55e0e2797480 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51beff5640 1 -- 172.21.15.143:0/1525610416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f51cc005020 con 0x55e0e2797480 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51beff5640 1 -- 172.21.15.143:0/1525610416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51cc04f050 con 0x55e0e2797480 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 10 client.15690 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0e27b8e40 con 0x55e0e2797480 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51beff5640 1 -- 172.21.15.143:0/1525610416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d0012220 con 0x55e0e2797480 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51beff5640 1 -- 172.21.15.143:0/1525610416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f51cc0035f0 con 0x55e0e2797480 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 20 client.15690 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e27bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 4 client.15690 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:18.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.870+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e27097f0 con 0x55e0e27ba250 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.871+0000 7f51da59c640 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e27bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.871+0000 7f51da59c640 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e27bc6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c1c292b893e3fa server_cookie=9a9a89b4452595ce in_seq=0 out_seq=0 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.871+0000 7f51beff5640 10 client.15690 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51beff5640 1 -- 172.21.15.143:0/1525610416 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0e27097f0 con 0x55e0e27ba250 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51beff5640 10 client.15690 handle_command_reply: tid=0 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 client.15690 shutdown 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 msgr2=0x55e0e279f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 0x55e0e279f1b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006010 tx=0x7f51cc031890 comp rx=0 tx=0).stop 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 shutdown_connections 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e27bc6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51beff5640 0 client.15690 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e27a1d40 0x55e0e279f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e2797480 0x55e0e279f1b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 --2- 172.21.15.143:0/1525610416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2796b60 0x55e0e279ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.872+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 >> 172.21.15.143:0/1525610416 conn(0x55e0e27930f0 msgr2=0x55e0e2798e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.873+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 shutdown_connections 2024-09-10T10:46:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.873+0000 7f51e355a740 1 -- 172.21.15.143:0/1525610416 wait complete. 2024-09-10T10:46:18.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.873+0000 7f51e355a740 20 client.15690 trim_cache size 0 max 16384 2024-09-10T10:46:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 Processor -- start 2024-09-10T10:46:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 -- start start 2024-09-10T10:46:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e2798cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51c800faa0 con 0x55e0e27a1d40 2024-09-10T10:46:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80115e0 con 0x55e0e2797480 2024-09-10T10:46:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51e355a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8010170 con 0x55e0e2796b60 2024-09-10T10:46:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57206/0 (socket says 172.21.15.143:57206) 2024-09-10T10:46:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51da59c640 1 -- 172.21.15.143:0/1366962292 learned_addr learned my addr 172.21.15.143:0/1366962292 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51da59c640 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e2798cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.890+0000 7f51dad9d640 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51da59c640 1 -- 172.21.15.143:0/1366962292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 msgr2=0x55e0e2798cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51da59c640 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e2798cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51da59c640 1 -- 172.21.15.143:0/1366962292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 msgr2=0x55e0e27c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51da59c640 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51da59c640 1 -- 172.21.15.143:0/1366962292 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e27972b0 con 0x55e0e27a1d40 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51dad9d640 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51da59c640 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27c2510 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005d00 tx=0x7f51d000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45778e5e4296aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51c0ff9640 1 -- 172.21.15.143:0/1366962292 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0016030 con 0x55e0e27a1d40 2024-09-10T10:46:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51c0ff9640 1 -- 172.21.15.143:0/1366962292 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d00054d0 con 0x55e0e27a1d40 2024-09-10T10:46:18.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.891+0000 7f51c0ff9640 1 -- 172.21.15.143:0/1366962292 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0004d00 con 0x55e0e27a1d40 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 -- 172.21.15.143:0/1366962292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 msgr2=0x55e0e27c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27c2510 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005d00 tx=0x7f51d000b390 comp rx=0 tx=0).stop 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 -- 172.21.15.143:0/1366962292 shutdown_connections 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e27c2510 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 --2- 172.21.15.143:0/1366962292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e2798cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.892+0000 7f51e355a740 1 -- 172.21.15.143:0/1366962292 >> 172.21.15.143:0/1366962292 conn(0x55e0e27930f0 msgr2=0x55e0e27985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.893+0000 7f51e355a740 1 -- 172.21.15.143:0/1366962292 shutdown_connections 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.893+0000 7f51e355a740 1 -- 172.21.15.143:0/1366962292 wait complete. 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.893+0000 7f51e355a740 1 -- start start 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.893+0000 7f51e355a740 1 Processor -- start 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e27aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e281df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8010f10 con 0x55e0e27a1d40 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8010840 con 0x55e0e2797480 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51c800da80 con 0x55e0e2796b60 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:37754/0 (socket says 172.21.15.143:37754) 2024-09-10T10:46:18.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 -- 172.21.15.143:0/3883305687 learned_addr learned my addr 172.21.15.143:0/3883305687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:18.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51e355a740 10 client.0 _ll_register_callbacks cb 0x7f51cc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:18.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51da59c640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e281df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 -- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 msgr2=0x55e0e27aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e27aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 -- 172.21.15.143:0/3883305687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 msgr2=0x55e0e281df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e281df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51dad9d640 1 -- 172.21.15.143:0/3883305687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e281e5b0 con 0x55e0e2797480 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.894+0000 7f51da59c640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e281df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51dad9d640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27af0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009850 tx=0x7f51cc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8734741aec95b709 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51beff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51beff5640 1 -- 172.21.15.143:0/3883305687 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc035050 con 0x55e0e2797480 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51beff5640 1 -- 172.21.15.143:0/3883305687 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f51cc005020 con 0x55e0e2797480 2024-09-10T10:46:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d00054d0 con 0x55e0e2797480 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51beff5640 1 -- 172.21.15.143:0/3883305687 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51cc061050 con 0x55e0e2797480 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51e355a740 10 client.25190 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.895+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0e2825950 con 0x55e0e2797480 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51beff5640 1 -- 172.21.15.143:0/3883305687 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d00054d0 con 0x55e0e2797480 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51beff5640 1 -- 172.21.15.143:0/3883305687 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f51cc004900 con 0x55e0e2797480 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51e355a740 10 client.25190 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51e355a740 10 client.25190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51e355a740 20 client.25190 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51e355a740 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e2828130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51e355a740 4 client.25190 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:46:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.896+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e0e2703d00 con 0x55e0e27ba250 2024-09-10T10:46:18.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.897+0000 7f51da59c640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e2828130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:18.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.898+0000 7f51da59c640 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e2828130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aed199b9ca6a58e server_cookie=8e01df9401f5fbb in_seq=0 out_seq=0 2024-09-10T10:46:18.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.898+0000 7f51beff5640 10 client.25190 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51beff5640 1 -- 172.21.15.143:0/3883305687 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e0e2703d00 con 0x55e0e27ba250 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51beff5640 10 client.25190 handle_command_reply: tid=0 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 client.25190 shutdown 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 msgr2=0x55e0e27af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27af0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009850 tx=0x7f51cc000bc0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 shutdown_connections 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0e27a1d40 0x55e0e281df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0e2797480 0x55e0e27af0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0e27ba250 0x55e0e2828130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 --2- 172.21.15.143:0/3883305687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0e2796b60 0x55e0e27aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 >> 172.21.15.143:0/3883305687 conn(0x55e0e27930f0 msgr2=0x55e0e2792270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.899+0000 7f51beff5640 0 client.25190 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 shutdown_connections 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e355a740 1 -- 172.21.15.143:0/3883305687 wait complete. 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e355a740 20 client.25190 trim_cache size 0 max 16384 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "8b238584-a5a2-4c6c-b834-6e65d280e7eb", 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:46:18.903 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f51c40a2c90 msgr2=0x7f51c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f51c40a2c90 0x7f51c40a5150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51d4004ab0 tx=0x7f51d4004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 msgr2=0x7f51dc19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 0x7f51dc19eed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005fd0 tx=0x7f51d400edd0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 shutdown_connections 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f51c40a2c90 0x7f51c40a5150 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51dc10e4c0 0x7f51dc19eed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51dc1012c0 0x7f51dc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 --2- 172.21.15.143:0/3857144116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51dc100910 0x7f51dc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.900+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 >> 172.21.15.143:0/3857144116 conn(0x7f51dc0fc780 msgr2=0x7f51dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.901+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 shutdown_connections 2024-09-10T10:46:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:18.901+0000 7f51e1181640 1 -- 172.21.15.143:0/3857144116 wait complete. 2024-09-10T10:46:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:18 smithi178 ceph-mon[34813]: pgmap v388: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T10:46:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:18 smithi144 ceph-mon[34230]: pgmap v388: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T10:46:19.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:46:19.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:19 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:46:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:19 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:19 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:46:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:19 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:19.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:19 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:46:19.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:19 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:46:19.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:19 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:19 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:46:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:19 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:19.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:19 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:46:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:19 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:46:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:19 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:19 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:46:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:19 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:19 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:46:19.944 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 -- 172.21.15.143:0/1272591751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd550102d30 msgr2=0x7fd550103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 --2- 172.21.15.143:0/1272591751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd550102d30 0x7fd550103110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009930 tx=0x7fd54c02a420 comp rx=0 tx=0).stop 2024-09-10T10:46:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 -- 172.21.15.143:0/1272591751 shutdown_connections 2024-09-10T10:46:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 --2- 172.21.15.143:0/1272591751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5501082a0 0x7fd55010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 --2- 172.21.15.143:0/1272591751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5501036e0 0x7fd550107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 --2- 172.21.15.143:0/1272591751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd550102d30 0x7fd550103110 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.525+0000 7fd558743640 1 -- 172.21.15.143:0/1272591751 >> 172.21.15.143:0/1272591751 conn(0x7fd5500fc7d0 msgr2=0x7fd5500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.526+0000 7fd558743640 1 -- 172.21.15.143:0/1272591751 shutdown_connections 2024-09-10T10:46:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.526+0000 7fd558743640 1 -- 172.21.15.143:0/1272591751 wait complete. 2024-09-10T10:46:20.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 Processor -- start 2024-09-10T10:46:20.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 -- start start 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 0x7fd5501985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5501036e0 0x7fd550198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5501082a0 0x7fd55019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd550111f90 con 0x7fd5501036e0 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd550111e10 con 0x7fd5501082a0 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd558743640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd550112110 con 0x7fd550102d30 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd5564df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 0x7fd5501985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd555cde640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5501036e0 0x7fd550198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd5564df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 0x7fd5501985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42338/0 (socket says 172.21.15.143:42338) 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.527+0000 7fd5564df640 1 -- 172.21.15.143:0/1129331282 learned_addr learned my addr 172.21.15.143:0/1129331282 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5501082a0 0x7fd55019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5564df640 1 -- 172.21.15.143:0/1129331282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5501082a0 msgr2=0x7fd55019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5564df640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5501082a0 0x7fd55019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5564df640 1 -- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5501036e0 msgr2=0x7fd550198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5564df640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5501036e0 0x7fd550198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5564df640 1 -- 172.21.15.143:0/1129331282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55019d580 con 0x7fd550102d30 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5564df640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 0x7fd5501985d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd54c005ab0 tx=0x7fd54c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5477fe640 1 -- 172.21.15.143:0/1129331282 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c03f030 con 0x7fd550102d30 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55019d810 con 0x7fd550102d30 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5477fe640 1 -- 172.21.15.143:0/1129331282 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd54c03b460 con 0x7fd550102d30 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.528+0000 7fd5477fe640 1 -- 172.21.15.143:0/1129331282 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c035070 con 0x7fd550102d30 2024-09-10T10:46:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.529+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5501a5150 con 0x7fd550102d30 2024-09-10T10:46:20.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.532+0000 7fd55ab1c740 1 Processor -- start 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.532+0000 7fd55ab1c740 1 -- start start 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.532+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 0x555be555b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.532+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be556a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be556aa80 0x555be556ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555be54c12e0 con 0x555be556aa80 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555be5468ce0 con 0x555be555bc70 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555be54bd2a0 con 0x555be555b350 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 0x555be555b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 0x555be555b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42352/0 (socket says 172.21.15.143:42352) 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 -- 172.21.15.143:0/528023058 learned_addr learned my addr 172.21.15.143:0/528023058 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd555cde640 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be556aa80 0x555be556ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 -- 172.21.15.143:0/528023058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 msgr2=0x555be556a540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:46:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be556a540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 -- 172.21.15.143:0/528023058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be556aa80 msgr2=0x555be556ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be556aa80 0x555be556ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd556ce0640 1 -- 172.21.15.143:0/528023058 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be544d440 con 0x555be555b350 2024-09-10T10:46:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.533+0000 7fd555cde640 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be556aa80 0x555be556ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.534+0000 7fd5477fe640 1 -- 172.21.15.143:0/1129331282 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd54c042030 con 0x7fd550102d30 2024-09-10T10:46:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.534+0000 7fd556ce0640 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 0x555be555b730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd548009f90 tx=0x7fd548010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973568d00a1522c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.534+0000 7fd5477fe640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd53c0a2cd0 0x7fd53c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.535+0000 7fd544ff9640 1 -- 172.21.15.143:0/528023058 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548011700 con 0x555be555b350 2024-09-10T10:46:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.535+0000 7fd5564df640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd53c0a2cd0 0x7fd53c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.535+0000 7fd5477fe640 1 -- 172.21.15.143:0/1129331282 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd54c0ef410 con 0x7fd550102d30 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.535+0000 7fd544ff9640 1 -- 172.21.15.143:0/528023058 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548011d50 con 0x555be555b350 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd544ff9640 1 -- 172.21.15.143:0/528023058 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548019da0 con 0x555be555b350 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 -- 172.21.15.143:0/528023058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 msgr2=0x555be555b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 0x555be555b730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd548009f90 tx=0x7fd548010860 comp rx=0 tx=0).stop 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd5564df640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd53c0a2cd0 0x7fd53c0a5190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd54c02a460 tx=0x7fd54c08d810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 -- 172.21.15.143:0/528023058 shutdown_connections 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be556aa80 0x555be556ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be556a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/528023058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555b350 0x555be555b730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 -- 172.21.15.143:0/528023058 >> 172.21.15.143:0/528023058 conn(0x555be5557a20 msgr2=0x555be5557e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.536+0000 7fd55ab1c740 1 -- 172.21.15.143:0/528023058 shutdown_connections 2024-09-10T10:46:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 -- 172.21.15.143:0/528023058 wait complete. 2024-09-10T10:46:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 -- start start 2024-09-10T10:46:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 Processor -- start 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be556a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5567670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5567bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555be556d6a0 con 0x555be555b350 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555be547c050 con 0x555be555bc70 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be556a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be556a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57262/0 (socket says 172.21.15.143:57262) 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 -- 172.21.15.143:0/141838756 learned_addr learned my addr 172.21.15.143:0/141838756 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.537+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555be556d820 con 0x555be556aa80 2024-09-10T10:46:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd55ab1c740 10 client.0 _ll_register_callbacks cb 0x555be54bc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5567bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 -- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 msgr2=0x555be5567bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd555cde640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5567670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5567bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 -- 172.21.15.143:0/141838756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 msgr2=0x555be5567670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5567670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 -- 172.21.15.143:0/141838756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be52323c0 con 0x555be555b350 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd555cde640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5567670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd556ce0640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be556a150 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd548019a00 tx=0x7fd548011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b2ed7e5542c72a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.538+0000 7fd539ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd539ffb640 1 -- 172.21.15.143:0/141838756 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548019e40 con 0x555be555b350 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd548011d50 con 0x555be555b350 2024-09-10T10:46:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd539ffb640 1 -- 172.21.15.143:0/141838756 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd548004c50 con 0x555be555b350 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd539ffb640 1 -- 172.21.15.143:0/141838756 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd548004e50 con 0x555be555b350 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd55ab1c740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555be556ecc0 con 0x555be555b350 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.539+0000 7fd539ffb640 1 -- 172.21.15.143:0/141838756 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548011d50 con 0x555be555b350 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd539ffb640 1 -- 172.21.15.143:0/141838756 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd5480054c0 con 0x555be555b350 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd55ab1c740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd55ab1c740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd55ab1c740 20 client.15705 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55700d0 0x555be5572520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd55ab1c740 4 client.15705 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555be54ce850 con 0x555be55700d0 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.540+0000 7fd555cde640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55700d0 0x555be5572520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.541+0000 7fd539ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.541+0000 7fd555cde640 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55700d0 0x555be5572520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0c89c0641db4a5 server_cookie=1e2c5d5eec9afb10 in_seq=0 out_seq=0 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.541+0000 7fd539ffb640 1 -- 172.21.15.143:0/141838756 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555be54ce850 con 0x555be55700d0 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.541+0000 7fd539ffb640 10 client.15705 handle_command_reply: tid=0 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 client.15705 shutdown 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 msgr2=0x555be556a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be556a150 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd548019a00 tx=0x7fd548011f00 comp rx=0 tx=0).stop 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 shutdown_connections 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55700d0 0x555be5572520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5567bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5567670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/141838756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be556a150 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 >> 172.21.15.143:0/141838756 conn(0x555be5557a20 msgr2=0x555be555d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd539ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 shutdown_connections 2024-09-10T10:46:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.542+0000 7fd55ab1c740 1 -- 172.21.15.143:0/141838756 wait complete. 2024-09-10T10:46:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.543+0000 7fd55ab1c740 20 client.15705 trim_cache size 0 max 16384 2024-09-10T10:46:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.579+0000 7fd55ab1c740 1 Processor -- start 2024-09-10T10:46:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.579+0000 7fd55ab1c740 1 -- start start 2024-09-10T10:46:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.579+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5565e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.579+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555bc70 0x555be5569ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.579+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be556aa80 0x555be5567f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501ac9d0 con 0x555be555b350 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501ae510 con 0x555be556aa80 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501ad0a0 con 0x555be555bc70 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5565e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5565e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57268/0 (socket says 172.21.15.143:57268) 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 -- 172.21.15.143:0/1415903629 learned_addr learned my addr 172.21.15.143:0/1415903629 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be556aa80 0x555be5567f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 -- 172.21.15.143:0/1415903629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555bc70 msgr2=0x555be5569ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555bc70 0x555be5569ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 -- 172.21.15.143:0/1415903629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be556aa80 msgr2=0x555be5567f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:20.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be556aa80 0x555be5567f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 -- 172.21.15.143:0/1415903629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be55586b0 con 0x555be555b350 2024-09-10T10:46:20.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.580+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5565e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd548009720 tx=0x7fd548009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cb4e5bb3de15ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.581+0000 7fd53bfff640 1 -- 172.21.15.143:0/1415903629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548019a30 con 0x555be555b350 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.581+0000 7fd53bfff640 1 -- 172.21.15.143:0/1415903629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548038070 con 0x555be555b350 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.581+0000 7fd53bfff640 1 -- 172.21.15.143:0/1415903629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548004c50 con 0x555be555b350 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1415903629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 msgr2=0x555be5565e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5565e10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd548009720 tx=0x7fd548009a00 comp rx=0 tx=0).stop 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1415903629 shutdown_connections 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be556aa80 0x555be5567f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be555bc70 0x555be5569ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1415903629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5565e10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1415903629 >> 172.21.15.143:0/1415903629 conn(0x555be5557a20 msgr2=0x555be5556ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1415903629 shutdown_connections 2024-09-10T10:46:20.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1415903629 wait complete. 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 -- start start 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.582+0000 7fd55ab1c740 1 Processor -- start 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5573ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5573ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5574530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501ade40 con 0x555be555b350 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501ad770 con 0x555be555bc70 2024-09-10T10:46:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501aa9b0 con 0x555be556aa80 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd55ab1c740 10 client.0 _ll_register_callbacks cb 0x7fd548009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5573ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5573ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57282/0 (socket says 172.21.15.143:57282) 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 -- 172.21.15.143:0/1433641930 learned_addr learned my addr 172.21.15.143:0/1433641930 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5574530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd555cde640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5573ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 -- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 msgr2=0x555be5574530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5574530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 -- 172.21.15.143:0/1433641930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 msgr2=0x555be5573ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5573ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.583+0000 7fd556ce0640 1 -- 172.21.15.143:0/1433641930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be55f0470 con 0x555be555b350 2024-09-10T10:46:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd556ce0640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5573ab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd548009990 tx=0x7fd548019da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e932a764d45091e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd539ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd539ffb640 1 -- 172.21.15.143:0/1433641930 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548004f30 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd548038070 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd539ffb640 1 -- 172.21.15.143:0/1433641930 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd5480050d0 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd539ffb640 1 -- 172.21.15.143:0/1433641930 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5480052d0 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd55ab1c740 10 client.15711 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555be55f1740 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.584+0000 7fd539ffb640 1 -- 172.21.15.143:0/1433641930 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548038070 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd539ffb640 1 -- 172.21.15.143:0/1433641930 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd54801abd0 con 0x555be555b350 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd55ab1c740 10 client.15711 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd55ab1c740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd55ab1c740 20 client.15711 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55f1cf0 0x555be55f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd55ab1c740 4 client.15711 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:46:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555be54c8d60 con 0x555be55f1cf0 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd555cde640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55f1cf0 0x555be55f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd539ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.585+0000 7fd555cde640 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55f1cf0 0x555be55f40e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9b65c0fdfee67d server_cookie=2282a55b54620400 in_seq=0 out_seq=0 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:46:20.591 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd539ffb640 1 -- 172.21.15.143:0/1433641930 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555be54c8d60 con 0x555be55f1cf0 2024-09-10T10:46:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd539ffb640 10 client.15711 handle_command_reply: tid=0 2024-09-10T10:46:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 client.15711 shutdown 2024-09-10T10:46:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 msgr2=0x555be5573ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5573ab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd548009990 tx=0x7fd548019da0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 shutdown_connections 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555be55f1cf0 0x555be55f40e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd539ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555be556aa80 0x555be5574530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555be555bc70 0x555be5573ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 --2- 172.21.15.143:0/1433641930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555be555b350 0x555be5573ab0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 >> 172.21.15.143:0/1433641930 conn(0x555be5557a20 msgr2=0x555be5568ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 shutdown_connections 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 1 -- 172.21.15.143:0/1433641930 wait complete. 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.586+0000 7fd55ab1c740 20 client.15711 trim_cache size 0 max 16384 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd53c0a2cd0 msgr2=0x7fd53c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd53c0a2cd0 0x7fd53c0a5190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd54c02a460 tx=0x7fd54c08d810 comp rx=0 tx=0).stop 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 msgr2=0x7fd5501985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:20.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 0x7fd5501985d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd54c005ab0 tx=0x7fd54c005dc0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 shutdown_connections 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd53c0a2cd0 0x7fd53c0a5190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5501082a0 0x7fd55019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5501036e0 0x7fd550198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 --2- 172.21.15.143:0/1129331282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd550102d30 0x7fd5501985d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 >> 172.21.15.143:0/1129331282 conn(0x7fd5500fc7d0 msgr2=0x7fd550106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 shutdown_connections 2024-09-10T10:46:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:20.587+0000 7fd558743640 1 -- 172.21.15.143:0/1129331282 wait complete. 2024-09-10T10:46:20.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:20 smithi143 ceph-mon[31057]: pgmap v389: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T10:46:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:20 smithi178 ceph-mon[34813]: pgmap v389: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T10:46:20.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:20 smithi144 ceph-mon[34230]: pgmap v389: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T10:46:21.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:46:21.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:46:21.714 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 -- 172.21.15.143:0/3159300125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c100e20 msgr2=0x7f959c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 --2- 172.21.15.143:0/3159300125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c100e20 0x7f959c101200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9598009900 tx=0x7f959802a480 comp rx=0 tx=0).stop 2024-09-10T10:46:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 -- 172.21.15.143:0/3159300125 shutdown_connections 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 --2- 172.21.15.143:0/3159300125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c102ef0 0x7f959c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 --2- 172.21.15.143:0/3159300125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c102570 0x7f959c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 --2- 172.21.15.143:0/3159300125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c100e20 0x7f959c101200 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 -- 172.21.15.143:0/3159300125 >> 172.21.15.143:0/3159300125 conn(0x7f959c0fc7d0 msgr2=0x7f959c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 -- 172.21.15.143:0/3159300125 shutdown_connections 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.227+0000 7f95a36c6640 1 -- 172.21.15.143:0/3159300125 wait complete. 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 Processor -- start 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 -- start start 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100e20 0x7f959c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c102570 0x7f959c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 0x7f959c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f959c111f00 con 0x7f959c102ef0 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f959c111d80 con 0x7f959c100e20 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a36c6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f959c112080 con 0x7f959c102570 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a1c63640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 0x7f959c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a1c63640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 0x7f959c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57294/0 (socket says 172.21.15.143:57294) 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a1c63640 1 -- 172.21.15.143:0/1705466838 learned_addr learned my addr 172.21.15.143:0/1705466838 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.228+0000 7f95a1462640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100e20 0x7f959c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a0c61640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c102570 0x7f959c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a1c63640 1 -- 172.21.15.143:0/1705466838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c102570 msgr2=0x7f959c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a1c63640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c102570 0x7f959c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a1c63640 1 -- 172.21.15.143:0/1705466838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100e20 msgr2=0x7f959c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a1c63640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100e20 0x7f959c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a1c63640 1 -- 172.21.15.143:0/1705466838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c19d410 con 0x7f959c102ef0 2024-09-10T10:46:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a1c63640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 0x7f959c19cd30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f959400eaa0 tx=0x7f959400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400cdb0 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c19d700 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c10e630 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9594005020 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.229+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940116d0 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.230+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9580005290 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.232+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f9594004a40 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.232+0000 7f95927fc640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95880a2c90 0x7f95880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.232+0000 7f95a1462640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95880a2c90 0x7f95880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.233+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95940cf4f0 con 0x7f959c102ef0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.233+0000 7f95a1462640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95880a2c90 0x7f95880a5150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9598009230 tx=0x7f95980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.238+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9594016030 con 0x7f959c102ef0 2024-09-10T10:46:22.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.447+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9580005850 con 0x7f959c102ef0 2024-09-10T10:46:22.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.448+0000 7f95927fc640 1 -- 172.21.15.143:0/1705466838 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f9594084e20 con 0x7f959c102ef0 2024-09-10T10:46:22.449 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:46:22.450 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:46:22.450 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:46:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95880a2c90 msgr2=0x7f95880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95880a2c90 0x7f95880a5150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9598009230 tx=0x7f95980058d0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 msgr2=0x7f959c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 0x7f959c19cd30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f959400eaa0 tx=0x7f959400ef60 comp rx=0 tx=0).stop 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 shutdown_connections 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95880a2c90 0x7f95880a5150 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f959c102ef0 0x7f959c19cd30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c102570 0x7f959c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 --2- 172.21.15.143:0/1705466838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100e20 0x7f959c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 >> 172.21.15.143:0/1705466838 conn(0x7f959c0fc7d0 msgr2=0x7f959c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 shutdown_connections 2024-09-10T10:46:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:22.451+0000 7f95a36c6640 1 -- 172.21.15.143:0/1705466838 wait complete. 2024-09-10T10:46:22.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:22 smithi143 ceph-mon[31057]: pgmap v390: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T10:46:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:22 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1705466838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:22 smithi178 ceph-mon[34813]: pgmap v390: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T10:46:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:22 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1705466838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:22.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:22 smithi144 ceph-mon[34230]: pgmap v390: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T10:46:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:22 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1705466838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:23.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:46:23.481 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:24.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 -- 172.21.15.143:0/2799752741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 msgr2=0x7f995c100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 --2- 172.21.15.143:0/2799752741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c100810 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f994c00aa10 tx=0x7f994c02a610 comp rx=0 tx=0).stop 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 -- 172.21.15.143:0/2799752741 shutdown_connections 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 --2- 172.21.15.143:0/2799752741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 --2- 172.21.15.143:0/2799752741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 0x7f995c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 --2- 172.21.15.143:0/2799752741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c100810 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 -- 172.21.15.143:0/2799752741 >> 172.21.15.143:0/2799752741 conn(0x7f995c077230 msgr2=0x7f995c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 -- 172.21.15.143:0/2799752741 shutdown_connections 2024-09-10T10:46:24.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.067+0000 7f996233f640 1 -- 172.21.15.143:0/2799752741 wait complete. 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 Processor -- start 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 -- start start 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 0x7f995c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10fc10 con 0x7f995c100430 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10fa90 con 0x7f995c105560 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.068+0000 7f996233f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10fd90 con 0x7f995c100de0 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 0x7f995c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 0x7f995c06cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42420/0 (socket says 172.21.15.143:42420) 2024-09-10T10:46:24.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:37818/0 (socket says 172.21.15.143:37818) 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f995b7fe640 1 -- 172.21.15.143:0/898768660 learned_addr learned my addr 172.21.15.143:0/898768660 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f995bfff640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 -- 172.21.15.143:0/898768660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 msgr2=0x7f995c06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 0x7f995c06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 -- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 msgr2=0x7f995c073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 -- 172.21.15.143:0/898768660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c06d6a0 con 0x7f995c105560 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f995bfff640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.069+0000 7f99608dc640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c06d060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f995400cab0 tx=0x7f995400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.070+0000 7f99597fa640 1 -- 172.21.15.143:0/898768660 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995400ae20 con 0x7f995c105560 2024-09-10T10:46:24.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.070+0000 7f99597fa640 1 -- 172.21.15.143:0/898768660 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f995400f900 con 0x7f995c105560 2024-09-10T10:46:24.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.071+0000 7f99597fa640 1 -- 172.21.15.143:0/898768660 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954018a00 con 0x7f995c105560 2024-09-10T10:46:24.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.071+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c071570 con 0x7f995c105560 2024-09-10T10:46:24.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.071+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c071a50 con 0x7f995c105560 2024-09-10T10:46:24.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.073+0000 7f99597fa640 1 -- 172.21.15.143:0/898768660 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f995400fab0 con 0x7f995c105560 2024-09-10T10:46:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.074+0000 7f9964718740 1 Processor -- start 2024-09-10T10:46:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.074+0000 7f9964718740 1 -- start start 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.074+0000 7f9964718740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x55746110b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.074+0000 7f9964718740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x55746110ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.074+0000 7f9964718740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x557461118760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f9964718740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5574610712f0 con 0x55746110b5d0 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f9964718740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557461018cf0 con 0x557461116370 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f9964718740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55746106d2b0 con 0x55746110cb40 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x55746110b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x55746110ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x55746110ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57324/0 (socket says 172.21.15.143:57324) 2024-09-10T10:46:24.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 -- 172.21.15.143:0/866499513 learned_addr learned my addr 172.21.15.143:0/866499513 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:24.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995b7fe640 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x557461118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 -- 172.21.15.143:0/866499513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 msgr2=0x55746110b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x55746110b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f99597fa640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99440a2c90 0x7f99440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 -- 172.21.15.143:0/866499513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 msgr2=0x557461118760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x557461118760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.075+0000 7f995bfff640 1 -- 172.21.15.143:0/866499513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557460ffd3a0 con 0x55746110b5d0 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.076+0000 7f99608dc640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99440a2c90 0x7f99440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.076+0000 7f995bfff640 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x55746110ba10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f994c00aa10 tx=0x7f994c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2376fd49a4027b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.076+0000 7f99597fa640 1 -- 172.21.15.143:0/898768660 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99540cfcd0 con 0x7f995c105560 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.076+0000 7f99608dc640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99440a2c90 0x7f99440a5150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f995401f000 tx=0x7f99540c6ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.076+0000 7f99427fc640 1 -- 172.21.15.143:0/866499513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c035030 con 0x55746110b5d0 2024-09-10T10:46:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.076+0000 7f99427fc640 1 -- 172.21.15.143:0/866499513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f994c0044d0 con 0x55746110b5d0 2024-09-10T10:46:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.077+0000 7f9964718740 1 -- 172.21.15.143:0/866499513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 msgr2=0x55746110ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.077+0000 7f9964718740 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x55746110ba10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f994c00aa10 tx=0x7f994c0042e0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 -- 172.21.15.143:0/866499513 shutdown_connections 2024-09-10T10:46:24.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x557461118760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x55746110ba10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 --2- 172.21.15.143:0/866499513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x55746110b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 -- 172.21.15.143:0/866499513 >> 172.21.15.143:0/866499513 conn(0x5574610d4310 msgr2=0x557461107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 -- 172.21.15.143:0/866499513 shutdown_connections 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 -- 172.21.15.143:0/866499513 wait complete. 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 -- start start 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.078+0000 7f9964718740 1 Processor -- start 2024-09-10T10:46:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x557461115c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746110cb40 0x557461112f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557461116370 0x557461113440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5574611190b0 con 0x55746110b5d0 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55746102c060 con 0x55746110cb40 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557461119230 con 0x557461116370 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f9964718740 10 client.0 _ll_register_callbacks cb 0x557460d7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x557461115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x557461115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57340/0 (socket says 172.21.15.143:57340) 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 -- 172.21.15.143:0/2032806009 learned_addr learned my addr 172.21.15.143:0/2032806009 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995bfff640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746110cb40 0x557461112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557461116370 0x557461113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 -- 172.21.15.143:0/2032806009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557461116370 msgr2=0x557461113440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557461116370 0x557461113440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 -- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746110cb40 msgr2=0x557461112f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746110cb40 0x557461112f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995b7fe640 1 -- 172.21.15.143:0/2032806009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557460de23c0 con 0x55746110b5d0 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.079+0000 7f995bfff640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746110cb40 0x557461112f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.080+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x557461115c50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9950005fd0 tx=0x7f99500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1973a33e07f645ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.080+0000 7f993b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.080+0000 7f993b7fe640 1 -- 172.21.15.143:0/2032806009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950005220 con 0x55746110b5d0 2024-09-10T10:46:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.080+0000 7f993b7fe640 1 -- 172.21.15.143:0/2032806009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99500053c0 con 0x55746110b5d0 2024-09-10T10:46:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.080+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c0044d0 con 0x55746110b5d0 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f993b7fe640 1 -- 172.21.15.143:0/2032806009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9950023050 con 0x55746110b5d0 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 10 client.15729 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55746112d380 con 0x55746110b5d0 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f993b7fe640 1 -- 172.21.15.143:0/2032806009 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f994c0044d0 con 0x55746110b5d0 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f993b7fe640 1 -- 172.21.15.143:0/2032806009 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f995000f890 con 0x55746110b5d0 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 10 client.15729 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 10 client.15729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 20 client.15729 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746112e840 0x557461130c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 4 client.15729 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.081+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55746107e860 con 0x55746112e840 2024-09-10T10:46:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.082+0000 7f995bfff640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746112e840 0x557461130c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.082+0000 7f993b7fe640 10 client.15729 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.083+0000 7f995bfff640 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746112e840 0x557461130c50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e62a40c727604cc0 server_cookie=3150c08893de36d7 in_seq=0 out_seq=0 2024-09-10T10:46:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.083+0000 7f993b7fe640 1 -- 172.21.15.143:0/2032806009 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55746107e860 con 0x55746112e840 2024-09-10T10:46:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.083+0000 7f993b7fe640 10 client.15729 handle_command_reply: tid=0 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.083+0000 7f9964718740 1 client.15729 shutdown 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 msgr2=0x557461115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x557461115c50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9950005fd0 tx=0x7f99500046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 shutdown_connections 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746112e840 0x557461130c50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557461116370 0x557461113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746110cb40 0x557461112f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 --2- 172.21.15.143:0/2032806009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746110b5d0 0x557461115c50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f993b7fe640 0 client.15729 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 >> 172.21.15.143:0/2032806009 conn(0x5574610d4310 msgr2=0x55746110d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 shutdown_connections 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.084+0000 7f9964718740 1 -- 172.21.15.143:0/2032806009 wait complete. 2024-09-10T10:46:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.085+0000 7f9964718740 20 client.15729 trim_cache size 0 max 16384 2024-09-10T10:46:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 Processor -- start 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 -- start start 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x557461117760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557461116370 0x557461117ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746112ac90 0x557461136520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b50d0 con 0x557461116370 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b6c10 con 0x55746112ac90 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f9964718740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b57a0 con 0x55746110cb40 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x557461117760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746112ac90 0x557461136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x557461117760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42448/0 (socket says 172.21.15.143:42448) 2024-09-10T10:46:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f995b7fe640 1 -- 172.21.15.143:0/2364824798 learned_addr learned my addr 172.21.15.143:0/2364824798 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.110+0000 7f995bfff640 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557461116370 0x557461117ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f995b7fe640 1 -- 172.21.15.143:0/2364824798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 msgr2=0x557461117760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x557461117760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f995b7fe640 1 -- 172.21.15.143:0/2364824798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557461116370 msgr2=0x557461117ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557461116370 0x557461117ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f995b7fe640 1 -- 172.21.15.143:0/2364824798 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557461118f50 con 0x55746112ac90 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746112ac90 0x557461136520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f995000fc60 tx=0x7f995000b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd37cbcadf6d2cac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f9941ffb640 1 -- 172.21.15.143:0/2364824798 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000f7d0 con 0x55746112ac90 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f9941ffb640 1 -- 172.21.15.143:0/2364824798 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f995000f970 con 0x55746112ac90 2024-09-10T10:46:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.111+0000 7f9941ffb640 1 -- 172.21.15.143:0/2364824798 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950028440 con 0x55746112ac90 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.112+0000 7f9964718740 1 -- 172.21.15.143:0/2364824798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746112ac90 msgr2=0x557461136520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.112+0000 7f9964718740 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746112ac90 0x557461136520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f995000fc60 tx=0x7f995000b9c0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.112+0000 7f9964718740 1 -- 172.21.15.143:0/2364824798 shutdown_connections 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55746112ac90 0x557461136520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557461116370 0x557461117ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 --2- 172.21.15.143:0/2364824798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x557461117760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 -- 172.21.15.143:0/2364824798 >> 172.21.15.143:0/2364824798 conn(0x5574610d4310 msgr2=0x557461112ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 -- 172.21.15.143:0/2364824798 shutdown_connections 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 -- 172.21.15.143:0/2364824798 wait complete. 2024-09-10T10:46:24.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 -- start start 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.113+0000 7f9964718740 1 Processor -- start 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x5574611272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x5574611102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 0x557461110830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b6540 con 0x55746112ac90 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b5e70 con 0x557461116370 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b30b0 con 0x55746110cb40 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f9964718740 10 client.0 _ll_register_callbacks cb 0x7f995000bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 0x557461110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 0x557461110830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57354/0 (socket says 172.21.15.143:57354) 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 -- 172.21.15.143:0/2492752637 learned_addr learned my addr 172.21.15.143:0/2492752637 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x5574611272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 -- 172.21.15.143:0/2492752637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 msgr2=0x5574611272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x5574611272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995bfff640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x5574611102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.114+0000 7f995b7fe640 1 -- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 msgr2=0x5574611102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x5574611102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f995b7fe640 1 -- 172.21.15.143:0/2492752637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557461110df0 con 0x55746112ac90 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f995bfff640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x5574611102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f995b7fe640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 0x557461110830 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f995000b5a0 tx=0x7f99500044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dce810dc9d2696 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f993b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f993b7fe640 1 -- 172.21.15.143:0/2492752637 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950010da0 con 0x55746112ac90 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995000f970 con 0x55746112ac90 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.115+0000 7f993b7fe640 1 -- 172.21.15.143:0/2492752637 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9950010460 con 0x55746112ac90 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f993b7fe640 1 -- 172.21.15.143:0/2492752637 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9950042050 con 0x55746112ac90 2024-09-10T10:46:24.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55746119a500 con 0x55746112ac90 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f993b7fe640 1 -- 172.21.15.143:0/2492752637 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f995000f970 con 0x55746112ac90 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f993b7fe640 1 -- 172.21.15.143:0/2492752637 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9950029ac0 con 0x55746112ac90 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 10 client.15735 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 20 client.15735 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746119aa10 0x55746119ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 4 client.15735 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.116+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557461078d70 con 0x55746119aa10 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.117+0000 7f995bfff640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746119aa10 0x55746119ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.117+0000 7f993b7fe640 10 client.15735 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:24.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.117+0000 7f995bfff640 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746119aa10 0x55746119ce00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21615652f54fd0e2 server_cookie=265d6aff64c2155c in_seq=0 out_seq=0 2024-09-10T10:46:24.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f993b7fe640 1 -- 172.21.15.143:0/2492752637 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557461078d70 con 0x55746119aa10 2024-09-10T10:46:24.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f993b7fe640 10 client.15735 handle_command_reply: tid=0 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 client.15735 shutdown 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 msgr2=0x557461110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 0x557461110830 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f995000b5a0 tx=0x7f99500044f0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 shutdown_connections 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55746119aa10 0x55746119ce00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f993b7fe640 0 client.15735 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55746112ac90 0x557461110830 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557461116370 0x5574611102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 --2- 172.21.15.143:0/2492752637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55746110cb40 0x5574611272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.118+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 >> 172.21.15.143:0/2492752637 conn(0x5574610d4310 msgr2=0x5574611064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.119+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 shutdown_connections 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.119+0000 7f9964718740 1 -- 172.21.15.143:0/2492752637 wait complete. 2024-09-10T10:46:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.119+0000 7f9964718740 20 client.15735 trim_cache size 0 max 16384 2024-09-10T10:46:24.124 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:24.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99440a2c90 msgr2=0x7f99440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99440a2c90 0x7f99440a5150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f995401f000 tx=0x7f99540c6ba0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 msgr2=0x7f995c06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c06d060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f995400cab0 tx=0x7f995400cf70 comp rx=0 tx=0).stop 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 shutdown_connections 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99440a2c90 0x7f99440a5150 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c105560 0x7f995c06d060 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f995c100de0 0x7f995c06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 --2- 172.21.15.143:0/898768660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f995c100430 0x7f995c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 >> 172.21.15.143:0/898768660 conn(0x7f995c077230 msgr2=0x7f995c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 shutdown_connections 2024-09-10T10:46:24.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:24.120+0000 7f996233f640 1 -- 172.21.15.143:0/898768660 wait complete. 2024-09-10T10:46:24.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:24 smithi143 ceph-mon[31057]: pgmap v391: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T10:46:24.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:46:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:24 smithi178 ceph-mon[34813]: pgmap v391: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T10:46:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:24 smithi144 ceph-mon[34230]: pgmap v391: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T10:46:25.256 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:25.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 -- 172.21.15.143:0/2044309182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102ef0 msgr2=0x7f072810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 --2- 172.21.15.143:0/2044309182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102ef0 0x7f072810da40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0720009ca0 tx=0x7f072002a950 comp rx=0 tx=0).stop 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 -- 172.21.15.143:0/2044309182 shutdown_connections 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 --2- 172.21.15.143:0/2044309182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102ef0 0x7f072810da40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 --2- 172.21.15.143:0/2044309182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728102570 0x7f07281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 --2- 172.21.15.143:0/2044309182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0728100e20 0x7f0728101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 -- 172.21.15.143:0/2044309182 >> 172.21.15.143:0/2044309182 conn(0x7f07280fc7d0 msgr2=0x7f07280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 -- 172.21.15.143:0/2044309182 shutdown_connections 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.814+0000 7f072fc9a640 1 -- 172.21.15.143:0/2044309182 wait complete. 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072fc9a640 1 Processor -- start 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072fc9a640 1 -- start start 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072fc9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728100e20 0x7f07281983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072fc9a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 0x7f0728198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072fc9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0728102ef0 0x7f072819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072fc9a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0728110e30 con 0x7f0728102570 2024-09-10T10:46:25.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.815+0000 7f072d235640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 0x7f0728198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728100e20 0x7f07281983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072d235640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 0x7f0728198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57370/0 (socket says 172.21.15.143:57370) 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072e237640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0728102ef0 0x7f072819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072d235640 1 -- 172.21.15.143:0/1331792431 learned_addr learned my addr 172.21.15.143:0/1331792431 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0728110cb0 con 0x7f0728100e20 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0728110fb0 con 0x7f0728102ef0 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 -- 172.21.15.143:0/1331792431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0728102ef0 msgr2=0x7f072819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0728102ef0 0x7f072819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 -- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 msgr2=0x7f0728198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 0x7f0728198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 -- 172.21.15.143:0/1331792431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072819d390 con 0x7f0728100e20 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072d235640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 0x7f0728198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f072da36640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728100e20 0x7f07281983e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f072400e9f0 tx=0x7f072400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.816+0000 7f071effd640 1 -- 172.21.15.143:0/1331792431 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072400ccc0 con 0x7f0728100e20 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.817+0000 7f071effd640 1 -- 172.21.15.143:0/1331792431 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0724005020 con 0x7f0728100e20 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.817+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072819d680 con 0x7f0728100e20 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.817+0000 7f071effd640 1 -- 172.21.15.143:0/1331792431 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724004a00 con 0x7f0728100e20 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.817+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07281a4f60 con 0x7f0728100e20 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.820+0000 7f071effd640 1 -- 172.21.15.143:0/1331792431 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f0724004ba0 con 0x7f0728100e20 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.820+0000 7f0732073740 1 Processor -- start 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.820+0000 7f0732073740 1 -- start start 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f0732073740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f0732073740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072e237640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072e237640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db55090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:37870/0 (socket says 172.21.15.143:37870) 2024-09-10T10:46:25.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f0732073740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072e237640 1 -- 172.21.15.143:0/1336645253 learned_addr learned my addr 172.21.15.143:0/1336645253 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55999dabb2f0 con 0x55999db555d0 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55999da62cf0 con 0x55999db56b40 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072da36640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55999dab72b0 con 0x55999db60370 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072da36640 1 -- 172.21.15.143:0/1336645253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 msgr2=0x55999db62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072d235640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072da36640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072da36640 1 -- 172.21.15.143:0/1336645253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 msgr2=0x55999db55090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.821+0000 7f072da36640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db55090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f072da36640 1 -- 172.21.15.143:0/1336645253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999da473a0 con 0x55999db555d0 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f072e237640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db55090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:25.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f072d235640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f072da36640 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db55a10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f07240cca60 tx=0x7f07240dfe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eeb75c56d5aa49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f071effd640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07140a2c90 0x7f07140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f071effd640 1 -- 172.21.15.143:0/1331792431 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07240cf4a0 con 0x7f0728100e20 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f072e237640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07140a2c90 0x7f07140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f0713fff640 1 -- 172.21.15.143:0/1336645253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724097a80 con 0x55999db555d0 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f0713fff640 1 -- 172.21.15.143:0/1336645253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f072409bee0 con 0x55999db555d0 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.822+0000 7f0713fff640 1 -- 172.21.15.143:0/1336645253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072409b140 con 0x55999db555d0 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.823+0000 7f072e237640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07140a2c90 0x7f07140a5150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0720009ca0 tx=0x7f0720005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 msgr2=0x55999db55a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db55a10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f07240cca60 tx=0x7f07240dfe70 comp rx=0 tx=0).stop 2024-09-10T10:46:25.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 shutdown_connections 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db55a10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 --2- 172.21.15.143:0/1336645253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db55090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 >> 172.21.15.143:0/1336645253 conn(0x55999db1e310 msgr2=0x55999db51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 shutdown_connections 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 -- 172.21.15.143:0/1336645253 wait complete. 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 -- start start 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.824+0000 7f0732073740 1 Processor -- start 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f0732073740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db5fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f0732073740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db56b40 0x55999db5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f072da36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db5fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f072da36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db5fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57376/0 (socket says 172.21.15.143:57376) 2024-09-10T10:46:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f072da36640 1 -- 172.21.15.143:0/2690966249 learned_addr learned my addr 172.21.15.143:0/2690966249 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db60370 0x55999db5d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55999db63070 con 0x55999db555d0 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55999da76060 con 0x55999db60370 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55999db631f0 con 0x55999db56b40 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.825+0000 7f072e237640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db60370 0x55999db5d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072d235640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db56b40 0x55999db5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072da36640 1 -- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db56b40 msgr2=0x55999db5d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072da36640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db56b40 0x55999db5d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072da36640 1 -- 172.21.15.143:0/2690966249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db60370 msgr2=0x55999db5d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072da36640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db60370 0x55999db5d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072da36640 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999d82c3c0 con 0x55999db555d0 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f072da36640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db5fd70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f07240a3f20 tx=0x7f07240d3c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bfa09902940665 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f07117fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f0732073740 10 client.0 _ll_register_callbacks cb 0x55999d7c98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f072409bee0 con 0x55999db555d0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f07117fa640 1 -- 172.21.15.143:0/2690966249 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072401c030 con 0x55999db555d0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f07117fa640 1 -- 172.21.15.143:0/2690966249 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07240cf190 con 0x55999db555d0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f07117fa640 1 -- 172.21.15.143:0/2690966249 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f072409b320 con 0x55999db555d0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f0732073740 10 client.15747 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.826+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55999db7f630 con 0x55999db555d0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f07117fa640 1 -- 172.21.15.143:0/2690966249 <== mon.0 v2:172.21.15.143:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f07240860b0 con 0x55999db555d0 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f0732073740 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f0732073740 10 client.15747 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f0732073740 20 client.15747 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999db82d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f0732073740 4 client.15747 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:25.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f072409bee0 con 0x55999db80980 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.827+0000 7f072d235640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999db82d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f07117fa640 10 client.15747 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f072d235640 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999db82d70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe876f90e1420f2 server_cookie=597b02d9d005c366 in_seq=0 out_seq=0 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f07117fa640 1 -- 172.21.15.143:0/2690966249 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f072409bee0 con 0x55999db80980 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f07117fa640 10 client.15747 handle_command_reply: tid=0 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f0732073740 1 client.15747 shutdown 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 msgr2=0x55999db5fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.828+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db5fd70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f07240a3f20 tx=0x7f07240d3c10 comp rx=0 tx=0).stop 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 shutdown_connections 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999db82d70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db60370 0x55999db5d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f07117fa640 0 client.15747 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db56b40 0x55999db5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 --2- 172.21.15.143:0/2690966249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db5fd70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 >> 172.21.15.143:0/2690966249 conn(0x55999db1e310 msgr2=0x55999db57930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 shutdown_connections 2024-09-10T10:46:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 1 -- 172.21.15.143:0/2690966249 wait complete. 2024-09-10T10:46:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.829+0000 7f0732073740 20 client.15747 trim_cache size 0 max 16384 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f0732073740 1 Processor -- start 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f0732073740 1 -- start start 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f0732073740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db7d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f0732073740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db5c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f0732073740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f0732073740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f07280732d0 con 0x55999db555d0 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.854+0000 7f072da36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db7d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072da36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db7d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57388/0 (socket says 172.21.15.143:57388) 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072da36640 1 -- 172.21.15.143:0/76346889 learned_addr learned my addr 172.21.15.143:0/76346889 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:25.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db5c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072e237640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07281a5b20 con 0x55999db56b40 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07280739a0 con 0x55999db60370 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 -- 172.21.15.143:0/76346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 msgr2=0x55999db884e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db884e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 -- 172.21.15.143:0/76346889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 msgr2=0x55999db7d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db7d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 -- 172.21.15.143:0/76346889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999db51dc0 con 0x55999db56b40 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072e237640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072da36640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db7d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f072d235640 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db5c020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55999db5c8b0 tx=0x7f071800a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dd0f81dcd5fd561 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.855+0000 7f07137fe640 1 -- 172.21.15.143:0/76346889 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071801f070 con 0x55999db56b40 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.856+0000 7f07137fe640 1 -- 172.21.15.143:0/76346889 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0718013b30 con 0x55999db56b40 2024-09-10T10:46:25.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.856+0000 7f07137fe640 1 -- 172.21.15.143:0/76346889 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071801b9f0 con 0x55999db56b40 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 msgr2=0x55999db5c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db5c020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55999db5c8b0 tx=0x7f071800a380 comp rx=0 tx=0).stop 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 shutdown_connections 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db5c020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 --2- 172.21.15.143:0/76346889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db7d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 >> 172.21.15.143:0/76346889 conn(0x55999db1e310 msgr2=0x55999db5b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 shutdown_connections 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 -- 172.21.15.143:0/76346889 wait complete. 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.857+0000 7f0732073740 1 -- start start 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.858+0000 7f0732073740 1 Processor -- start 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.858+0000 7f0732073740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.858+0000 7f0732073740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db59e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.858+0000 7f0732073740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db5a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.858+0000 7f072da36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db79d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57402/0 (socket says 172.21.15.143:57402) 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 -- 172.21.15.143:0/274451656 learned_addr learned my addr 172.21.15.143:0/274451656 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f07281a5450 con 0x55999db555d0 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0728074070 con 0x55999db56b40 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07280712b0 con 0x55999db60370 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f0732073740 10 client.0 _ll_register_callbacks cb 0x7f0720009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072d235640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db59e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 -- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 msgr2=0x55999db5a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072e237640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db5a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db5a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 -- 172.21.15.143:0/274451656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 msgr2=0x55999db59e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db59e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.859+0000 7f072da36640 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999db5ac10 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f07117fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f072da36640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db79d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f072409b4e0 tx=0x7f07240054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2fdd3c7a0d9f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f07117fa640 1 -- 172.21.15.143:0/274451656 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072401c070 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f07117fa640 1 -- 172.21.15.143:0/274451656 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07240051d0 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0718013b30 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f07117fa640 1 -- 172.21.15.143:0/274451656 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f072409b5b0 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.860+0000 7f0732073740 10 client.15753 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55999db5ac10 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f07117fa640 1 -- 172.21.15.143:0/274451656 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0718013b30 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f07117fa640 1 -- 172.21.15.143:0/274451656 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f07240ccce0 con 0x55999db555d0 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 10 client.15753 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:25.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 10 client.15753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 20 client.15753 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999dbed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 4 client.15753 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.861+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55999dac8860 con 0x55999db80980 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.862+0000 7f072d235640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999dbed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.862+0000 7f07117fa640 10 client.15753 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.862+0000 7f072d235640 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999dbed930 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9b5949319883f6a server_cookie=e11e5543986965d in_seq=0 out_seq=0 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f07117fa640 1 -- 172.21.15.143:0/274451656 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55999dac8860 con 0x55999db80980 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f07117fa640 10 client.15753 handle_command_reply: tid=0 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 client.15753 shutdown 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 msgr2=0x55999db79d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db79d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f072409b4e0 tx=0x7f07240054d0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 shutdown_connections 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55999db80980 0x55999dbed930 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999db60370 0x55999db5a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f07117fa640 0 client.15753 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55999db56b40 0x55999db59e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 --2- 172.21.15.143:0/274451656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55999db555d0 0x55999db79d10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.863+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 >> 172.21.15.143:0/274451656 conn(0x55999db1e310 msgr2=0x55999db5c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 shutdown_connections 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f0732073740 1 -- 172.21.15.143:0/274451656 wait complete. 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f0732073740 20 client.15753 trim_cache size 0 max 16384 2024-09-10T10:46:25.871 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:25.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07140a2c90 msgr2=0x7f07140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f072fc9a640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07140a2c90 0x7f07140a5150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0720009ca0 tx=0x7f0720005c50 comp rx=0 tx=0).stop 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728100e20 msgr2=0x7f07281983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.864+0000 7f072fc9a640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728100e20 0x7f07281983e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f072400e9f0 tx=0x7f072400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 shutdown_connections 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07140a2c90 0x7f07140a5150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0728102ef0 0x7f072819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0728102570 0x7f0728198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 --2- 172.21.15.143:0/1331792431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0728100e20 0x7f07281983e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 >> 172.21.15.143:0/1331792431 conn(0x7f07280fc7d0 msgr2=0x7f07281038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 shutdown_connections 2024-09-10T10:46:25.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:25.865+0000 7f072fc9a640 1 -- 172.21.15.143:0/1331792431 wait complete. 2024-09-10T10:46:26.618 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:46:26.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:26 smithi143 ceph-mon[31057]: pgmap v392: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:46:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:26 smithi178 ceph-mon[34813]: pgmap v392: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:46:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:26 smithi144 ceph-mon[34230]: pgmap v392: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:46:26.988 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.508+0000 7f645defa640 1 -- 172.21.15.143:0/2368766126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 msgr2=0x7f6458101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.508+0000 7f645defa640 1 --2- 172.21.15.143:0/2368766126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f6458101200 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6448009900 tx=0x7f644802a480 comp rx=0 tx=0).stop 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 -- 172.21.15.143:0/2368766126 shutdown_connections 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 --2- 172.21.15.143:0/2368766126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6458102ef0 0x7f645810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 --2- 172.21.15.143:0/2368766126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6458102570 0x7f64581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 --2- 172.21.15.143:0/2368766126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f6458101200 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 -- 172.21.15.143:0/2368766126 >> 172.21.15.143:0/2368766126 conn(0x7f64580fc7d0 msgr2=0x7f64580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 -- 172.21.15.143:0/2368766126 shutdown_connections 2024-09-10T10:46:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.509+0000 7f645defa640 1 -- 172.21.15.143:0/2368766126 wait complete. 2024-09-10T10:46:27.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 Processor -- start 2024-09-10T10:46:27.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 -- start start 2024-09-10T10:46:27.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f64581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6458102570 0x7f6458198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6458102ef0 0x7f645819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6458110e00 con 0x7f6458100e20 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6458110c80 con 0x7f6458102ef0 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f645defa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6458110f80 con 0x7f6458102570 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f64577fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f64581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f64577fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f64581983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42762/0 (socket says 172.21.15.143:42762) 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f64577fe640 1 -- 172.21.15.143:0/3235894602 learned_addr learned my addr 172.21.15.143:0/3235894602 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f6457fff640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6458102ef0 0x7f645819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.510+0000 7f6456ffd640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6458102570 0x7f6458198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f64577fe640 1 -- 172.21.15.143:0/3235894602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6458102570 msgr2=0x7f6458198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f64577fe640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6458102570 0x7f6458198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f64577fe640 1 -- 172.21.15.143:0/3235894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6458102ef0 msgr2=0x7f645819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f64577fe640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6458102ef0 0x7f645819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f64577fe640 1 -- 172.21.15.143:0/3235894602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645819d380 con 0x7f6458100e20 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f64577fe640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f64581983d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6448006010 tx=0x7f64480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f6454ff9640 1 -- 172.21.15.143:0/3235894602 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644803f030 con 0x7f6458100e20 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645819d610 con 0x7f6458100e20 2024-09-10T10:46:27.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f6454ff9640 1 -- 172.21.15.143:0/3235894602 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6448003cb0 con 0x7f6458100e20 2024-09-10T10:46:27.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f6454ff9640 1 -- 172.21.15.143:0/3235894602 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448035070 con 0x7f6458100e20 2024-09-10T10:46:27.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.511+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64580fcbb0 con 0x7f6458100e20 2024-09-10T10:46:27.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.514+0000 7f64602d3740 1 Processor -- start 2024-09-10T10:46:27.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.514+0000 7f64602d3740 1 -- start start 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f64602d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859135090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f64602d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859135a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f64602d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859140370 0x562859142760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f64602d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56285909b2f0 con 0x562859140370 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f64602d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562859042cf0 con 0x562859136b40 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f64602d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5628590972b0 con 0x5628591355d0 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859140370 0x562859142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859135090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56038/0 (socket says 172.21.15.143:56038) 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 -- 172.21.15.143:0/2259408262 learned_addr learned my addr 172.21.15.143:0/2259408262 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6454ff9640 1 -- 172.21.15.143:0/3235894602 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6448042030 con 0x7f6458100e20 2024-09-10T10:46:27.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 -- 172.21.15.143:0/2259408262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 msgr2=0x562859135a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859135a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 -- 172.21.15.143:0/2259408262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859140370 msgr2=0x562859142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859140370 0x562859142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.515+0000 7f6457fff640 1 -- 172.21.15.143:0/2259408262 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628590273a0 con 0x562859136b40 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.516+0000 7f6457fff640 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859135090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6450009f70 tx=0x7f6450010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9906d586bb6a034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.516+0000 7f643dffb640 1 -- 172.21.15.143:0/2259408262 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450016070 con 0x562859136b40 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.516+0000 7f643dffb640 1 -- 172.21.15.143:0/2259408262 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6450004d20 con 0x562859136b40 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.516+0000 7f643dffb640 1 -- 172.21.15.143:0/2259408262 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450012790 con 0x562859136b40 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.516+0000 7f6454ff9640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64400a2d10 0x7f64400a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.516+0000 7f6454ff9640 1 -- 172.21.15.143:0/3235894602 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64480ef990 con 0x7f6458100e20 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64577fe640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64400a2d10 0x7f64400a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 -- 172.21.15.143:0/2259408262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 msgr2=0x562859135090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859135090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6450009f70 tx=0x7f6450010800 comp rx=0 tx=0).stop 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 -- 172.21.15.143:0/2259408262 shutdown_connections 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859140370 0x562859142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859135a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 --2- 172.21.15.143:0/2259408262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859135090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 -- 172.21.15.143:0/2259408262 >> 172.21.15.143:0/2259408262 conn(0x5628590fe310 msgr2=0x562859131c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64577fe640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64400a2d10 0x7f64400a51d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6448005ab0 tx=0x7f6448005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 -- 172.21.15.143:0/2259408262 shutdown_connections 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.517+0000 7f64602d3740 1 -- 172.21.15.143:0/2259408262 wait complete. 2024-09-10T10:46:27.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.518+0000 7f64602d3740 1 -- start start 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.518+0000 7f64602d3740 1 Processor -- start 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.518+0000 7f64602d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x56285913fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.518+0000 7f64602d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.518+0000 7f64602d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.518+0000 7f64602d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562859143070 con 0x562859136b40 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f64602d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562859056060 con 0x562859140370 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f64602d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5628591431f0 con 0x5628591355d0 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f64602d3740 10 client.0 _ll_register_callbacks cb 0x562858da98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42778/0 (socket says 172.21.15.143:42778) 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 -- 172.21.15.143:0/1579189067 learned_addr learned my addr 172.21.15.143:0/1579189067 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:27.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6457fff640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x56285913fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 -- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 msgr2=0x56285913fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x56285913fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 -- 172.21.15.143:0/1579189067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 msgr2=0x56285913d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 -- 172.21.15.143:0/1579189067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562858e0c3c0 con 0x562859136b40 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6457fff640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x56285913fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6456ffd640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913d0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f644c010820 tx=0x7f644c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e2ba95c5050769 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.519+0000 7f6436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f6436ffd640 1 -- 172.21.15.143:0/1579189067 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c00fb90 con 0x562859136b40 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f6436ffd640 1 -- 172.21.15.143:0/1579189067 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f644c00fd30 con 0x562859136b40 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6450004d20 con 0x562859136b40 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f6436ffd640 1 -- 172.21.15.143:0/1579189067 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f644c024050 con 0x562859136b40 2024-09-10T10:46:27.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f64602d3740 10 client.15765 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562859157410 con 0x562859136b40 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.520+0000 7f6436ffd640 1 -- 172.21.15.143:0/1579189067 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6450004d20 con 0x562859136b40 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f6436ffd640 1 -- 172.21.15.143:0/1579189067 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f644c00e4e0 con 0x562859136b40 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f64602d3740 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f64602d3740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f64602d3740 20 client.15765 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f64602d3740 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628591589c0 0x56285915ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f64602d3740 4 client.15765 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628590a8860 con 0x5628591589c0 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.521+0000 7f6457fff640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628591589c0 0x56285915ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.522+0000 7f6457fff640 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628591589c0 0x56285915ae20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a225fcdfcf642a9e server_cookie=d467ffc39df46360 in_seq=0 out_seq=0 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.522+0000 7f6436ffd640 10 client.15765 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:27.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f6436ffd640 1 -- 172.21.15.143:0/1579189067 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628590a8860 con 0x5628591589c0 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f6436ffd640 10 client.15765 handle_command_reply: tid=0 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 client.15765 shutdown 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 msgr2=0x56285913d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913d0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f644c010820 tx=0x7f644c010ce0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 shutdown_connections 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628591589c0 0x56285915ae20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913d0a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 --2- 172.21.15.143:0/1579189067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x56285913fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 >> 172.21.15.143:0/1579189067 conn(0x5628590fe310 msgr2=0x562859137960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f6436ffd640 0 client.15765 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 shutdown_connections 2024-09-10T10:46:27.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 1 -- 172.21.15.143:0/1579189067 wait complete. 2024-09-10T10:46:27.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.523+0000 7f64602d3740 20 client.15765 trim_cache size 0 max 16384 2024-09-10T10:46:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.548+0000 7f64602d3740 1 Processor -- start 2024-09-10T10:46:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.548+0000 7f64602d3740 1 -- start start 2024-09-10T10:46:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f64602d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f64602d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42786/0 (socket says 172.21.15.143:42786) 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f64602d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 -- 172.21.15.143:0/4293640804 learned_addr learned my addr 172.21.15.143:0/4293640804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f64581ac8c0 con 0x562859136b40 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64581ae400 con 0x562859140370 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64581acf90 con 0x5628591355d0 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6457fff640 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6457fff640 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 -- 172.21.15.143:0/4293640804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 msgr2=0x562859154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 -- 172.21.15.143:0/4293640804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 msgr2=0x56285913f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.549+0000 7f6456ffd640 1 -- 172.21.15.143:0/4293640804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562859132dd0 con 0x562859136b40 2024-09-10T10:46:27.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.550+0000 7f6456ffd640 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913c960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f644c00ee30 tx=0x7f644c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569cbde3be7bef59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.550+0000 7f643d7fa640 1 -- 172.21.15.143:0/4293640804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c02d070 con 0x562859136b40 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.550+0000 7f643d7fa640 1 -- 172.21.15.143:0/4293640804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f644c00a340 con 0x562859136b40 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.550+0000 7f643d7fa640 1 -- 172.21.15.143:0/4293640804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c004660 con 0x562859136b40 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 msgr2=0x56285913c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913c960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f644c00ee30 tx=0x7f644c00ee60 comp rx=0 tx=0).stop 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 shutdown_connections 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859140370 0x56285913f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562859136b40 0x56285913c960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 --2- 172.21.15.143:0/4293640804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628591355d0 0x562859154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 >> 172.21.15.143:0/4293640804 conn(0x5628590fe310 msgr2=0x56285913e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 shutdown_connections 2024-09-10T10:46:27.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.551+0000 7f64602d3740 1 -- 172.21.15.143:0/4293640804 wait complete. 2024-09-10T10:46:27.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 -- start start 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 Processor -- start 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 0x562859150d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859151250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562859140370 0x5628591396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f64581add30 con 0x5628591355d0 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64581ad660 con 0x562859136b40 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64581aa8a0 con 0x562859140370 2024-09-10T10:46:27.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f64602d3740 10 client.0 _ll_register_callbacks cb 0x7f644c00a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f6457fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 0x562859150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f6457fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 0x562859150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42802/0 (socket says 172.21.15.143:42802) 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.552+0000 7f6457fff640 1 -- 172.21.15.143:0/2305839569 learned_addr learned my addr 172.21.15.143:0/2305839569 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6457fff640 1 -- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562859140370 msgr2=0x5628591396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6457fff640 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562859140370 0x5628591396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6457fff640 1 -- 172.21.15.143:0/2305839569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 msgr2=0x562859151250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6457fff640 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859151250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6457fff640 1 -- 172.21.15.143:0/2305839569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562859139c10 con 0x5628591355d0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6457fff640 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 0x562859150d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6450009f70 tx=0x7f6450004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2a13baf845c4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.553+0000 7f6436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f6436ffd640 1 -- 172.21.15.143:0/2305839569 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645002a070 con 0x5628591355d0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f6436ffd640 1 -- 172.21.15.143:0/2305839569 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6450027ba0 con 0x5628591355d0 2024-09-10T10:46:27.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644c00a340 con 0x5628591355d0 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f6436ffd640 1 -- 172.21.15.143:0/2305839569 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6450016050 con 0x5628591355d0 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f64602d3740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562859143d00 con 0x5628591355d0 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.554+0000 7f6436ffd640 1 -- 172.21.15.143:0/2305839569 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f645005a920 con 0x5628591355d0 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f6436ffd640 1 -- 172.21.15.143:0/2305839569 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f645000f040 con 0x5628591355d0 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f64602d3740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f64602d3740 10 client.15771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f64602d3740 20 client.15771 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f64602d3740 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562859144250 0x5628591466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f64602d3740 4 client.15771 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f644c00a340 con 0x562859144250 2024-09-10T10:46:27.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.555+0000 7f6456ffd640 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562859144250 0x5628591466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.556+0000 7f6436ffd640 10 client.15771 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:27.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.556+0000 7f6456ffd640 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562859144250 0x5628591466b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6682c1d80aff41d5 server_cookie=9d327783c3c184f2 in_seq=0 out_seq=0 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.556+0000 7f6436ffd640 1 -- 172.21.15.143:0/2305839569 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f644c00a340 con 0x562859144250 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.556+0000 7f6436ffd640 10 client.15771 handle_command_reply: tid=0 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.556+0000 7f64602d3740 1 client.15771 shutdown 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 msgr2=0x562859150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 0x562859150d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6450009f70 tx=0x7f6450004a10 comp rx=0 tx=0).stop 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 shutdown_connections 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562859144250 0x5628591466b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562859140370 0x5628591396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562859136b40 0x562859151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f6436ffd640 0 client.15771 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 --2- 172.21.15.143:0/2305839569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628591355d0 0x562859150d10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 >> 172.21.15.143:0/2305839569 conn(0x5628590fe310 msgr2=0x56285913e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 shutdown_connections 2024-09-10T10:46:27.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 1 -- 172.21.15.143:0/2305839569 wait complete. 2024-09-10T10:46:27.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.557+0000 7f64602d3740 20 client.15771 trim_cache size 0 max 16384 2024-09-10T10:46:27.561 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:27.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64400a2d10 msgr2=0x7f64400a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64400a2d10 0x7f64400a51d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6448005ab0 tx=0x7f6448005e00 comp rx=0 tx=0).stop 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 msgr2=0x7f64581983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f64581983d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6448006010 tx=0x7f64480042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 shutdown_connections 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64400a2d10 0x7f64400a51d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6458102ef0 0x7f645819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6458102570 0x7f6458198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 --2- 172.21.15.143:0/3235894602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458100e20 0x7f64581983d0 secure :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6448006010 tx=0x7f64480042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.558+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 >> 172.21.15.143:0/3235894602 conn(0x7f64580fc7d0 msgr2=0x7f645806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.559+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 shutdown_connections 2024-09-10T10:46:27.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:27.559+0000 7f645defa640 1 -- 172.21.15.143:0/3235894602 wait complete. 2024-09-10T10:46:28.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:46:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:28.876 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:28 smithi143 ceph-mon[31057]: pgmap v393: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:28 smithi178 ceph-mon[34813]: pgmap v393: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:28 smithi144 ceph-mon[34230]: pgmap v393: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:29.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.097+0000 7f23a10b8640 1 -- 172.21.15.143:0/273614036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c031880 con 0x7f239c103640 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.097+0000 7f23a431e640 1 -- 172.21.15.143:0/273614036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c103640 msgr2=0x7f239c10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.097+0000 7f23a431e640 1 --2- 172.21.15.143:0/273614036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c103640 0x7f239c10bb30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f238c009900 tx=0x7f238c02a480 comp rx=0 tx=0).stop 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 -- 172.21.15.143:0/273614036 shutdown_connections 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 --2- 172.21.15.143:0/273614036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 0x7f239c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 --2- 172.21.15.143:0/273614036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c103640 0x7f239c10bb30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 --2- 172.21.15.143:0/273614036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c102d20 0x7f239c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 -- 172.21.15.143:0/273614036 >> 172.21.15.143:0/273614036 conn(0x7f239c0fc780 msgr2=0x7f239c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 -- 172.21.15.143:0/273614036 shutdown_connections 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 -- 172.21.15.143:0/273614036 wait complete. 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.098+0000 7f23a431e640 1 Processor -- start 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 -- start start 2024-09-10T10:46:29.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 0x7f239c100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c103640 0x7f239c100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 0x7f239c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f239c111e00 con 0x7f239c102d20 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 0x7f239c100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f239c111c80 con 0x7f239c10c150 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a431e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f239c111f80 con 0x7f239c103640 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 0x7f239c100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42816/0 (socket says 172.21.15.143:42816) 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 -- 172.21.15.143:0/157320578 learned_addr learned my addr 172.21.15.143:0/157320578 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a18b9640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c103640 0x7f239c100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a28bb640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 0x7f239c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 -- 172.21.15.143:0/157320578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c103640 msgr2=0x7f239c100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c103640 0x7f239c100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 -- 172.21.15.143:0/157320578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 msgr2=0x7f239c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.099+0000 7f23a20ba640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 0x7f239c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23a20ba640 1 -- 172.21.15.143:0/157320578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c1a6880 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23a28bb640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 0x7f239c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23a20ba640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 0x7f239c100980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f239800d770 tx=0x7f239800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23937fe640 1 -- 172.21.15.143:0/157320578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398017070 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c1a6b70 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23937fe640 1 -- 172.21.15.143:0/157320578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23980054d0 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23937fe640 1 -- 172.21.15.143:0/157320578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398020060 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.100+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c1a7080 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.103+0000 7f23937fe640 1 -- 172.21.15.143:0/157320578 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2398004a90 con 0x7f239c102d20 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.103+0000 7f23a66f7740 1 Processor -- start 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 -- start start 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53db40 0x5625dc53c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc53ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc549760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23937fe640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23880a2e10 0x7f23880a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5625dc4a22f0 con 0x5625dc547370 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625dc449cf0 con 0x5625dc53db40 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.104+0000 7f23a66f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5625dc49e2b0 con 0x5625dc53c5d0 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc549760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc549760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42830/0 (socket says 172.21.15.143:42830) 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 -- 172.21.15.143:0/2180937750 learned_addr learned my addr 172.21.15.143:0/2180937750 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23937fe640 1 -- 172.21.15.143:0/157320578 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f23980cec80 con 0x7f239c102d20 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a28bb640 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53db40 0x5625dc53c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a28bb640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23880a2e10 0x7f23880a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a20ba640 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc53ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 -- 172.21.15.143:0/2180937750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 msgr2=0x5625dc53ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc53ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 -- 172.21.15.143:0/2180937750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53db40 msgr2=0x5625dc53c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53db40 0x5625dc53c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 -- 172.21.15.143:0/2180937750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625dc42e3a0 con 0x5625dc547370 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a20ba640 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc53ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.105+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc549760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f238c009900 tx=0x7f238c032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae030c39dc5e612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.106+0000 7f2390dea640 1 -- 172.21.15.143:0/2180937750 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c040070 con 0x5625dc547370 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.106+0000 7f2390dea640 1 -- 172.21.15.143:0/2180937750 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f238c004030 con 0x5625dc547370 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.106+0000 7f2390dea640 1 -- 172.21.15.143:0/2180937750 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c035040 con 0x5625dc547370 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.106+0000 7f23a66f7740 1 -- 172.21.15.143:0/2180937750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 msgr2=0x5625dc549760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.106+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc549760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f238c009900 tx=0x7f238c032b00 comp rx=0 tx=0).stop 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.106+0000 7f23a28bb640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23880a2e10 0x7f23880a52d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2394009270 tx=0x7f2394009af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 -- 172.21.15.143:0/2180937750 shutdown_connections 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc549760 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc53ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2180937750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53db40 0x5625dc53c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 -- 172.21.15.143:0/2180937750 >> 172.21.15.143:0/2180937750 conn(0x5625dc505310 msgr2=0x5625dc538c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 -- 172.21.15.143:0/2180937750 shutdown_connections 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 -- 172.21.15.143:0/2180937750 wait complete. 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 -- start start 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.107+0000 7f23a66f7740 1 Processor -- start 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 0x5625dc546d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc543fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc544560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5625dc54a010 con 0x5625dc547370 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625dc45d060 con 0x5625dc53c5d0 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5625dc54a190 con 0x5625dc53db40 2024-09-10T10:46:29.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a66f7740 10 client.0 _ll_register_callbacks cb 0x5625dc1b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 0x5625dc546d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 0x5625dc546d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56098/0 (socket says 172.21.15.143:56098) 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 -- 172.21.15.143:0/2167336850 learned_addr learned my addr 172.21.15.143:0/2167336850 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc543fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a28bb640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc544560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 -- 172.21.15.143:0/2167336850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 msgr2=0x5625dc543fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc543fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 -- 172.21.15.143:0/2167336850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 msgr2=0x5625dc544560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc544560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 -- 172.21.15.143:0/2167336850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625dc2133c0 con 0x5625dc53c5d0 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc543fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a28bb640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc544560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f23a20ba640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 0x5625dc546d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f239809ad20 tx=0x7f239809ac20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61478630b859478c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.108+0000 7f2385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f2385ffb640 1 -- 172.21.15.143:0/2167336850 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398017040 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f2385ffb640 1 -- 172.21.15.143:0/2167336850 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f239809f0d0 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f238c004030 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f2385ffb640 1 -- 172.21.15.143:0/2167336850 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f239809b020 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f23a66f7740 10 client.25259 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625dc55e260 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.109+0000 7f2385ffb640 1 -- 172.21.15.143:0/2167336850 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f238c004030 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f2385ffb640 1 -- 172.21.15.143:0/2167336850 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2398095b90 con 0x5625dc53c5d0 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a66f7740 10 client.25259 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a66f7740 10 client.25259 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a66f7740 20 client.25259 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc55f950 0x5625dc561d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a66f7740 4 client.25259 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625dc4af860 con 0x5625dc55f950 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.110+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc55f950 0x5625dc561d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a18b9640 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc55f950 0x5625dc561d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9f894a555ee223b server_cookie=81417bfedf636331 in_seq=0 out_seq=0 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f2385ffb640 10 client.25259 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f2385ffb640 1 -- 172.21.15.143:0/2167336850 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625dc4af860 con 0x5625dc55f950 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f2385ffb640 10 client.25259 handle_command_reply: tid=0 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 client.25259 shutdown 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 msgr2=0x5625dc546d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 0x5625dc546d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f239809ad20 tx=0x7f239809ac20 comp rx=0 tx=0).stop 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 shutdown_connections 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc547370 0x5625dc544560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc543fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc55f950 0x5625dc561d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 --2- 172.21.15.143:0/2167336850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc53c5d0 0x5625dc546d10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 >> 172.21.15.143:0/2167336850 conn(0x5625dc505310 msgr2=0x5625dc53e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.111+0000 7f2385ffb640 0 client.25259 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:29.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.112+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 shutdown_connections 2024-09-10T10:46:29.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.112+0000 7f23a66f7740 1 -- 172.21.15.143:0/2167336850 wait complete. 2024-09-10T10:46:29.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.112+0000 7f23a66f7740 20 client.25259 trim_cache size 0 max 16384 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 Processor -- start 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 -- start start 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc545f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53db40 0x5625dc53f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 0x5625dc53f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f239c19a490 con 0x5625dc53db40 2024-09-10T10:46:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f239c19bfd0 con 0x5625dc547370 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.125+0000 7f23a66f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f239c19ab60 con 0x5625dc53c5d0 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc545f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc545f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43138/0 (socket says 172.21.15.143:43138) 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 -- 172.21.15.143:0/1106681151 learned_addr learned my addr 172.21.15.143:0/1106681151 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 -- 172.21.15.143:0/1106681151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 msgr2=0x5625dc53f660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 0x5625dc53f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 -- 172.21.15.143:0/1106681151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53db40 msgr2=0x5625dc53f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53db40 0x5625dc53f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.126+0000 7f23a20ba640 1 -- 172.21.15.143:0/1106681151 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625dc549ad0 con 0x5625dc53c5d0 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.127+0000 7f23a20ba640 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc545f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f239809afb0 tx=0x7f2398099530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7858114f5a52732b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.127+0000 7f2387fff640 1 -- 172.21.15.143:0/1106681151 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398017070 con 0x5625dc53c5d0 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.127+0000 7f2387fff640 1 -- 172.21.15.143:0/1106681151 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23980a3c90 con 0x5625dc53c5d0 2024-09-10T10:46:29.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.127+0000 7f2387fff640 1 -- 172.21.15.143:0/1106681151 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398096fb0 con 0x5625dc53c5d0 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.128+0000 7f23a66f7740 1 -- 172.21.15.143:0/1106681151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 msgr2=0x5625dc545f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.128+0000 7f23a66f7740 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc545f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f239809afb0 tx=0x7f2398099530 comp rx=0 tx=0).stop 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.128+0000 7f23a66f7740 1 -- 172.21.15.143:0/1106681151 shutdown_connections 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.128+0000 7f23a66f7740 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 0x5625dc53f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.128+0000 7f23a66f7740 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53db40 0x5625dc53f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.129+0000 7f23a66f7740 1 --2- 172.21.15.143:0/1106681151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53c5d0 0x5625dc545f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.129+0000 7f23a66f7740 1 -- 172.21.15.143:0/1106681151 >> 172.21.15.143:0/1106681151 conn(0x5625dc505310 msgr2=0x5625dc545820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.129+0000 7f23a66f7740 1 -- 172.21.15.143:0/1106681151 shutdown_connections 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.129+0000 7f23a66f7740 1 -- 172.21.15.143:0/1106681151 wait complete. 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.129+0000 7f23a66f7740 1 -- start start 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.129+0000 7f23a66f7740 1 Processor -- start 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 0x5625dc54f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc54f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 0x5625dc541190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f239c19b900 con 0x5625dc53c5d0 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f239c19b230 con 0x5625dc547370 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f239c198520 con 0x5625dc53db40 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a66f7740 10 client.0 _ll_register_callbacks cb 0x7f23980046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 0x5625dc54f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a20ba640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 0x5625dc54f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42864/0 (socket says 172.21.15.143:42864) 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a20ba640 1 -- 172.21.15.143:0/4059814855 learned_addr learned my addr 172.21.15.143:0/4059814855 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.130+0000 7f23a18b9640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc54f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a18b9640 1 -- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 msgr2=0x5625dc541190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a18b9640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 0x5625dc541190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a18b9640 1 -- 172.21.15.143:0/4059814855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 msgr2=0x5625dc54f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a18b9640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 0x5625dc54f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a18b9640 1 -- 172.21.15.143:0/4059814855 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625dc5c6030 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a20ba640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 0x5625dc54f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f23a18b9640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc54f990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f238c02a9a0 tx=0x7f238c02ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d66d988890bf5d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f2385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.131+0000 7f2385ffb640 1 -- 172.21.15.143:0/4059814855 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c040070 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.132+0000 7f2385ffb640 1 -- 172.21.15.143:0/4059814855 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f238c031770 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.132+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23980a3c90 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.132+0000 7f2385ffb640 1 -- 172.21.15.143:0/4059814855 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f238c059050 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.132+0000 7f23a66f7740 10 client.25288 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.132+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625dc5db8d0 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.132+0000 7f2385ffb640 1 -- 172.21.15.143:0/4059814855 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f238c004040 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f2385ffb640 1 -- 172.21.15.143:0/4059814855 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f238c003710 con 0x5625dc53db40 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a66f7740 10 client.25288 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a66f7740 10 client.25288 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a66f7740 20 client.25288 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a66f7740 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc5dbc10 0x5625dc5de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a66f7740 4 client.25288 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f23980a3c90 con 0x5625dc5dbc10 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.133+0000 7f23a20ba640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc5dbc10 0x5625dc5de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a20ba640 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc5dbc10 0x5625dc5de000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdd311a5610ea0d6 server_cookie=96ec0cd9ead1ab5e in_seq=0 out_seq=0 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f2385ffb640 10 client.25288 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f2385ffb640 1 -- 172.21.15.143:0/4059814855 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f23980a3c90 con 0x5625dc5dbc10 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f2385ffb640 10 client.25288 handle_command_reply: tid=0 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 client.25288 shutdown 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 msgr2=0x5625dc54f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc54f990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f238c02a9a0 tx=0x7f238c02ab80 comp rx=0 tx=0).stop 2024-09-10T10:46:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 shutdown_connections 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5625dc5dbc10 0x5625dc5de000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625dc547370 0x5625dc541190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5625dc53db40 0x5625dc54f990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 --2- 172.21.15.143:0/4059814855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5625dc53c5d0 0x5625dc54f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.134+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 >> 172.21.15.143:0/4059814855 conn(0x5625dc505310 msgr2=0x5625dc546040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.135+0000 7f2385ffb640 0 client.25288 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.135+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 shutdown_connections 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.135+0000 7f23a66f7740 1 -- 172.21.15.143:0/4059814855 wait complete. 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.135+0000 7f23a66f7740 20 client.25288 trim_cache size 0 max 16384 2024-09-10T10:46:29.143 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.135+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23880a2e10 msgr2=0x7f23880a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.135+0000 7f23a431e640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23880a2e10 0x7f23880a52d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2394009270 tx=0x7f2394009af0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 msgr2=0x7f239c100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 0x7f239c100980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f239800d770 tx=0x7f239800dc30 comp rx=0 tx=0).stop 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 shutdown_connections 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23880a2e10 0x7f23880a52d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f239c10c150 0x7f239c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c103640 0x7f239c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 --2- 172.21.15.143:0/157320578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f239c102d20 0x7f239c100980 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 >> 172.21.15.143:0/157320578 conn(0x7f239c0fc780 msgr2=0x7f239c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 shutdown_connections 2024-09-10T10:46:29.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:29.136+0000 7f23a431e640 1 -- 172.21.15.143:0/157320578 wait complete. 2024-09-10T10:46:29.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:30 smithi143 ceph-mon[31057]: pgmap v394: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T10:46:30.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:30 smithi143 ceph-mon[31057]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-10T10:46:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:30 smithi178 ceph-mon[34813]: pgmap v394: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T10:46:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:30 smithi178 ceph-mon[34813]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-10T10:46:30.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 -- 172.21.15.143:0/1165132239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec102d40 msgr2=0x7f14ec103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 --2- 172.21.15.143:0/1165132239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec102d40 0x7f14ec103120 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009900 tx=0x7f14dc02a480 comp rx=0 tx=0).stop 2024-09-10T10:46:30.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 -- 172.21.15.143:0/1165132239 shutdown_connections 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 --2- 172.21.15.143:0/1165132239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 0x7f14ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 --2- 172.21.15.143:0/1165132239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 --2- 172.21.15.143:0/1165132239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec102d40 0x7f14ec103120 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 -- 172.21.15.143:0/1165132239 >> 172.21.15.143:0/1165132239 conn(0x7f14ec0fc7a0 msgr2=0x7f14ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.921+0000 7f14f1261640 1 -- 172.21.15.143:0/1165132239 shutdown_connections 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.922+0000 7f14f1261640 1 -- 172.21.15.143:0/1165132239 wait complete. 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.922+0000 7f14f1261640 1 Processor -- start 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.922+0000 7f14f1261640 1 -- start start 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14f1261640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14f1261640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 0x7f14ec10af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14f1261640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec10b470 0x7f14ec10b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14f1261640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec111e70 con 0x7f14ec10b470 2024-09-10T10:46:30.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14f1261640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec111cf0 con 0x7f14ec10c170 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14f1261640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec111ff0 con 0x7f14ec103660 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 0x7f14ec10af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec10b470 0x7f14ec10b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec19f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43162/0 (socket says 172.21.15.143:43162) 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14eaffd640 1 -- 172.21.15.143:0/2825623606 learned_addr learned my addr 172.21.15.143:0/2825623606 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2825623606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 msgr2=0x7f14ec19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec10b470 msgr2=0x7f14ec10b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec10b470 0x7f14ec10b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2825623606 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec19ac50 con 0x7f14ec10c170 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.923+0000 7f14eaffd640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:30.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.924+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 0x7f14ec10af30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14e000e7c0 tx=0x7f14e000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.924+0000 7f14d3fff640 1 -- 172.21.15.143:0/2825623606 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0009530 con 0x7f14ec10c170 2024-09-10T10:46:30.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.924+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec19af40 con 0x7f14ec10c170 2024-09-10T10:46:30.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.924+0000 7f14d3fff640 1 -- 172.21.15.143:0/2825623606 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14e000ccc0 con 0x7f14ec10c170 2024-09-10T10:46:30.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.924+0000 7f14d3fff640 1 -- 172.21.15.143:0/2825623606 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004a00 con 0x7f14ec10c170 2024-09-10T10:46:30.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.924+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec19b430 con 0x7f14ec10c170 2024-09-10T10:46:30.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.927+0000 7f14d3fff640 1 -- 172.21.15.143:0/2825623606 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f14e0004ba0 con 0x7f14ec10c170 2024-09-10T10:46:30.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 Processor -- start 2024-09-10T10:46:30.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 -- start start 2024-09-10T10:46:30.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564bb150d2f0 con 0x564bb15b2370 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564bb14b4cf0 con 0x564bb15a75d0 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14f363a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564bb15092b0 con 0x564bb15a8b40 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42906/0 (socket says 172.21.15.143:42906) 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.928+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2692705049 learned_addr learned my addr 172.21.15.143:0/2692705049 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14eaffd640 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2692705049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 msgr2=0x564bb15a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2692705049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 msgr2=0x564bb15a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14ea7fc640 1 -- 172.21.15.143:0/2692705049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb14993a0 con 0x564bb15b2370 2024-09-10T10:46:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14d3fff640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b4760 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14e00d74b0 tx=0x7f14e00d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d56b6385a75a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.929+0000 7f14d3fff640 1 -- 172.21.15.143:0/2825623606 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f14e00cf2c0 con 0x7f14ec10c170 2024-09-10T10:46:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.930+0000 7f14d17fa640 1 -- 172.21.15.143:0/2692705049 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e001c070 con 0x564bb15b2370 2024-09-10T10:46:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.930+0000 7f14d17fa640 1 -- 172.21.15.143:0/2692705049 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14e0005020 con 0x564bb15b2370 2024-09-10T10:46:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.930+0000 7f14d17fa640 1 -- 172.21.15.143:0/2692705049 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0097a00 con 0x564bb15b2370 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- 172.21.15.143:0/2692705049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 msgr2=0x564bb15b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005fd0 tx=0x7f14e4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b4760 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14e00d74b0 tx=0x7f14e00d2f90 comp rx=0 tx=0).stop 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- 172.21.15.143:0/2692705049 shutdown_connections 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b4760 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- 172.21.15.143:0/2692705049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- 172.21.15.143:0/2692705049 >> 172.21.15.143:0/2692705049 conn(0x564bb1570310 msgr2=0x564bb15a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- 172.21.15.143:0/2692705049 shutdown_connections 2024-09-10T10:46:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- 172.21.15.143:0/2692705049 wait complete. 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- start start 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 Processor -- start 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 0x564bb15af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15b2370 0x564bb15af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564bb15b5070 con 0x564bb15a8b40 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.931+0000 7f14f363a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564bb14c8060 con 0x564bb15a75d0 2024-09-10T10:46:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14f363a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564bb15b51f0 con 0x564bb15b2370 2024-09-10T10:46:30.939 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:30 smithi144 ceph-mon[34230]: pgmap v394: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T10:46:30.939 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:30 smithi144 ceph-mon[34230]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-10T10:46:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14f363a740 10 client.0 _ll_register_callbacks cb 0x564bb121b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 0x564bb15af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 0x564bb15af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42910/0 (socket says 172.21.15.143:42910) 2024-09-10T10:46:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15b2370 0x564bb15af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14ea7fc640 1 -- 172.21.15.143:0/3847770802 learned_addr learned my addr 172.21.15.143:0/3847770802 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 -- 172.21.15.143:0/3847770802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 msgr2=0x564bb15b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 -- 172.21.15.143:0/3847770802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 msgr2=0x564bb15af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 0x564bb15af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 -- 172.21.15.143:0/3847770802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb127e3c0 con 0x564bb15b2370 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eaffd640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14ea7fc640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 0x564bb15af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15b2370 0x564bb15af850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14e4011dd0 tx=0x7f14e4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332acd8f3190501e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.932+0000 7f14ceff5640 1 -- 172.21.15.143:0/3847770802 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4004b20 con 0x564bb15b2370 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14ceff5640 1 -- 172.21.15.143:0/3847770802 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f14e4004cc0 con 0x564bb15b2370 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564bb151a860 con 0x564bb15b2370 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14ceff5640 1 -- 172.21.15.143:0/3847770802 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14e400f810 con 0x564bb15b2370 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 10 client.25300 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bb15c0c20 con 0x564bb15b2370 2024-09-10T10:46:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14ceff5640 1 -- 172.21.15.143:0/3847770802 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564bb151a860 con 0x564bb15b2370 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14ceff5640 1 -- 172.21.15.143:0/3847770802 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f14e4026070 con 0x564bb15b2370 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 10 client.25300 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 10 client.25300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 20 client.25300 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb15c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 4 client.25300 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.933+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb1514d70 con 0x564bb15c2080 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.934+0000 7f14eaffd640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb15c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.934+0000 7f14ceff5640 10 client.25300 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.934+0000 7f14eaffd640 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb15c4490 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747fad24828379cb server_cookie=9027ae9d79d57170 in_seq=0 out_seq=0 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14ceff5640 1 -- 172.21.15.143:0/3847770802 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bb1514d70 con 0x564bb15c2080 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14ceff5640 10 client.25300 handle_command_reply: tid=0 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 client.25300 shutdown 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15b2370 msgr2=0x564bb15af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15b2370 0x564bb15af850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14e4011dd0 tx=0x7f14e4004ab0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 shutdown_connections 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb15c4490 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14ceff5640 0 client.25300 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15b2370 0x564bb15af850 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15a8b40 0x564bb15af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 --2- 172.21.15.143:0/3847770802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.935+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 >> 172.21.15.143:0/3847770802 conn(0x564bb1570310 msgr2=0x564bb15a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.936+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 shutdown_connections 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.936+0000 7f14f363a740 1 -- 172.21.15.143:0/3847770802 wait complete. 2024-09-10T10:46:30.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.936+0000 7f14f363a740 20 client.25300 trim_cache size 0 max 16384 2024-09-10T10:46:30.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 Processor -- start 2024-09-10T10:46:30.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 -- start start 2024-09-10T10:46:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 0x564bb15ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a8b40 0x564bb15c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1d7790 con 0x564bb15b2370 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1d92d0 con 0x564bb15a8b40 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.949+0000 7f14f363a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1d7e60 con 0x564bb15a75d0 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 0x564bb15ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a8b40 0x564bb15c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 0x564bb15ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43188/0 (socket says 172.21.15.143:43188) 2024-09-10T10:46:30.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 -- 172.21.15.143:0/571644813 learned_addr learned my addr 172.21.15.143:0/571644813 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 -- 172.21.15.143:0/571644813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a8b40 msgr2=0x564bb15c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a8b40 0x564bb15c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 -- 172.21.15.143:0/571644813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 msgr2=0x564bb15ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 -- 172.21.15.143:0/571644813 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb15aa120 con 0x564bb15a75d0 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:30.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.950+0000 7f14eaffd640 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 0x564bb15ae0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14dc031440 tx=0x7f14dc005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99ea76d378cee47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.951+0000 7f14d0ff9640 1 -- 172.21.15.143:0/571644813 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc04a030 con 0x564bb15a75d0 2024-09-10T10:46:30.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.951+0000 7f14d0ff9640 1 -- 172.21.15.143:0/571644813 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14dc003d50 con 0x564bb15a75d0 2024-09-10T10:46:30.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.951+0000 7f14d0ff9640 1 -- 172.21.15.143:0/571644813 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc000e60 con 0x564bb15a75d0 2024-09-10T10:46:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 -- 172.21.15.143:0/571644813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 msgr2=0x564bb15ae0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 0x564bb15ae0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f14dc031440 tx=0x7f14dc005a40 comp rx=0 tx=0).stop 2024-09-10T10:46:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 -- 172.21.15.143:0/571644813 shutdown_connections 2024-09-10T10:46:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a8b40 0x564bb15c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 --2- 172.21.15.143:0/571644813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a75d0 0x564bb15ae0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 -- 172.21.15.143:0/571644813 >> 172.21.15.143:0/571644813 conn(0x564bb1570310 msgr2=0x564bb15ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 -- 172.21.15.143:0/571644813 shutdown_connections 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 -- 172.21.15.143:0/571644813 wait complete. 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.952+0000 7f14f363a740 1 -- start start 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.953+0000 7f14f363a740 1 Processor -- start 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.953+0000 7f14f363a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.953+0000 7f14f363a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.953+0000 7f14f363a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.953+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.953+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15ba980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56166/0 (socket says 172.21.15.143:56166) 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14ea7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43194/0 (socket says 172.21.15.143:43194) 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42916/0 (socket says 172.21.15.143:42916) 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eaffd640 1 -- 172.21.15.143:0/3380896446 learned_addr learned my addr 172.21.15.143:0/3380896446 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1d8c00 con 0x564bb15b2370 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1d8530 con 0x564bb15a75d0 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec06b070 con 0x564bb15a8b40 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 -- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 msgr2=0x564bb15b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 -- 172.21.15.143:0/3380896446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 msgr2=0x564bb15ba980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15ba980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eb7fe640 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb15b6800 con 0x564bb15b2370 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.954+0000 7f14eaffd640 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15ba980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14f363a740 10 client.0 _ll_register_callbacks cb 0x7f14e4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:30.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc003d50 con 0x564bb15b2370 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14eb7fe640 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b6240 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009f10 tx=0x7f14e4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b1317b318c412b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14ceff5640 10 client.15804 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14ceff5640 1 -- 172.21.15.143:0/3380896446 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011030 con 0x564bb15b2370 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14ceff5640 1 -- 172.21.15.143:0/3380896446 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f14e40111d0 con 0x564bb15b2370 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14ceff5640 1 -- 172.21.15.143:0/3380896446 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14e4004b20 con 0x564bb15b2370 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14f363a740 10 client.15804 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.955+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({fsmap=29}) -- 0x564bb162c810 con 0x564bb15b2370 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14ceff5640 1 -- 172.21.15.143:0/3380896446 <== mon.0 v2:172.21.15.143:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f14e4026040 con 0x564bb15b2370 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14f363a740 10 client.15804 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14f363a740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14f363a740 20 client.15804 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14f363a740 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb162f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14f363a740 4 client.15804 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:30.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bb1491170 con 0x564bb15c2080 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.956+0000 7f14eaffd640 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb162f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14eaffd640 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb162f000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be524aaba0026a22 server_cookie=7cfad0d4795ee35e in_seq=0 out_seq=0 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14ceff5640 10 client.15804 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14ceff5640 1 -- 172.21.15.143:0/3380896446 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bb1491170 con 0x564bb15c2080 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14ceff5640 10 client.15804 handle_command_reply: tid=0 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14f363a740 1 client.15804 shutdown 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 msgr2=0x564bb15b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14f363a740 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b6240 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009f10 tx=0x7f14e4003d80 comp rx=0 tx=0).stop 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 shutdown_connections 2024-09-10T10:46:30.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14f363a740 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564bb15b2370 0x564bb15b6240 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.957+0000 7f14f363a740 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x564bb15c2080 0x564bb162f000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f363a740 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564bb15a8b40 0x564bb15b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f363a740 1 --2- 172.21.15.143:0/3380896446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564bb15a75d0 0x564bb15ba980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 >> 172.21.15.143:0/3380896446 conn(0x564bb1570310 msgr2=0x564bb15ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14ceff5640 0 client.15804 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 shutdown_connections 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f363a740 1 -- 172.21.15.143:0/3380896446 wait complete. 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f363a740 20 client.15804 trim_cache size 0 max 16384 2024-09-10T10:46:30.965 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:30.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 msgr2=0x7f14d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f1261640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005fd0 tx=0x7f14e4009500 comp rx=0 tx=0).stop 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 msgr2=0x7f14ec10af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.958+0000 7f14f1261640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 0x7f14ec10af30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14e000e7c0 tx=0x7f14e000ec80 comp rx=0 tx=0).stop 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 shutdown_connections 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f14d40a2b80 0x7f14d40a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec10b470 0x7f14ec10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14ec10c170 0x7f14ec10af30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 --2- 172.21.15.143:0/2825623606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14ec103660 0x7f14ec19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 >> 172.21.15.143:0/2825623606 conn(0x7f14ec0fc7a0 msgr2=0x7f14ec101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 shutdown_connections 2024-09-10T10:46:30.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:30.959+0000 7f14f1261640 1 -- 172.21.15.143:0/2825623606 wait complete. 2024-09-10T10:46:32.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:46:32.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:46:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:32 smithi178 ceph-mon[34813]: pgmap v395: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T10:46:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:32 smithi144 ceph-mon[34230]: pgmap v395: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T10:46:33.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:32 smithi143 ceph-mon[31057]: pgmap v395: 129 pgs: 129 active+clean; 665 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T10:46:33.120 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 -- 172.21.15.143:0/681687048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 msgr2=0x7f9bd010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 --2- 172.21.15.143:0/681687048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 0x7f9bd010ea90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009ca0 tx=0x7f9bc802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 -- 172.21.15.143:0/681687048 shutdown_connections 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 --2- 172.21.15.143:0/681687048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 0x7f9bd010ea90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 --2- 172.21.15.143:0/681687048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0102790 0x7f9bd01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 --2- 172.21.15.143:0/681687048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bd0101de0 0x7f9bd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 -- 172.21.15.143:0/681687048 >> 172.21.15.143:0/681687048 conn(0x7f9bd00fd8c0 msgr2=0x7f9bd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 -- 172.21.15.143:0/681687048 shutdown_connections 2024-09-10T10:46:33.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.637+0000 7f9bd5657640 1 -- 172.21.15.143:0/681687048 wait complete. 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.638+0000 7f9bd5657640 1 Processor -- start 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.638+0000 7f9bd5657640 1 -- start start 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.638+0000 7f9bd5657640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 0x7f9bd0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bd5657640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bd0102790 0x7f9bd01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bd5657640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 0x7f9bd019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bd5657640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0111e60 con 0x7f9bd0106f10 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bd5657640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0111ce0 con 0x7f9bd0102790 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bd5657640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0111fe0 con 0x7f9bd0101de0 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 0x7f9bd0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 0x7f9bd0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43216/0 (socket says 172.21.15.143:43216) 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 -- 172.21.15.143:0/1588650811 learned_addr learned my addr 172.21.15.143:0/1588650811 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bd0102790 0x7f9bd01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 -- 172.21.15.143:0/1588650811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bd0102790 msgr2=0x7f9bd01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bd0102790 0x7f9bd01988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 -- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 msgr2=0x7f9bd019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 0x7f9bd019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 -- 172.21.15.143:0/1588650811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd019d330 con 0x7f9bd0101de0 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.639+0000 7f9bceffd640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 0x7f9bd0198380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000b4a0 tx=0x7f9bc000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.640+0000 7f9bb7fff640 1 -- 172.21.15.143:0/1588650811 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0014070 con 0x7f9bd0101de0 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.640+0000 7f9bb7fff640 1 -- 172.21.15.143:0/1588650811 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc0010ac0 con 0x7f9bd0101de0 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.640+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd019d620 con 0x7f9bd0101de0 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.640+0000 7f9bb7fff640 1 -- 172.21.15.143:0/1588650811 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0017b60 con 0x7f9bd0101de0 2024-09-10T10:46:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.640+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd0074420 con 0x7f9bd0101de0 2024-09-10T10:46:33.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.643+0000 7f9bb7fff640 1 -- 172.21.15.143:0/1588650811 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bd0101de0 2024-09-10T10:46:33.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.643+0000 7f9bd7a30740 1 Processor -- start 2024-09-10T10:46:33.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 -- start start 2024-09-10T10:46:33.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9b8d40 0x561cba9bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561cba913280 con 0x561cba9adb60 2024-09-10T10:46:33.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bb7fff640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9bb80a2c40 0x7f9bb80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43226/0 (socket says 172.21.15.143:43226) 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bceffd640 1 -- 172.21.15.143:0/2177347695 learned_addr learned my addr 172.21.15.143:0/2177347695 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cba8bab10 con 0x561cba9b8d40 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.644+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2177347695 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561cba90f240 con 0x561cba9ae480 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bb7fff640 1 -- 172.21.15.143:0/1588650811 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9bc00ce660 con 0x7f9bd0101de0 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bcf7fe640 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9b8d40 0x561cba9bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bce7fc640 1 -- 172.21.15.143:0/2177347695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 msgr2=0x561cba9b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bce7fc640 1 -- 172.21.15.143:0/2177347695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 msgr2=0x561cba9adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.645+0000 7f9bce7fc640 1 -- 172.21.15.143:0/2177347695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cba8c71f0 con 0x561cba9b8d40 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.646+0000 7f9bcf7fe640 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.646+0000 7f9bcf7fe640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9bb80a2c40 0x7f9bb80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.646+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9b8d40 0x561cba9bb130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40044f0 tx=0x7f9bc4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3d8f6d2a30dcc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.646+0000 7f9bcf7fe640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9bb80a2c40 0x7f9bb80a5100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005e30 tx=0x7f9bc80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.646+0000 7f9bb57fa640 1 -- 172.21.15.143:0/2177347695 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4009c40 con 0x561cba9b8d40 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.646+0000 7f9bb57fa640 1 -- 172.21.15.143:0/2177347695 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc4009de0 con 0x561cba9b8d40 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.647+0000 7f9bb57fa640 1 -- 172.21.15.143:0/2177347695 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4004dd0 con 0x561cba9b8d40 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.647+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2177347695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9b8d40 msgr2=0x561cba9bb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.647+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9b8d40 0x561cba9bb130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40044f0 tx=0x7f9bc4005f90 comp rx=0 tx=0).stop 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2177347695 shutdown_connections 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9b8d40 0x561cba9bb130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2177347695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2177347695 >> 172.21.15.143:0/2177347695 conn(0x561cba9aa0f0 msgr2=0x561cba9aa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2177347695 shutdown_connections 2024-09-10T10:46:33.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2177347695 wait complete. 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 -- start start 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.648+0000 7f9bd7a30740 1 Processor -- start 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 0x561cba9b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9ae480 0x561cba9b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 0x561cba9c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 0x561cba9b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 0x561cba9b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43242/0 (socket says 172.21.15.143:43242) 2024-09-10T10:46:33.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bcf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 0x561cba9c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bcf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 0x561cba9c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42948/0 (socket says 172.21.15.143:42948) 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561cba9bb910 con 0x561cba9b6810 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cba8cdf10 con 0x561cba9ae480 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561cba9bba90 con 0x561cba9adb60 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bd7a30740 10 client.0 _ll_register_callbacks cb 0x561cba896a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.649+0000 7f9bceffd640 1 -- 172.21.15.143:0/2265669809 learned_addr learned my addr 172.21.15.143:0/2265669809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9ae480 0x561cba9b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bceffd640 1 -- 172.21.15.143:0/2265669809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9ae480 msgr2=0x561cba9b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bceffd640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9ae480 0x561cba9b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bceffd640 1 -- 172.21.15.143:0/2265669809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 msgr2=0x561cba9c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bceffd640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 0x561cba9c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bceffd640 1 -- 172.21.15.143:0/2265669809 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cba6846a0 con 0x561cba9adb60 2024-09-10T10:46:33.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bcf7fe640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 0x561cba9c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bceffd640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 0x561cba9b5d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0017630 tx=0x7f9bc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1d78b9dd4c34b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/2265669809 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc001e070 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/2265669809 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9bc00ce3c0 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.650+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc4009de0 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/2265669809 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9bc00b8050 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bd7a30740 10 client.25318 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cba9c7b10 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/2265669809 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc009ab60 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/2265669809 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9bc00d0b40 con 0x561cba9adb60 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bd7a30740 10 client.25318 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.651+0000 7f9bd7a30740 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:33.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.652+0000 7f9bd7a30740 20 client.25318 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:33.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.652+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9c93f0 0x561cba9cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.652+0000 7f9bd7a30740 4 client.25318 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:33.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.652+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9bc4009de0 con 0x561cba9c93f0 2024-09-10T10:46:33.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.652+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9c93f0 0x561cba9cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.653+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9c93f0 0x561cba9cb860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69852df3270455b9 server_cookie=7d0bee38c05bcffd in_seq=0 out_seq=0 2024-09-10T10:46:33.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.653+0000 7f9bb2ff5640 10 client.25318 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.653+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/2265669809 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9bc4009de0 con 0x561cba9c93f0 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.653+0000 7f9bb2ff5640 10 client.25318 handle_command_reply: tid=0 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.653+0000 7f9bd7a30740 1 client.25318 shutdown 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 msgr2=0x561cba9b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 0x561cba9b5d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0017630 tx=0x7f9bc00046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 shutdown_connections 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9b6810 0x561cba9c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9ae480 0x561cba9b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9c93f0 0x561cba9cb860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/2265669809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9adb60 0x561cba9b5d90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bb2ff5640 0 client.25318 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 >> 172.21.15.143:0/2265669809 conn(0x561cba9aa0f0 msgr2=0x561cba9afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 shutdown_connections 2024-09-10T10:46:33.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.654+0000 7f9bd7a30740 1 -- 172.21.15.143:0/2265669809 wait complete. 2024-09-10T10:46:33.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.655+0000 7f9bd7a30740 20 client.25318 trim_cache size 0 max 16384 2024-09-10T10:46:33.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.688+0000 7f9bd7a30740 1 Processor -- start 2024-09-10T10:46:33.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 -- start start 2024-09-10T10:46:33.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ac9b0 con 0x561cba9adb60 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ae4f0 con 0x561cba9aa0f0 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ad080 con 0x561cba9ae480 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bce7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bce7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42962/0 (socket says 172.21.15.143:42962) 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bce7fc640 1 -- 172.21.15.143:0/989885115 learned_addr learned my addr 172.21.15.143:0/989885115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bceffd640 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.689+0000 7f9bcf7fe640 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bce7fc640 1 -- 172.21.15.143:0/989885115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 msgr2=0x561cba9d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bce7fc640 1 -- 172.21.15.143:0/989885115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 msgr2=0x561cba9c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bce7fc640 1 -- 172.21.15.143:0/989885115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cba9b8700 con 0x561cba9adb60 2024-09-10T10:46:33.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bceffd640 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:33.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9d1140 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x561cba9ab470 tx=0x7f9bc4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0abf72af1390b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bb4ff9640 1 -- 172.21.15.143:0/989885115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40045e0 con 0x561cba9adb60 2024-09-10T10:46:33.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bb4ff9640 1 -- 172.21.15.143:0/989885115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc4004780 con 0x561cba9adb60 2024-09-10T10:46:33.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.690+0000 7f9bb4ff9640 1 -- 172.21.15.143:0/989885115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc402e900 con 0x561cba9adb60 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 -- 172.21.15.143:0/989885115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 msgr2=0x561cba9d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9d1140 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x561cba9ab470 tx=0x7f9bc4004a50 comp rx=0 tx=0).stop 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 -- 172.21.15.143:0/989885115 shutdown_connections 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cba9d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9d1140 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/989885115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.691+0000 7f9bd7a30740 1 -- 172.21.15.143:0/989885115 >> 172.21.15.143:0/989885115 conn(0x561cba9b6430 msgr2=0x561cba9b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 -- 172.21.15.143:0/989885115 shutdown_connections 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 -- 172.21.15.143:0/989885115 wait complete. 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 -- start start 2024-09-10T10:46:33.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 Processor -- start 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cbaa3b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ade20 con 0x561cba9adb60 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ad750 con 0x561cba9aa0f0 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01aa990 con 0x561cba9ae480 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.692+0000 7f9bd7a30740 10 client.0 _ll_register_callbacks cb 0x7f9bc00092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bce7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bce7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9bd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42970/0 (socket says 172.21.15.143:42970) 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bce7fc640 1 -- 172.21.15.143:0/3601690886 learned_addr learned my addr 172.21.15.143:0/3601690886 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 -- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 msgr2=0x561cbaa3b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cbaa3b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 -- 172.21.15.143:0/3601690886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 msgr2=0x561cba9bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 -- 172.21.15.143:0/3601690886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbaa3be60 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bceffd640 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9bcdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0099fb0 tx=0x7f9bc009b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738dffe91101f320 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.693+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/3601690886 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc001e070 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc4004780 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/3601690886 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9bc00ce3c0 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/3601690886 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9bc009a6a0 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 10 client.25286 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cbaa3cf60 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/3601690886 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc4004780 con 0x561cba9aa0f0 2024-09-10T10:46:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/3601690886 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9bc00968b0 con 0x561cba9aa0f0 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 10 client.25286 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 10 client.25286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 20 client.25286 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9b6810 0x561cbaa3f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 4 client.25286 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.694+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561cba9207f0 con 0x561cba9b6810 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.695+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9b6810 0x561cbaa3f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.695+0000 7f9bb2ff5640 10 client.25286 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.695+0000 7f9bce7fc640 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9b6810 0x561cbaa3f4a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131b7e4d17943e82 server_cookie=be31cd01f65018dc in_seq=0 out_seq=0 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.696+0000 7f9bb2ff5640 1 -- 172.21.15.143:0/3601690886 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561cba9207f0 con 0x561cba9b6810 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.696+0000 7f9bb2ff5640 10 client.25286 handle_command_reply: tid=0 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.696+0000 7f9bd7a30740 1 client.25286 shutdown 2024-09-10T10:46:33.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 msgr2=0x561cba9bcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9bcdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0099fb0 tx=0x7f9bc009b260 comp rx=0 tx=0).stop 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 shutdown_connections 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561cba9b6810 0x561cbaa3f4a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "f443c3ca-401c-4cb0-824e-6d9b031ed925", 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:46:33.703 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cba9ae480 0x561cbaa3b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cba9adb60 0x561cba9bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 --2- 172.21.15.143:0/3601690886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cba9aa0f0 0x561cba9bcdf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 >> 172.21.15.143:0/3601690886 conn(0x561cba9b6430 msgr2=0x561cba9af600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bb2ff5640 0 client.25286 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:33.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 shutdown_connections 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 1 -- 172.21.15.143:0/3601690886 wait complete. 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.697+0000 7f9bd7a30740 20 client.25286 trim_cache size 0 max 16384 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9bb80a2c40 msgr2=0x7f9bb80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9bb80a2c40 0x7f9bb80a5100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005e30 tx=0x7f9bc80095c0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 msgr2=0x7f9bd0198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 0x7f9bd0198380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000b4a0 tx=0x7f9bc000b960 comp rx=0 tx=0).stop 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 shutdown_connections 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9bb80a2c40 0x7f9bb80a5100 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd0106f10 0x7f9bd019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bd0102790 0x7f9bd01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 --2- 172.21.15.143:0/1588650811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd0101de0 0x7f9bd0198380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 >> 172.21.15.143:0/1588650811 conn(0x7f9bd00fd8c0 msgr2=0x7f9bd00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 shutdown_connections 2024-09-10T10:46:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:33.698+0000 7f9bd5657640 1 -- 172.21.15.143:0/1588650811 wait complete. 2024-09-10T10:46:34.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:46:34.510 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:46:34.587 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:34.761 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:34 smithi143 ceph-mon[31057]: pgmap v396: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:46:34.761 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:34 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:46:34.761 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:34 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:34.761 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:34 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:46:34.761 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:34 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:34.761 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:34 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:46:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:34 smithi178 ceph-mon[34813]: pgmap v396: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:46:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:34 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:46:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:34 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:34 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:46:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:34 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:34 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:46:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:34 smithi144 ceph-mon[34230]: pgmap v396: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:46:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:34 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:46:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:34 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:34 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:46:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:34 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:34 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:46:35.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 -- 172.21.15.143:0/194624151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c1082a0 msgr2=0x7f3c4c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 --2- 172.21.15.143:0/194624151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c10ec40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009cd0 tx=0x7f3c4402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 -- 172.21.15.143:0/194624151 shutdown_connections 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 --2- 172.21.15.143:0/194624151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c10ec40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 --2- 172.21.15.143:0/194624151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4c1036e0 0x7f3c4c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 --2- 172.21.15.143:0/194624151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c102d30 0x7f3c4c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 -- 172.21.15.143:0/194624151 >> 172.21.15.143:0/194624151 conn(0x7f3c4c0fc7d0 msgr2=0x7f3c4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 -- 172.21.15.143:0/194624151 shutdown_connections 2024-09-10T10:46:35.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.162+0000 7f3c532a7640 1 -- 172.21.15.143:0/194624151 wait complete. 2024-09-10T10:46:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 Processor -- start 2024-09-10T10:46:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 -- start start 2024-09-10T10:46:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 0x7f3c4c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4c1036e0 0x7f3c4c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111f80 con 0x7f3c4c102d30 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111e00 con 0x7f3c4c1082a0 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.163+0000 7f3c532a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c112100 con 0x7f3c4c1036e0 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51043640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 0x7f3c4c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51043640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 0x7f3c4c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42994/0 (socket says 172.21.15.143:42994) 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51043640 1 -- 172.21.15.143:0/421818193 learned_addr learned my addr 172.21.15.143:0/421818193 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c50842640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4c1036e0 0x7f3c4c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 -- 172.21.15.143:0/421818193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4c1036e0 msgr2=0x7f3c4c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4c1036e0 0x7f3c4c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 -- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 msgr2=0x7f3c4c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 0x7f3c4c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 -- 172.21.15.143:0/421818193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c19d510 con 0x7f3c4c1082a0 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51043640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 0x7f3c4c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.164+0000 7f3c51844640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c19ce30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009ca0 tx=0x7f3c44003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.165+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004040 con 0x7f3c4c1082a0 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.165+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c44004690 con 0x7f3c4c1082a0 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.165+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c19d7a0 con 0x7f3c4c1082a0 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.165+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1008c0 con 0x7f3c4c1082a0 2024-09-10T10:46:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.165+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44030de0 con 0x7f3c4c1082a0 2024-09-10T10:46:35.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.166+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c30005290 con 0x7f3c4c1082a0 2024-09-10T10:46:35.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.169+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3c440305a0 con 0x7f3c4c1082a0 2024-09-10T10:46:35.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.169+0000 7f3c427fc640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c380a2c00 0x7f3c380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.170+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3c4402e030 con 0x7f3c4c1082a0 2024-09-10T10:46:35.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.170+0000 7f3c51043640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c380a2c00 0x7f3c380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.170+0000 7f3c51043640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c380a2c00 0x7f3c380a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c480115a0 tx=0x7f3c48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.172+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c440bd190 con 0x7f3c4c1082a0 2024-09-10T10:46:35.216 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.583+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c30005580 con 0x7f3c4c1082a0 2024-09-10T10:46:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.584+0000 7f3c427fc640 1 -- 172.21.15.143:0/421818193 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f3c440bca90 con 0x7f3c4c1082a0 2024-09-10T10:46:35.586 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:46:35.586 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:46:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.587+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c380a2c00 msgr2=0x7f3c380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c380a2c00 0x7f3c380a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c480115a0 tx=0x7f3c48005f90 comp rx=0 tx=0).stop 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c1082a0 msgr2=0x7f3c4c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c19ce30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009ca0 tx=0x7f3c44003970 comp rx=0 tx=0).stop 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 shutdown_connections 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3c380a2c00 0x7f3c380a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c4c1082a0 0x7f3c4c19ce30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4c1036e0 0x7f3c4c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 --2- 172.21.15.143:0/421818193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c4c102d30 0x7f3c4c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 >> 172.21.15.143:0/421818193 conn(0x7f3c4c0fc7d0 msgr2=0x7f3c4c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 shutdown_connections 2024-09-10T10:46:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.588+0000 7f3c532a7640 1 -- 172.21.15.143:0/421818193 wait complete. 2024-09-10T10:46:35.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.760+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/100846964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc781082a0 msgr2=0x7fbc7810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.760+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/100846964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc781082a0 0x7fbc7810ec40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009cd0 tx=0x7fbc7002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.760+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/100846964 shutdown_connections 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.760+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/100846964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc781082a0 0x7fbc7810ec40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.760+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/100846964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 0x7fbc78107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.760+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/100846964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc78102d30 0x7fbc78103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.761+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/100846964 >> 172.21.15.143:0/100846964 conn(0x7fbc780fc7d0 msgr2=0x7fbc780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.761+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/100846964 shutdown_connections 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.761+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/100846964 wait complete. 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.761+0000 7fbc7e6fb640 1 Processor -- start 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 -- start start 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 0x7fbc78198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 0x7fbc781989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 0x7fbc78198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc781082a0 0x7fbc7819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 0x7fbc78198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43016/0 (socket says 172.21.15.143:43016) 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc77fff640 1 -- 172.21.15.143:0/3472007216 learned_addr learned my addr 172.21.15.143:0/3472007216 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111f70 con 0x7fbc78102d30 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111df0 con 0x7fbc781036e0 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.762+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781120f0 con 0x7fbc781082a0 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc781082a0 0x7fbc7819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc77fff640 1 -- 172.21.15.143:0/3472007216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc781082a0 msgr2=0x7fbc7819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc777fe640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 0x7fbc781989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc77fff640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc781082a0 0x7fbc7819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc77fff640 1 -- 172.21.15.143:0/3472007216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 msgr2=0x7fbc781989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc77fff640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 0x7fbc781989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc77fff640 1 -- 172.21.15.143:0/3472007216 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7819d440 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc781082a0 0x7fbc7819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc777fe640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 0x7fbc781989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc77fff640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 0x7fbc78198490 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800e9f0 tx=0x7fbc6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc757fa640 1 -- 172.21.15.143:0/3472007216 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800cd80 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7819d730 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781a5010 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc757fa640 1 -- 172.21.15.143:0/3472007216 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc680054d0 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.763+0000 7fbc757fa640 1 -- 172.21.15.143:0/3472007216 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68010490 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.766+0000 7fbc757fa640 1 -- 172.21.15.143:0/3472007216 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbc68005020 con 0x7fbc78102d30 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc757fa640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc600a2b80 0x7fbc600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 Processor -- start 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc757fa640 1 -- 172.21.15.143:0/3472007216 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbc680cf880 con 0x7fbc78102d30 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 -- start start 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc777fe640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc600a2b80 0x7fbc600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x56197708d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x56197708dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 0x5619770a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561976ff32e0 con 0x56197708d880 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561976f9ace0 con 0x56197708ea20 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.767+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561976fef2a0 con 0x5619770a0d20 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 0x5619770a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 0x5619770a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43312/0 (socket says 172.21.15.143:43312) 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x56197708d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 -- 172.21.15.143:0/3536374801 learned_addr learned my addr 172.21.15.143:0/3536374801 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc777fe640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc600a2b80 0x7fbc600a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c005fd0 tx=0x7fbc6c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc777fe640 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x56197708dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 -- 172.21.15.143:0/3536374801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 msgr2=0x56197708d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x56197708d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 -- 172.21.15.143:0/3536374801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 msgr2=0x56197708dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x56197708dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 -- 172.21.15.143:0/3536374801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561976f7f440 con 0x5619770a0d20 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc777fe640 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x56197708dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.768+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 0x5619770a3110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005f90 tx=0x7fbc70009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ce271275971213 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.769+0000 7fbc5e7fc640 1 -- 172.21.15.143:0/3536374801 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70031850 con 0x5619770a0d20 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.769+0000 7fbc5e7fc640 1 -- 172.21.15.143:0/3536374801 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc70036040 con 0x5619770a0d20 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.769+0000 7fbc5e7fc640 1 -- 172.21.15.143:0/3536374801 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70004500 con 0x5619770a0d20 2024-09-10T10:46:35.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 -- 172.21.15.143:0/3536374801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 msgr2=0x5619770a3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 0x5619770a3110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005f90 tx=0x7fbc70009a20 comp rx=0 tx=0).stop 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 -- 172.21.15.143:0/3536374801 shutdown_connections 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619770a0d20 0x5619770a3110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x56197708dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/3536374801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x56197708d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 -- 172.21.15.143:0/3536374801 >> 172.21.15.143:0/3536374801 conn(0x561977089a30 msgr2=0x561977089e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 -- 172.21.15.143:0/3536374801 shutdown_connections 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.770+0000 7fbc80ad4740 1 -- 172.21.15.143:0/3536374801 wait complete. 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.771+0000 7fbc80ad4740 1 -- start start 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.771+0000 7fbc80ad4740 1 Processor -- start 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708d880 0x561977093c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x561977090eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x561977091640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5619770a39f0 con 0x5619770a0d20 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561976fae050 con 0x56197708d880 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708d880 0x561977093c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708d880 0x561977093c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56272/0 (socket says 172.21.15.143:56272) 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc77fff640 1 -- 172.21.15.143:0/1695594754 learned_addr learned my addr 172.21.15.143:0/1695594754 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x561977091640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5619770a3b70 con 0x56197708ea20 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc777fe640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x561977090eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.772+0000 7fbc80ad4740 10 client.0 _ll_register_callbacks cb 0x561976fee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc7cc98640 1 -- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 msgr2=0x561977090eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x561977090eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc7cc98640 1 -- 172.21.15.143:0/1695594754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708d880 msgr2=0x561977093c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708d880 0x561977093c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc7cc98640 1 -- 172.21.15.143:0/1695594754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561976d643c0 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc777fe640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x561977090eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x561977091640 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005e30 tx=0x7fbc70031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9651e03b596ba96f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc577fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc577fe640 1 -- 172.21.15.143:0/1695594754 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003d80 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc577fe640 1 -- 172.21.15.143:0/1695594754 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbc70004280 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc70036040 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc577fe640 1 -- 172.21.15.143:0/1695594754 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc70057050 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc80ad4740 10 client.15837 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.773+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619770af4f0 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc577fe640 1 -- 172.21.15.143:0/1695594754 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc70036040 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc577fe640 1 -- 172.21.15.143:0/1695594754 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbc70030070 con 0x5619770a0d20 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc80ad4740 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:35.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc80ad4740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc80ad4740 20 client.15837 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619770b0840 0x5619770b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc80ad4740 4 client.15837 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561977000850 con 0x5619770b0840 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.774+0000 7fbc77fff640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619770b0840 0x5619770b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.775+0000 7fbc77fff640 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619770b0840 0x5619770b2c30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63cbc7a92014e076 server_cookie=ccf717ddb0a6a142 in_seq=0 out_seq=0 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.775+0000 7fbc577fe640 10 client.15837 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.775+0000 7fbc577fe640 1 -- 172.21.15.143:0/1695594754 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561977000850 con 0x5619770b0840 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.775+0000 7fbc577fe640 10 client.15837 handle_command_reply: tid=0 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 client.15837 shutdown 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 msgr2=0x561977091640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x561977091640 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc70005e30 tx=0x7fbc70031c50 comp rx=0 tx=0).stop 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 shutdown_connections 2024-09-10T10:46:35.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x561977091640 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619770b0840 0x5619770b2c30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x561977090eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1695594754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708d880 0x561977093c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 >> 172.21.15.143:0/1695594754 conn(0x561977089a30 msgr2=0x56197708f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc577fe640 0 client.15837 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 shutdown_connections 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1695594754 wait complete. 2024-09-10T10:46:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.776+0000 7fbc80ad4740 20 client.15837 trim_cache size 0 max 16384 2024-09-10T10:46:35.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.793+0000 7fbc80ad4740 1 Processor -- start 2024-09-10T10:46:35.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.793+0000 7fbc80ad4740 1 -- start start 2024-09-10T10:46:35.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.793+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x561977090770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x56197709cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619770a0d20 0x5619770a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ac990 con 0x56197708d880 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ae4d0 con 0x5619770a0d20 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ad060 con 0x56197708ea20 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x561977090770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x561977090770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43040/0 (socket says 172.21.15.143:43040) 2024-09-10T10:46:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 -- 172.21.15.143:0/1878503113 learned_addr learned my addr 172.21.15.143:0/1878503113 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:35.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc777fe640 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x56197709cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619770a0d20 0x5619770a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 -- 172.21.15.143:0/1878503113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 msgr2=0x56197709cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x56197709cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 -- 172.21.15.143:0/1878503113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619770a0d20 msgr2=0x5619770a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619770a0d20 0x5619770a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 -- 172.21.15.143:0/1878503113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197708e290 con 0x56197708d880 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619770a0d20 0x5619770a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.794+0000 7fbc77fff640 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x561977090770 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c6a0 tx=0x7fbc680980a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451af4c7c50e25a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.795+0000 7fbc5dffb640 1 -- 172.21.15.143:0/1878503113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6809bd80 con 0x56197708d880 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.795+0000 7fbc5dffb640 1 -- 172.21.15.143:0/1878503113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc680a0130 con 0x56197708d880 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.795+0000 7fbc5dffb640 1 -- 172.21.15.143:0/1878503113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68016e80 con 0x56197708d880 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.795+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1878503113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 msgr2=0x561977090770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.795+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x561977090770 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c6a0 tx=0x7fbc680980a0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1878503113 shutdown_connections 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619770a0d20 0x5619770a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708ea20 0x56197709cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/1878503113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56197708d880 0x561977090770 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1878503113 >> 172.21.15.143:0/1878503113 conn(0x561977089a30 msgr2=0x561977089e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1878503113 shutdown_connections 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 -- 172.21.15.143:0/1878503113 wait complete. 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 -- start start 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.796+0000 7fbc80ad4740 1 Processor -- start 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708d880 0x5619770a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x5619770a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x5619770a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ade00 con 0x5619770a0d20 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781ad730 con 0x56197708ea20 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781aa970 con 0x56197708d880 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc80ad4740 10 client.0 _ll_register_callbacks cb 0x7fbc7003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x5619770a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x5619770a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56296/0 (socket says 172.21.15.143:56296) 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 -- 172.21.15.143:0/782249700 learned_addr learned my addr 172.21.15.143:0/782249700 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc77fff640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708d880 0x5619770a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x5619770a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 -- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708d880 msgr2=0x5619770a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708d880 0x5619770a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 -- 172.21.15.143:0/782249700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 msgr2=0x5619770a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x5619770a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 -- 172.21.15.143:0/782249700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619770a29f0 con 0x56197708ea20 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc7cc98640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x5619770a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc777fe640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x5619770a1f70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc7806a370 tx=0x7fbc6c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e984852c51b810 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.797+0000 7fbc577fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:46:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc577fe640 1 -- 172.21.15.143:0/782249700 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c010920 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc680a0130 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc577fe640 1 -- 172.21.15.143:0/782249700 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbc6c010ac0 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc577fe640 1 -- 172.21.15.143:0/782249700 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c010cc0 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 10 client.25304 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619770a7b80 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc577fe640 1 -- 172.21.15.143:0/782249700 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc680a0130 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc577fe640 1 -- 172.21.15.143:0/782249700 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbc6c00f040 con 0x56197708ea20 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 10 client.25304 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 10 client.25304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 20 client.25304 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56197711b6d0 0x56197711daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 4 client.25304 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:46:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.798+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561976ffad60 con 0x56197711b6d0 2024-09-10T10:46:35.806 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:35.806 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:46:35.806 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:46:35.806 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.799+0000 7fbc77fff640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56197711b6d0 0x56197711daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.799+0000 7fbc577fe640 10 client.25304 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.799+0000 7fbc77fff640 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56197711b6d0 0x56197711daf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875d6d54eff66ff7 server_cookie=df485cee127c5df7 in_seq=0 out_seq=0 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.799+0000 7fbc577fe640 1 -- 172.21.15.143:0/782249700 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561976ffad60 con 0x56197711b6d0 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.799+0000 7fbc577fe640 10 client.25304 handle_command_reply: tid=0 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.799+0000 7fbc80ad4740 1 client.25304 shutdown 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 msgr2=0x5619770a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x5619770a1f70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc7806a370 tx=0x7fbc6c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 shutdown_connections 2024-09-10T10:46:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619770a0d20 0x5619770a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56197708ea20 0x5619770a1f70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56197711b6d0 0x56197711daf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc577fe640 0 client.25304 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 --2- 172.21.15.143:0/782249700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56197708d880 0x5619770a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 >> 172.21.15.143:0/782249700 conn(0x561977089a30 msgr2=0x5619770a02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 shutdown_connections 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 1 -- 172.21.15.143:0/782249700 wait complete. 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc80ad4740 20 client.25304 trim_cache size 0 max 16384 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc600a2b80 msgr2=0x7fbc600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.800+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc600a2b80 0x7fbc600a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c005fd0 tx=0x7fbc6c0094d0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 msgr2=0x7fbc78198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 0x7fbc78198490 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800e9f0 tx=0x7fbc6800edf0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 shutdown_connections 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbc600a2b80 0x7fbc600a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc781082a0 0x7fbc7819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781036e0 0x7fbc781989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 --2- 172.21.15.143:0/3472007216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc78102d30 0x7fbc78198490 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 >> 172.21.15.143:0/3472007216 conn(0x7fbc780fc7d0 msgr2=0x7fbc78105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 shutdown_connections 2024-09-10T10:46:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:35.801+0000 7fbc7e6fb640 1 -- 172.21.15.143:0/3472007216 wait complete. 2024-09-10T10:46:35.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:35 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/421818193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:35.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:35 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/421818193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:36.041 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:35 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/421818193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:36.771 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:36 smithi143 ceph-mon[31057]: pgmap v397: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T10:46:36.801 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:46:36.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:46:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:36 smithi178 ceph-mon[34813]: pgmap v397: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T10:46:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:36 smithi144 ceph-mon[34230]: pgmap v397: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T10:46:37.352 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 9c3f80de --release --await' 2024-09-10T10:46:37.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 9c3f80de --release --await 2024-09-10T10:46:37.432 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:37.830 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:37 smithi143 ceph-mon[31057]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-10T10:46:37.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:37 smithi178 ceph-mon[34813]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-10T10:46:37.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:37 smithi144 ceph-mon[34230]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 -- 172.21.15.143:0/4053323493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8108290 msgr2=0x7f86f810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 --2- 172.21.15.143:0/4053323493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8108290 0x7f86f810ec30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009cd0 tx=0x7f86f002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 -- 172.21.15.143:0/4053323493 shutdown_connections 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 --2- 172.21.15.143:0/4053323493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8108290 0x7f86f810ec30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 --2- 172.21.15.143:0/4053323493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 0x7f86f8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 --2- 172.21.15.143:0/4053323493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86f8102d90 0x7f86f8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 -- 172.21.15.143:0/4053323493 >> 172.21.15.143:0/4053323493 conn(0x7f86f80fc7d0 msgr2=0x7f86f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 -- 172.21.15.143:0/4053323493 shutdown_connections 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.074+0000 7f86fea1d640 1 -- 172.21.15.143:0/4053323493 wait complete. 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.075+0000 7f86fea1d640 1 Processor -- start 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.075+0000 7f86fea1d640 1 -- start start 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fea1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 0x7f86f819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fea1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 0x7f86f819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 0x7f86f819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fea1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86f8108290 0x7f86f810dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 0x7f86f819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56134/0 (socket says 172.21.15.143:56134) 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f7fff640 1 -- 172.21.15.143:0/1336049528 learned_addr learned my addr 172.21.15.143:0/1336049528 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8111f40 con 0x7f86f8102d90 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8111dc0 con 0x7f86f8103740 2024-09-10T10:46:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81120c0 con 0x7f86f8108290 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f77fe640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 0x7f86f819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86fcfba640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86f8108290 0x7f86f810dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f77fe640 1 -- 172.21.15.143:0/1336049528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86f8108290 msgr2=0x7f86f810dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f77fe640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86f8108290 0x7f86f810dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f77fe640 1 -- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 msgr2=0x7f86f819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f77fe640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 0x7f86f819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f77fe640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f810e8b0 con 0x7f86f8103740 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.076+0000 7f86f7fff640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 0x7f86f819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.077+0000 7f86f77fe640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 0x7f86f819f6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f86f80697d0 tx=0x7f86ec00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.077+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec017070 con 0x7f86f8103740 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.077+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86ec0099d0 con 0x7f86f8103740 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.077+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f810eba0 con 0x7f86f8103740 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.078+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0047a0 con 0x7f86f8103740 2024-09-10T10:46:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.078+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f819ace0 con 0x7f86f8103740 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.079+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f806a7f0 con 0x7f86f8103740 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.080+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f86ec009b80 con 0x7f86f8103740 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.081+0000 7f86f57fa640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86e00a2ad0 0x7f86e00a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.081+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f86ec0ceba0 con 0x7f86f8103740 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.082+0000 7f86f7fff640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86e00a2ad0 0x7f86e00a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.082+0000 7f86f7fff640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86e00a2ad0 0x7f86e00a4f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009ce0 tx=0x7f86e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.085+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86ec020030 con 0x7f86f8103740 2024-09-10T10:46:38.132 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:38.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.359+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86f8103170 con 0x7f86f8103740 2024-09-10T10:46:38.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.360+0000 7f86f57fa640 1 -- 172.21.15.143:0/1336049528 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f86ec09b7a0 con 0x7f86f8103740 2024-09-10T10:46:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:46:38.362 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:46:38.362 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86e00a2ad0 msgr2=0x7f86e00a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86e00a2ad0 0x7f86e00a4f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009ce0 tx=0x7f86e8005f90 comp rx=0 tx=0).stop 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 msgr2=0x7f86f819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 0x7f86f819f6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f86f80697d0 tx=0x7f86ec00dbb0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 shutdown_connections 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86e00a2ad0 0x7f86e00a4f90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86f8108290 0x7f86f810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f8103740 0x7f86f819f6c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 --2- 172.21.15.143:0/1336049528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f8102d90 0x7f86f819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.363+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 >> 172.21.15.143:0/1336049528 conn(0x7f86f80fc7d0 msgr2=0x7f86f8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.364+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 shutdown_connections 2024-09-10T10:46:38.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:38.364+0000 7f86fea1d640 1 -- 172.21.15.143:0/1336049528 wait complete. 2024-09-10T10:46:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:38 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1336049528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:38.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:38 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1336049528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:38.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:38 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1336049528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:39.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:46:39.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.326+0000 7f240b552640 1 -- 172.21.15.143:0/2029790565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404100a80 msgr2=0x7f2404100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.326+0000 7f240b552640 1 --2- 172.21.15.143:0/2029790565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404100a80 0x7f2404100ee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009930 tx=0x7f23f402a420 comp rx=0 tx=0).stop 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 -- 172.21.15.143:0/2029790565 shutdown_connections 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 --2- 172.21.15.143:0/2029790565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404101420 0x7f240410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 --2- 172.21.15.143:0/2029790565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404100a80 0x7f2404100ee0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 --2- 172.21.15.143:0/2029790565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404106a80 0x7f2404106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 -- 172.21.15.143:0/2029790565 >> 172.21.15.143:0/2029790565 conn(0x7f24040fc7a0 msgr2=0x7f24040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 -- 172.21.15.143:0/2029790565 shutdown_connections 2024-09-10T10:46:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.327+0000 7f240b552640 1 -- 172.21.15.143:0/2029790565 wait complete. 2024-09-10T10:46:39.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.328+0000 7f240b552640 1 Processor -- start 2024-09-10T10:46:39.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.328+0000 7f240b552640 1 -- start start 2024-09-10T10:46:39.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.329+0000 7f240b552640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404100a80 0x7f2404198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.329+0000 7f240b552640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 0x7f2404198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.329+0000 7f240b552640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 0x7f240419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.329+0000 7f24092ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404100a80 0x7f2404198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.330+0000 7f24092ee640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404100a80 0x7f2404198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42068/0 (socket says 172.21.15.143:42068) 2024-09-10T10:46:39.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.330+0000 7f2408aed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 0x7f2404198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.330+0000 7f2408aed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 0x7f2404198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:48308/0 (socket says 172.21.15.143:48308) 2024-09-10T10:46:39.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.330+0000 7f2409aef640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 0x7f240419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.330+0000 7f2409aef640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 0x7f240419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56152/0 (socket says 172.21.15.143:56152) 2024-09-10T10:46:39.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.330+0000 7f24092ee640 1 -- 172.21.15.143:0/3314866486 learned_addr learned my addr 172.21.15.143:0/3314866486 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:39.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.331+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2404111ea0 con 0x7f2404106a80 2024-09-10T10:46:39.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.331+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2404111d20 con 0x7f2404101420 2024-09-10T10:46:39.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.331+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2404112020 con 0x7f2404100a80 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.331+0000 7f2408aed640 1 -- 172.21.15.143:0/3314866486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404100a80 msgr2=0x7f2404198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.331+0000 7f2408aed640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404100a80 0x7f2404198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.331+0000 7f2408aed640 1 -- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 msgr2=0x7f240419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.332+0000 7f2408aed640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 0x7f240419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.332+0000 7f2408aed640 1 -- 172.21.15.143:0/3314866486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240419d3d0 con 0x7f2404101420 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.332+0000 7f2409aef640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 0x7f240419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.332+0000 7f2408aed640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 0x7f2404198960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009850 tx=0x7f23f4003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.332+0000 7f23fa7fc640 1 -- 172.21.15.143:0/3314866486 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f403f070 con 0x7f2404101420 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.333+0000 7f23fa7fc640 1 -- 172.21.15.143:0/3314866486 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23f4031b50 con 0x7f2404101420 2024-09-10T10:46:39.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.333+0000 7f23fa7fc640 1 -- 172.21.15.143:0/3314866486 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4035030 con 0x7f2404101420 2024-09-10T10:46:39.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.333+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240419d660 con 0x7f2404101420 2024-09-10T10:46:39.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.333+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240419da00 con 0x7f2404101420 2024-09-10T10:46:39.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.336+0000 7f23fa7fc640 1 -- 172.21.15.143:0/3314866486 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f23f4042070 con 0x7f2404101420 2024-09-10T10:46:39.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 Processor -- start 2024-09-10T10:46:39.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 -- start start 2024-09-10T10:46:39.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 0x55a19d8f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f9190 0x55a19d903510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d905e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a19d855620 con 0x55a19d8f8870 2024-09-10T10:46:39.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a19d7fce50 con 0x55a19d8f9190 2024-09-10T10:46:39.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.337+0000 7f240d92b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a19d8515e0 con 0x55a19d903a50 2024-09-10T10:46:39.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 0x55a19d8f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 0x55a19d8f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56160/0 (socket says 172.21.15.143:56160) 2024-09-10T10:46:39.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 -- 172.21.15.143:0/3280631288 learned_addr learned my addr 172.21.15.143:0/3280631288 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:39.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f23fa7fc640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23f00a2c40 0x7f23f00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f23fa7fc640 1 -- 172.21.15.143:0/3314866486 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f23f40ef9b0 con 0x7f2404101420 2024-09-10T10:46:39.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 -- 172.21.15.143:0/3280631288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 msgr2=0x55a19d905e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:39.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d905e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 -- 172.21.15.143:0/3280631288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f9190 msgr2=0x55a19d903510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:39.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f9190 0x55a19d903510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.338+0000 7f2409aef640 1 -- 172.21.15.143:0/3280631288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19d7edd40 con 0x55a19d8f8870 2024-09-10T10:46:39.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.339+0000 7f2409aef640 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 0x55a19d8f8c50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00aad0 tx=0x7f23fc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c7527598d1b264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.339+0000 7f2409aef640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23f00a2c40 0x7f23f00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.340+0000 7f23ef7fe640 1 -- 172.21.15.143:0/3280631288 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc018020 con 0x55a19d8f8870 2024-09-10T10:46:39.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.340+0000 7f23ef7fe640 1 -- 172.21.15.143:0/3280631288 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23fc01d8d0 con 0x55a19d8f8870 2024-09-10T10:46:39.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.340+0000 7f23ef7fe640 1 -- 172.21.15.143:0/3280631288 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc014ad0 con 0x55a19d8f8870 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f2409aef640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23f00a2c40 0x7f23f00a5100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23fc019000 tx=0x7f23fc01c780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 -- 172.21.15.143:0/3280631288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 msgr2=0x55a19d8f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 0x55a19d8f8c50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00aad0 tx=0x7f23fc009650 comp rx=0 tx=0).stop 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 -- 172.21.15.143:0/3280631288 shutdown_connections 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d905e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f9190 0x55a19d903510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 --2- 172.21.15.143:0/3280631288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f8870 0x55a19d8f8c50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 -- 172.21.15.143:0/3280631288 >> 172.21.15.143:0/3280631288 conn(0x55a19d8f4e00 msgr2=0x55a19d8f51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 -- 172.21.15.143:0/3280631288 shutdown_connections 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 -- 172.21.15.143:0/3280631288 wait complete. 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 -- start start 2024-09-10T10:46:39.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.341+0000 7f240d92b740 1 Processor -- start 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d900930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 0x55a19d900e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d9013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a19d906670 con 0x55a19d8f9190 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a19d7ddf10 con 0x55a19d8f8870 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a19d9067f0 con 0x55a19d903a50 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.343+0000 7f240d92b740 10 client.0 _ll_register_callbacks cb 0x55a19d801db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 0x55a19d900e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 0x55a19d900e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56174/0 (socket says 172.21.15.143:56174) 2024-09-10T10:46:39.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 -- 172.21.15.143:0/3901860051 learned_addr learned my addr 172.21.15.143:0/3901860051 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 -- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 msgr2=0x55a19d9013b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d9013b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 -- 172.21.15.143:0/3901860051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 msgr2=0x55a19d900930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d900930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 -- 172.21.15.143:0/3901860051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19d564580 con 0x55a19d8f9190 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f2408aed640 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 0x55a19d900e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f23f402aa10 tx=0x7f23f40bba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc5b57c4707da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.344+0000 7f23ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f23ecff9640 1 -- 172.21.15.143:0/3901860051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4049050 con 0x55a19d8f9190 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f23ecff9640 1 -- 172.21.15.143:0/3901860051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f23f40b8570 con 0x55a19d8f9190 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc01d8d0 con 0x55a19d8f9190 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f23ecff9640 1 -- 172.21.15.143:0/3901860051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23f40bbf00 con 0x55a19d8f9190 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f240d92b740 10 client.15858 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a19d62e960 con 0x55a19d8f9190 2024-09-10T10:46:39.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.345+0000 7f23ecff9640 1 -- 172.21.15.143:0/3901860051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23fc01d8d0 con 0x55a19d8f9190 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f23ecff9640 1 -- 172.21.15.143:0/3901860051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f23f40c46c0 con 0x55a19d8f9190 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f240d92b740 10 client.15858 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f240d92b740 10 client.15858 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f240d92b740 20 client.15858 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f240d92b740 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d8eedd0 0x55a19d8f1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f240d92b740 4 client.15858 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19d862b90 con 0x55a19d8eedd0 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f24092ee640 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d8eedd0 0x55a19d8f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f23ecff9640 10 client.15858 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:39.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.346+0000 7f24092ee640 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d8eedd0 0x55a19d8f1270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d206375be699d1 server_cookie=8d919113695346e5 in_seq=0 out_seq=0 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f23ecff9640 1 -- 172.21.15.143:0/3901860051 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a19d862b90 con 0x55a19d8eedd0 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f23ecff9640 10 client.15858 handle_command_reply: tid=0 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 client.15858 shutdown 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 msgr2=0x55a19d900e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 0x55a19d900e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f23f402aa10 tx=0x7f23f40bba40 comp rx=0 tx=0).stop 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 shutdown_connections 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d8eedd0 0x55a19d8f1270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d9013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f9190 0x55a19d900e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 --2- 172.21.15.143:0/3901860051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d900930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 >> 172.21.15.143:0/3901860051 conn(0x55a19d8f4e00 msgr2=0x55a19d8fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.347+0000 7f23ecff9640 0 client.15858 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.348+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 shutdown_connections 2024-09-10T10:46:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.348+0000 7f240d92b740 1 -- 172.21.15.143:0/3901860051 wait complete. 2024-09-10T10:46:39.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.348+0000 7f240d92b740 20 client.15858 trim_cache size 0 max 16384 2024-09-10T10:46:39.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.381+0000 7f240d92b740 1 Processor -- start 2024-09-10T10:46:39.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 -- start start 2024-09-10T10:46:39.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d926880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d926dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f43c0 0x55a19d8fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ac9c0 con 0x55a19d8f43c0 2024-09-10T10:46:39.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ae500 con 0x55a19d8f8870 2024-09-10T10:46:39.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f2408aed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d926dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f2408aed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d926dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42080/0 (socket says 172.21.15.143:42080) 2024-09-10T10:46:39.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f2408aed640 1 -- 172.21.15.143:0/4237763682 learned_addr learned my addr 172.21.15.143:0/4237763682 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:39.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f24092ee640 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d926880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f240d92b740 1 -- 172.21.15.143:0/4237763682 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ad090 con 0x55a19d903a50 2024-09-10T10:46:39.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.382+0000 7f2409aef640 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f43c0 0x55a19d8fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.383+0000 7f2409aef640 1 -- 172.21.15.143:0/4237763682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 msgr2=0x55a19d926dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:39.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.383+0000 7f2409aef640 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d926dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.383+0000 7f2409aef640 1 -- 172.21.15.143:0/4237763682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 msgr2=0x55a19d926880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:39.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.383+0000 7f2409aef640 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d926880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.383+0000 7f2409aef640 1 -- 172.21.15.143:0/4237763682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19d8f4c30 con 0x55a19d8f43c0 2024-09-10T10:46:39.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.383+0000 7f2409aef640 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f43c0 0x55a19d8fdb90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f23fc014fd0 tx=0x7f23fc024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7441e05553bdf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.384+0000 7f23eeffd640 1 -- 172.21.15.143:0/4237763682 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc036070 con 0x55a19d8f43c0 2024-09-10T10:46:39.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.384+0000 7f23eeffd640 1 -- 172.21.15.143:0/4237763682 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23fc01ce00 con 0x55a19d8f43c0 2024-09-10T10:46:39.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.384+0000 7f23eeffd640 1 -- 172.21.15.143:0/4237763682 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc018070 con 0x55a19d8f43c0 2024-09-10T10:46:39.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.385+0000 7f240d92b740 1 -- 172.21.15.143:0/4237763682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f43c0 msgr2=0x55a19d8fdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.385+0000 7f240d92b740 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f43c0 0x55a19d8fdb90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f23fc014fd0 tx=0x7f23fc024000 comp rx=0 tx=0).stop 2024-09-10T10:46:39.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 -- 172.21.15.143:0/4237763682 shutdown_connections 2024-09-10T10:46:39.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d8f43c0 0x55a19d8fdb90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d903a50 0x55a19d926dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 --2- 172.21.15.143:0/4237763682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f8870 0x55a19d926880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 -- 172.21.15.143:0/4237763682 >> 172.21.15.143:0/4237763682 conn(0x55a19d8f4e00 msgr2=0x55a19d9029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 -- 172.21.15.143:0/4237763682 shutdown_connections 2024-09-10T10:46:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.386+0000 7f240d92b740 1 -- 172.21.15.143:0/4237763682 wait complete. 2024-09-10T10:46:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.387+0000 7f240d92b740 1 -- start start 2024-09-10T10:46:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.387+0000 7f240d92b740 1 Processor -- start 2024-09-10T10:46:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f240d92b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f43c0 0x55a19d907fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f240d92b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d8f8870 0x55a19d908520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f240d92b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d903a50 0x55a19d980810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f24092ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f43c0 0x55a19d907fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f24092ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f43c0 0x55a19d907fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:48330/0 (socket says 172.21.15.143:48330) 2024-09-10T10:46:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f24092ee640 1 -- 172.21.15.143:0/2270329099 learned_addr learned my addr 172.21.15.143:0/2270329099 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ade30 con 0x55a19d903a50 2024-09-10T10:46:39.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24041ad760 con 0x55a19d8f43c0 2024-09-10T10:46:39.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f2408aed640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d8f8870 0x55a19d908520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24041aa9a0 con 0x55a19d8f8870 2024-09-10T10:46:39.396 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:39.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f240d92b740 10 client.0 _ll_register_callbacks cb 0x7f23f40b81a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:39.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.388+0000 7f2409aef640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d903a50 0x55a19d980810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2408aed640 1 -- 172.21.15.143:0/2270329099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f43c0 msgr2=0x55a19d907fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:39.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2408aed640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f43c0 0x55a19d907fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2408aed640 1 -- 172.21.15.143:0/2270329099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d903a50 msgr2=0x55a19d980810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2408aed640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d903a50 0x55a19d980810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2408aed640 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19d980e00 con 0x55a19d8f8870 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2409aef640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d903a50 0x55a19d980810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f2408aed640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d8f8870 0x55a19d908520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23f40bbe90 tx=0x7f23f4005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1093eba07c8a643 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.389+0000 7f23ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f23ecff9640 1 -- 172.21.15.143:0/2270329099 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4049070 con 0x55a19d8f8870 2024-09-10T10:46:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc01ce00 con 0x55a19d8f8870 2024-09-10T10:46:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f23ecff9640 1 -- 172.21.15.143:0/2270329099 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f23f40b8570 con 0x55a19d8f8870 2024-09-10T10:46:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f23ecff9640 1 -- 172.21.15.143:0/2270329099 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23f40c4630 con 0x55a19d8f8870 2024-09-10T10:46:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f240d92b740 10 client.25351 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a19d988180 con 0x55a19d8f8870 2024-09-10T10:46:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.390+0000 7f23ecff9640 1 -- 172.21.15.143:0/2270329099 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23fc01ce00 con 0x55a19d8f8870 2024-09-10T10:46:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f23ecff9640 1 -- 172.21.15.143:0/2270329099 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f23f40b7ce0 con 0x55a19d8f8870 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f240d92b740 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f240d92b740 10 client.25351 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f240d92b740 20 client.25351 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f240d92b740 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d9886e0 0x55a19d98ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f240d92b740 4 client.25351 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9c3f80de", "release": true, "await": true}] 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9c3f80de", "release": true, "await": true}) -- 0x55a19d85d0a0 con 0x55a19d9886e0 2024-09-10T10:46:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.391+0000 7f24092ee640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d9886e0 0x55a19d98ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.392+0000 7f24092ee640 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d9886e0 0x55a19d98ab00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9072074f1ab5e98b server_cookie=1048c299709d1f3a in_seq=0 out_seq=0 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.392+0000 7f23ecff9640 10 client.25351 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.394+0000 7f23ecff9640 1 -- 172.21.15.143:0/2270329099 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a19d85d0a0 con 0x55a19d9886e0 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.394+0000 7f23ecff9640 10 client.25351 handle_command_reply: tid=0 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.394+0000 7f240d92b740 1 client.25351 shutdown 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d8f8870 msgr2=0x55a19d908520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d8f8870 0x55a19d908520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23f40bbe90 tx=0x7f23f4005be0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 shutdown_connections 2024-09-10T10:46:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a19d903a50 0x55a19d980810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a19d9886e0 0x55a19d98ab00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a19d8f8870 0x55a19d908520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 --2- 172.21.15.143:0/2270329099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a19d8f43c0 0x55a19d907fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 >> 172.21.15.143:0/2270329099 conn(0x55a19d8f4e00 msgr2=0x55a19d8fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f23ecff9640 0 client.25351 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 shutdown_connections 2024-09-10T10:46:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 1 -- 172.21.15.143:0/2270329099 wait complete. 2024-09-10T10:46:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.395+0000 7f240d92b740 20 client.25351 trim_cache size 0 max 16384 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 29, 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 4, 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "9c3f80de": { 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 4, 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:46:39.409 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 150.0, 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:39.410 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:39.411 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:39.411 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:39.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.396+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23f00a2c40 msgr2=0x7f23f00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.396+0000 7f240b552640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23f00a2c40 0x7f23f00a5100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23fc019000 tx=0x7f23fc01c780 comp rx=0 tx=0).stop 2024-09-10T10:46:39.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.396+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 msgr2=0x7f2404198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.396+0000 7f240b552640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 0x7f2404198960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009850 tx=0x7f23f4003c00 comp rx=0 tx=0).stop 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 shutdown_connections 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f23f00a2c40 0x7f23f00a5100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2404106a80 0x7f240419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2404101420 0x7f2404198960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 --2- 172.21.15.143:0/3314866486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2404100a80 0x7f2404198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 >> 172.21.15.143:0/3314866486 conn(0x7f24040fc7a0 msgr2=0x7f24040fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 shutdown_connections 2024-09-10T10:46:39.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:39.397+0000 7f240b552640 1 -- 172.21.15.143:0/3314866486 wait complete. 2024-09-10T10:46:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:39 smithi178 ceph-mon[34813]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-10T10:46:39.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:39 smithi144 ceph-mon[34230]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-10T10:46:39.988 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:39 smithi143 ceph-mon[31057]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-10T10:46:40.418 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9c3f80de', total seconds elapsed: 36.3 2024-09-10T10:46:40.418 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T10:46:40.419 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:46:40.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.687+0000 7f7e554c9640 1 -- 172.21.15.143:0/428401753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50102ef0 msgr2=0x7f7e5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.687+0000 7f7e554c9640 1 --2- 172.21.15.143:0/428401753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50102ef0 0x7f7e5010da40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009ca0 tx=0x7f7e4802a950 comp rx=0 tx=0).stop 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.687+0000 7f7e554c9640 1 -- 172.21.15.143:0/428401753 shutdown_connections 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.687+0000 7f7e554c9640 1 --2- 172.21.15.143:0/428401753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50102ef0 0x7f7e5010da40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.688+0000 7f7e554c9640 1 --2- 172.21.15.143:0/428401753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.688+0000 7f7e554c9640 1 --2- 172.21.15.143:0/428401753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50100e20 0x7f7e50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.688+0000 7f7e554c9640 1 -- 172.21.15.143:0/428401753 >> 172.21.15.143:0/428401753 conn(0x7f7e500fc7d0 msgr2=0x7f7e500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.688+0000 7f7e554c9640 1 -- 172.21.15.143:0/428401753 shutdown_connections 2024-09-10T10:46:40.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.688+0000 7f7e554c9640 1 -- 172.21.15.143:0/428401753 wait complete. 2024-09-10T10:46:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.689+0000 7f7e554c9640 1 Processor -- start 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.689+0000 7f7e554c9640 1 -- start start 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e554c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 0x7f7e50196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e554c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e50196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 0x7f7e50196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e554c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50102ef0 0x7f7e5019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e554c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5010fd70 con 0x7f7e50100e20 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e4e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e50196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e4e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e50196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:48334/0 (socket says 172.21.15.143:48334) 2024-09-10T10:46:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 0x7f7e50196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56208/0 (socket says 172.21.15.143:56208) 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2203334411 learned_addr learned my addr 172.21.15.143:0/2203334411 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5010fbf0 con 0x7f7e50102570 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5010fef0 con 0x7f7e50102ef0 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.690+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50102ef0 0x7f7e5019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2203334411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50102ef0 msgr2=0x7f7e5019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50102ef0 0x7f7e5019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 msgr2=0x7f7e50196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 0x7f7e50196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2203334411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5019b100 con 0x7f7e50102570 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50102ef0 0x7f7e5019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.691+0000 7f7e4effd640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 0x7f7e50196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.692+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e50196690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009a60 tx=0x7f7e44009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.692+0000 7f7e37fff640 1 -- 172.21.15.143:0/2203334411 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44007d30 con 0x7f7e50102570 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.692+0000 7f7e37fff640 1 -- 172.21.15.143:0/2203334411 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e4401c070 con 0x7f7e50102570 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.692+0000 7f7e37fff640 1 -- 172.21.15.143:0/2203334411 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44016640 con 0x7f7e50102570 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.692+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5019b3f0 con 0x7f7e50102570 2024-09-10T10:46:40.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.692+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501a2cd0 con 0x7f7e50102570 2024-09-10T10:46:40.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.696+0000 7f7e37fff640 1 -- 172.21.15.143:0/2203334411 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7e44017a30 con 0x7f7e50102570 2024-09-10T10:46:40.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.696+0000 7f7e578a2740 1 Processor -- start 2024-09-10T10:46:40.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 -- start start 2024-09-10T10:46:40.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e37fff640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7e380a2c80 0x7f7e380a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x5634187118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e37fff640 1 -- 172.21.15.143:0/2203334411 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7e440ced90 con 0x7f7e50102570 2024-09-10T10:46:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x563418712220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56341866f2f0 con 0x56341871cb80 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563418616cf0 con 0x563418711de0 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.697+0000 7f7e578a2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56341866b2b0 con 0x563418713350 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7e380a2c80 0x7f7e380a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x5634187118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x5634187118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42128/0 (socket says 172.21.15.143:42128) 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4effd640 1 -- 172.21.15.143:0/349454751 learned_addr learned my addr 172.21.15.143:0/349454751 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x563418712220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/349454751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 msgr2=0x5634187118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x5634187118a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/349454751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 msgr2=0x56341871ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.698+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/349454751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634185fb3a0 con 0x563418711de0 2024-09-10T10:46:40.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.699+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x563418712220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e440cc4d0 tx=0x7f7e4409b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf07e83ae43e68f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.699+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7e380a2c80 0x7f7e380a5140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e48006010 tx=0x7f7e48009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.699+0000 7f7e357fa640 1 -- 172.21.15.143:0/349454751 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4409f5e0 con 0x563418711de0 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.699+0000 7f7e357fa640 1 -- 172.21.15.143:0/349454751 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e440a3ea0 con 0x563418711de0 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.699+0000 7f7e357fa640 1 -- 172.21.15.143:0/349454751 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e440bb2d0 con 0x563418711de0 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.700+0000 7f7e578a2740 1 -- 172.21.15.143:0/349454751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 msgr2=0x563418712220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.700+0000 7f7e578a2740 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x563418712220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e440cc4d0 tx=0x7f7e4409b930 comp rx=0 tx=0).stop 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.700+0000 7f7e578a2740 1 -- 172.21.15.143:0/349454751 shutdown_connections 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x563418712220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 --2- 172.21.15.143:0/349454751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x5634187118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 -- 172.21.15.143:0/349454751 >> 172.21.15.143:0/349454751 conn(0x5634186d2310 msgr2=0x56341870e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 -- 172.21.15.143:0/349454751 shutdown_connections 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 -- 172.21.15.143:0/349454751 wait complete. 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 -- start start 2024-09-10T10:46:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.701+0000 7f7e578a2740 1 Processor -- start 2024-09-10T10:46:40.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x56341871c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x563418719900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e578a2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56341871f820 con 0x56341871cb80 2024-09-10T10:46:40.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e4f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e4f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56226/0 (socket says 172.21.15.143:56226) 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e4f7fe640 1 -- 172.21.15.143:0/1786891419 learned_addr learned my addr 172.21.15.143:0/1786891419 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56341862a060 con 0x563418711de0 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.702+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56341871f9a0 con 0x563418713350 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e578a2740 10 client.0 _ll_register_callbacks cb 0x56341837d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4f7fe640 1 -- 172.21.15.143:0/1786891419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 msgr2=0x563418719900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4effd640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x56341871c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x563418719900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4f7fe640 1 -- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 msgr2=0x56341871c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x56341871c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4f7fe640 1 -- 172.21.15.143:0/1786891419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634183e03c0 con 0x56341871cb80 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4effd640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x56341871c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871a090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7e480098b0 tx=0x7f7e480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcaad9eff6f4d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.703+0000 7f7e32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:40.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.705+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1786891419 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48055060 con 0x56341871cb80 2024-09-10T10:46:40.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.705+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1786891419 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7e48042030 con 0x56341871cb80 2024-09-10T10:46:40.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.705+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e440a3ea0 con 0x56341871cb80 2024-09-10T10:46:40.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.705+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1786891419 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e48034040 con 0x56341871cb80 2024-09-10T10:46:40.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.705+0000 7f7e578a2740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:40.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.705+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56341872b390 con 0x56341871cb80 2024-09-10T10:46:40.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.706+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1786891419 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e480047c0 con 0x56341871cb80 2024-09-10T10:46:40.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.706+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1786891419 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7e48050ba0 con 0x56341871cb80 2024-09-10T10:46:40.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.706+0000 7f7e578a2740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:40.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.707+0000 7f7e578a2740 10 client.15873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:46:40.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.707+0000 7f7e578a2740 20 client.15873 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:40.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.707+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56341872c850 0x56341872ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.707+0000 7f7e578a2740 4 client.15873 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:40.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.708+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7e440a3ea0 con 0x56341872c850 2024-09-10T10:46:40.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.708+0000 7f7e4effd640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56341872c850 0x56341872ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.709+0000 7f7e4effd640 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56341872c850 0x56341872ec80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c187949d5240a362 server_cookie=d2062ca9dea74f10 in_seq=0 out_seq=0 2024-09-10T10:46:40.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.709+0000 7f7e32ff5640 10 client.15873 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:40.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.710+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1786891419 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7e440a3ea0 con 0x56341872c850 2024-09-10T10:46:40.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.710+0000 7f7e32ff5640 10 client.15873 handle_command_reply: tid=0 2024-09-10T10:46:40.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.710+0000 7f7e578a2740 1 client.15873 shutdown 2024-09-10T10:46:40.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 msgr2=0x56341871a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871a090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7e480098b0 tx=0x7f7e480095c0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 shutdown_connections 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56341872c850 0x56341872ec80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x56341871a090 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418713350 0x563418719900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1786891419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418711de0 0x56341871c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 >> 172.21.15.143:0/1786891419 conn(0x5634186d2310 msgr2=0x5634187141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e32ff5640 0 client.15873 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 shutdown_connections 2024-09-10T10:46:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 1 -- 172.21.15.143:0/1786891419 wait complete. 2024-09-10T10:46:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.711+0000 7f7e578a2740 20 client.15873 trim_cache size 0 max 16384 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 Processor -- start 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 -- start start 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563418711de0 0x56341870e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418734550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56341871cb80 0x563418734a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ac8c0 con 0x563418711de0 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ae400 con 0x563418713350 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.738+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563418711de0 0x56341870e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.738+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563418711de0 0x56341870e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56240/0 (socket says 172.21.15.143:56240) 2024-09-10T10:46:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.738+0000 7f7e4effd640 1 -- 172.21.15.143:0/2344618058 learned_addr learned my addr 172.21.15.143:0/2344618058 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.737+0000 7f7e578a2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501acf90 con 0x56341871cb80 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.738+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418734550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.738+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56341871cb80 0x563418734a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2344618058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56341871cb80 msgr2=0x563418734a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56341871cb80 0x563418734a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2344618058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563418711de0 msgr2=0x56341870e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563418711de0 0x56341870e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/2344618058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563418717e50 con 0x563418713350 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418734550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x563418713980 tx=0x7f7e44097600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7023ae912ca6642f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e34ff9640 1 -- 172.21.15.143:0/2344618058 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e440a3ea0 con 0x563418713350 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e34ff9640 1 -- 172.21.15.143:0/2344618058 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e4409f5e0 con 0x563418713350 2024-09-10T10:46:40.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.739+0000 7f7e34ff9640 1 -- 172.21.15.143:0/2344618058 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4409f3d0 con 0x563418713350 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 -- 172.21.15.143:0/2344618058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 msgr2=0x563418734550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418734550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x563418713980 tx=0x7f7e44097600 comp rx=0 tx=0).stop 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 -- 172.21.15.143:0/2344618058 shutdown_connections 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56341871cb80 0x563418734a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418734550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 --2- 172.21.15.143:0/2344618058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563418711de0 0x56341870e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.740+0000 7f7e578a2740 1 -- 172.21.15.143:0/2344618058 >> 172.21.15.143:0/2344618058 conn(0x5634186d2310 msgr2=0x56341870df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.741+0000 7f7e578a2740 1 -- 172.21.15.143:0/2344618058 shutdown_connections 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.741+0000 7f7e578a2740 1 -- 172.21.15.143:0/2344618058 wait complete. 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.741+0000 7f7e578a2740 1 -- start start 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.741+0000 7f7e578a2740 1 Processor -- start 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418711de0 0x5634187251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418716410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x563418716950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418711de0 0x5634187251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418711de0 0x5634187251d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42154/0 (socket says 172.21.15.143:42154) 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418716410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4effd640 1 -- 172.21.15.143:0/1348990091 learned_addr learned my addr 172.21.15.143:0/1348990091 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501add30 con 0x56341871cb80 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ad660 con 0x563418713350 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501aa8a0 con 0x563418711de0 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e578a2740 10 client.0 _ll_register_callbacks cb 0x7f7e40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/1348990091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418711de0 msgr2=0x5634187251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418711de0 0x5634187251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/1348990091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 msgr2=0x563418716950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.742+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x563418716950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x563418716950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e4e7fc640 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563418716f90 con 0x563418713350 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e4409f5e0 con 0x563418713350 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e4f7fe640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x563418716950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e4e7fc640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418716410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56341871a520 tx=0x7f7e4409f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108ccdc3d5ff1949 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e32ff5640 10 client.25337 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1348990091 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4409a880 con 0x563418713350 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1348990091 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7e44096f80 con 0x563418713350 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1348990091 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e440971c0 con 0x563418713350 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e578a2740 10 client.25337 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.743+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=29}) -- 0x563418797220 con 0x563418713350 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1348990091 <== mon.2 v2:172.21.15.144:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7e4409a0f0 con 0x563418713350 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e578a2740 10 client.25337 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e578a2740 10 client.25337 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e578a2740 20 client.25337 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5634187973f0 0x5634187997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e578a2740 4 client.25337 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56341867c860 con 0x5634187973f0 2024-09-10T10:46:40.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.744+0000 7f7e4effd640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5634187973f0 0x5634187997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e4effd640 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5634187973f0 0x5634187997e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0026b165d82dddc server_cookie=1d0c764ac622f221 in_seq=0 out_seq=0 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e32ff5640 10 client.25337 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e32ff5640 1 -- 172.21.15.143:0/1348990091 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56341867c860 con 0x5634187973f0 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e32ff5640 10 client.25337 handle_command_reply: tid=0 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e578a2740 1 client.25337 shutdown 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 msgr2=0x563418716410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.745+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418716410 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56341871a520 tx=0x7f7e4409f420 comp rx=0 tx=0).stop 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 shutdown_connections 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5634187973f0 0x5634187997e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e32ff5640 0 client.25337 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56341871cb80 0x563418716950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563418713350 0x563418716410 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 --2- 172.21.15.143:0/1348990091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563418711de0 0x5634187251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 >> 172.21.15.143:0/1348990091 conn(0x5634186d2310 msgr2=0x56341871b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 shutdown_connections 2024-09-10T10:46:40.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 1 -- 172.21.15.143:0/1348990091 wait complete. 2024-09-10T10:46:40.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.746+0000 7f7e578a2740 20 client.25337 trim_cache size 0 max 16384 2024-09-10T10:46:40.753 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7e380a2c80 msgr2=0x7f7e380a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7e380a2c80 0x7f7e380a5140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e48006010 tx=0x7f7e48009800 comp rx=0 tx=0).stop 2024-09-10T10:46:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 msgr2=0x7f7e50196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e50196690 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009a60 tx=0x7f7e44009f20 comp rx=0 tx=0).stop 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 shutdown_connections 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f7e380a2c80 0x7f7e380a5140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e50102ef0 0x7f7e5019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e50102570 0x7f7e50196690 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 --2- 172.21.15.143:0/2203334411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e50100e20 0x7f7e50196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 >> 172.21.15.143:0/2203334411 conn(0x7f7e500fc7d0 msgr2=0x7f7e50105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 shutdown_connections 2024-09-10T10:46:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:40.748+0000 7f7e554c9640 1 -- 172.21.15.143:0/2203334411 wait complete. 2024-09-10T10:46:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:41.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:46:42.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.082+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/3348851301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0102d90 msgr2=0x7f8ba0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.082+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/3348851301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0102d90 0x7f8ba0103170 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009930 tx=0x7f8b9002a420 comp rx=0 tx=0).stop 2024-09-10T10:46:42.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.083+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/3348851301 shutdown_connections 2024-09-10T10:46:42.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.083+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/3348851301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba0108290 0x7f8ba010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.083+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/3348851301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0103740 0x7f8ba0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.083+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/3348851301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0102d90 0x7f8ba0103170 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.083+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/3348851301 >> 172.21.15.143:0/3348851301 conn(0x7f8ba00fc7d0 msgr2=0x7f8ba00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.083+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/3348851301 shutdown_connections 2024-09-10T10:46:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.084+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/3348851301 wait complete. 2024-09-10T10:46:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.084+0000 7f8ba5a8f640 1 Processor -- start 2024-09-10T10:46:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.084+0000 7f8ba5a8f640 1 -- start start 2024-09-10T10:46:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8ba5a8f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0102d90 0x7f8ba01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8ba5a8f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba0103740 0x7f8ba0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8ba5a8f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 0x7f8ba0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0102d90 0x7f8ba01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 0x7f8ba0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0102d90 0x7f8ba01009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42164/0 (socket says 172.21.15.143:42164) 2024-09-10T10:46:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8b9ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 0x7f8ba0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56266/0 (socket says 172.21.15.143:56266) 2024-09-10T10:46:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8b9f7fe640 1 -- 172.21.15.143:0/4265168704 learned_addr learned my addr 172.21.15.143:0/4265168704 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0111f80 con 0x7f8ba0108290 2024-09-10T10:46:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0111e00 con 0x7f8ba0103740 2024-09-10T10:46:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.085+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0112100 con 0x7f8ba0102d90 2024-09-10T10:46:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9ffff640 1 -- 172.21.15.143:0/4265168704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0102d90 msgr2=0x7f8ba01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9ffff640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0102d90 0x7f8ba01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9ffff640 1 -- 172.21.15.143:0/4265168704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba0103740 msgr2=0x7f8ba0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9ffff640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba0103740 0x7f8ba0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9ffff640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba01a69e0 con 0x7f8ba0108290 2024-09-10T10:46:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9ffff640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 0x7f8ba0101470 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009fd0 tx=0x7f8b9800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:42.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800ce20 con 0x7f8ba0108290 2024-09-10T10:46:42.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b98005020 con 0x7f8ba0108290 2024-09-10T10:46:42.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba01a6cd0 con 0x7f8ba0108290 2024-09-10T10:46:42.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.086+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98010690 con 0x7f8ba0108290 2024-09-10T10:46:42.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.088+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba01a71c0 con 0x7f8ba0108290 2024-09-10T10:46:42.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.089+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba006a7f0 con 0x7f8ba0108290 2024-09-10T10:46:42.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.089+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8b98004b20 con 0x7f8ba0108290 2024-09-10T10:46:42.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.090+0000 7f8b9cff9640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8b880a2cd0 0x7f8b880a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:42.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.090+0000 7f8b9f7fe640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8b880a2cd0 0x7f8b880a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:42.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.090+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8b980d2170 con 0x7f8ba0108290 2024-09-10T10:46:42.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.091+0000 7f8b9f7fe640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8b880a2cd0 0x7f8b880a5190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b90000c00 tx=0x7f8b90005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:42.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.097+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b9809b940 con 0x7f8ba0108290 2024-09-10T10:46:42.137 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.332+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ba0103170 con 0x7f8ba0108290 2024-09-10T10:46:42.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.333+0000 7f8b9cff9640 1 -- 172.21.15.143:0/4265168704 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f8b9809b240 con 0x7f8ba0108290 2024-09-10T10:46:42.336 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:46:42.336 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:46:42.336 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:46:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8b880a2cd0 msgr2=0x7f8b880a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8b880a2cd0 0x7f8b880a5190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b90000c00 tx=0x7f8b90005a20 comp rx=0 tx=0).stop 2024-09-10T10:46:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 msgr2=0x7f8ba0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 0x7f8ba0101470 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009fd0 tx=0x7f8b9800ef10 comp rx=0 tx=0).stop 2024-09-10T10:46:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 shutdown_connections 2024-09-10T10:46:42.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8b880a2cd0 0x7f8b880a5190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ba0108290 0x7f8ba0101470 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ba0103740 0x7f8ba0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 --2- 172.21.15.143:0/4265168704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ba0102d90 0x7f8ba01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 >> 172.21.15.143:0/4265168704 conn(0x7f8ba00fc7d0 msgr2=0x7f8ba0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 shutdown_connections 2024-09-10T10:46:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:42.336+0000 7f8ba5a8f640 1 -- 172.21.15.143:0/4265168704 wait complete. 2024-09-10T10:46:42.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:42 smithi143 ceph-mon[31057]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T10:46:42.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:42 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/4265168704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:42 smithi178 ceph-mon[34813]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T10:46:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:42 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/4265168704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:42 smithi144 ceph-mon[34230]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T10:46:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:42 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/4265168704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:46:43.071 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T10:46:43.072 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T10:46:43.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 -- 172.21.15.143:0/3808804933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8101df0 msgr2=0x7f08e81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 --2- 172.21.15.143:0/3808804933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8101df0 0x7f08e81021d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009900 tx=0x7f08e402a410 comp rx=0 tx=0).stop 2024-09-10T10:46:43.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 -- 172.21.15.143:0/3808804933 shutdown_connections 2024-09-10T10:46:43.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 --2- 172.21.15.143:0/3808804933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e8106f20 0x7f08e810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 --2- 172.21.15.143:0/3808804933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 0x7f08e81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 --2- 172.21.15.143:0/3808804933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8101df0 0x7f08e81021d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 -- 172.21.15.143:0/3808804933 >> 172.21.15.143:0/3808804933 conn(0x7f08e80fd8f0 msgr2=0x7f08e80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 -- 172.21.15.143:0/3808804933 shutdown_connections 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 -- 172.21.15.143:0/3808804933 wait complete. 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.272+0000 7f08ef427640 1 Processor -- start 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 -- start start 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e8101df0 0x7f08e8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 0x7f08e8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 0x7f08e819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111e30 con 0x7f08e8106f20 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111cb0 con 0x7f08e81027a0 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ef427640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111fb0 con 0x7f08e8101df0 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 0x7f08e819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 0x7f08e819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56294/0 (socket says 172.21.15.143:56294) 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 -- 172.21.15.143:0/3301779650 learned_addr learned my addr 172.21.15.143:0/3301779650 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 0x7f08e8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 -- 172.21.15.143:0/3301779650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e8101df0 msgr2=0x7f08e8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e8101df0 0x7f08e8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e8101df0 0x7f08e8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 -- 172.21.15.143:0/3301779650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 msgr2=0x7f08e8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 0x7f08e8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 -- 172.21.15.143:0/3301779650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e819d2e0 con 0x7f08e8106f20 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 0x7f08e8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:43.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08ed9c4640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 0x7f08e819cc00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f08e000eaa0 tx=0x7f08e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.273+0000 7f08de7fc640 1 -- 172.21.15.143:0/3301779650 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e000cdb0 con 0x7f08e8106f20 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.274+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e819d5d0 con 0x7f08e8106f20 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.274+0000 7f08de7fc640 1 -- 172.21.15.143:0/3301779650 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e0005020 con 0x7f08e8106f20 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.274+0000 7f08de7fc640 1 -- 172.21.15.143:0/3301779650 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00116d0 con 0x7f08e8106f20 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.274+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e81a4eb0 con 0x7f08e8106f20 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08de7fc640 1 -- 172.21.15.143:0/3301779650 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f08e00051d0 con 0x7f08e8106f20 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 Processor -- start 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 -- start start 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb31a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589cfa972f0 con 0x5589cfb315d0 2024-09-10T10:46:43.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5589cfa3ecf0 con 0x5589cfb32b40 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.276+0000 7f08f1800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589cfa932b0 con 0x5589cfb3c370 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08de7fc640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08d40a2b80 0x7f08d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb31a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb31a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56310/0 (socket says 172.21.15.143:56310) 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 -- 172.21.15.143:0/2364482453 learned_addr learned my addr 172.21.15.143:0/2364482453 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed9c4640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08d40a2b80 0x7f08d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08de7fc640 1 -- 172.21.15.143:0/3301779650 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f08e00cf560 con 0x7f08e8106f20 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 -- 172.21.15.143:0/2364482453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 msgr2=0x5589cfb3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb3e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 -- 172.21.15.143:0/2364482453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 msgr2=0x5589cfb31090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb31090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 -- 172.21.15.143:0/2364482453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cfa233a0 con 0x5589cfb315d0 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb31a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f08e4006010 tx=0x7f08e4032b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca6a829b03b9030 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08d37fe640 1 -- 172.21.15.143:0/2364482453 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4035030 con 0x5589cfb315d0 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08d37fe640 1 -- 172.21.15.143:0/2364482453 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e40314a0 con 0x5589cfb315d0 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.277+0000 7f08d37fe640 1 -- 172.21.15.143:0/2364482453 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4005040 con 0x5589cfb315d0 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 -- 172.21.15.143:0/2364482453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 msgr2=0x5589cfb31a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb31a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f08e4006010 tx=0x7f08e4032b80 comp rx=0 tx=0).stop 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08ed9c4640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08d40a2b80 0x7f08d40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f08e00054c0 tx=0x7f08e0000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 -- 172.21.15.143:0/2364482453 shutdown_connections 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb31a10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 --2- 172.21.15.143:0/2364482453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb31090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 -- 172.21.15.143:0/2364482453 >> 172.21.15.143:0/2364482453 conn(0x5589cfafa310 msgr2=0x5589cfb2dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 -- 172.21.15.143:0/2364482453 shutdown_connections 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 -- 172.21.15.143:0/2364482453 wait complete. 2024-09-10T10:46:43.458 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.278+0000 7f08f1800740 1 -- start start 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 Processor -- start 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb3bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb38f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589cfb3f010 con 0x5589cfb315d0 2024-09-10T10:46:43.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5589cfa52060 con 0x5589cfb32b40 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589cfb3f190 con 0x5589cfb3c370 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08f1800740 10 client.0 _ll_register_callbacks cb 0x5589cf7a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb3bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb3bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56326/0 (socket says 172.21.15.143:56326) 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 -- 172.21.15.143:0/4156707849 learned_addr learned my addr 172.21.15.143:0/4156707849 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb38f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 -- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 msgr2=0x5589cfb394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 -- 172.21.15.143:0/4156707849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 msgr2=0x5589cfb38f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb38f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.279+0000 7f08ec9c2640 1 -- 172.21.15.143:0/4156707849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cf8083c0 con 0x5589cfb315d0 2024-09-10T10:46:43.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb3bce0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f08d8012d60 tx=0x7f08d8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63421dd706635335 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:43.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08d0ff9640 1 -- 172.21.15.143:0/4156707849 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004050 con 0x5589cfb315d0 2024-09-10T10:46:43.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08d0ff9640 1 -- 172.21.15.143:0/4156707849 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f08d80041f0 con 0x5589cfb315d0 2024-09-10T10:46:43.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e40314a0 con 0x5589cfb315d0 2024-09-10T10:46:43.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08d0ff9640 1 -- 172.21.15.143:0/4156707849 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08d8026050 con 0x5589cfb315d0 2024-09-10T10:46:43.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 10 client.15894 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:43.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589cfb4aa20 con 0x5589cfb315d0 2024-09-10T10:46:43.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08d0ff9640 1 -- 172.21.15.143:0/4156707849 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e40314a0 con 0x5589cfb315d0 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08d0ff9640 1 -- 172.21.15.143:0/4156707849 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f08d800f440 con 0x5589cfb315d0 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 10 client.15894 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 10 client.15894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 20 client.15894 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfb4bf30 0x5589cfb4e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 4 client.15894 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:43.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.280+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589cfaa4860 con 0x5589cfb4bf30 2024-09-10T10:46:43.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.281+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfb4bf30 0x5589cfb4e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.281+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfb4bf30 0x5589cfb4e370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dbcf04637919a138 server_cookie=1d494734f69f1403 in_seq=0 out_seq=0 2024-09-10T10:46:43.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.281+0000 7f08d0ff9640 10 client.15894 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:43.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08d0ff9640 1 -- 172.21.15.143:0/4156707849 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589cfaa4860 con 0x5589cfb4bf30 2024-09-10T10:46:43.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08d0ff9640 10 client.15894 handle_command_reply: tid=0 2024-09-10T10:46:43.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 client.15894 shutdown 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 msgr2=0x5589cfb3bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb3bce0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f08d8012d60 tx=0x7f08d8016040 comp rx=0 tx=0).stop 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 shutdown_connections 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfb4bf30 0x5589cfb4e370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb32b40 0x5589cfb38f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08d0ff9640 0 client.15894 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 --2- 172.21.15.143:0/4156707849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb3bce0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 >> 172.21.15.143:0/4156707849 conn(0x5589cfafa310 msgr2=0x5589cfb33930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 shutdown_connections 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 1 -- 172.21.15.143:0/4156707849 wait complete. 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.282+0000 7f08f1800740 20 client.15894 trim_cache size 0 max 16384 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 Processor -- start 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 -- start start 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb32b40 0x5589cfb38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb3c370 0x5589cfb37f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81ac970 con 0x5589cfb315d0 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81ae4b0 con 0x5589cfb3c370 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08f1800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81ad040 con 0x5589cfb32b40 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08ec9c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08ec9c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56330/0 (socket says 172.21.15.143:56330) 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.295+0000 7f08ec9c2640 1 -- 172.21.15.143:0/3203717549 learned_addr learned my addr 172.21.15.143:0/3203717549 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08ec9c2640 1 -- 172.21.15.143:0/3203717549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb32b40 msgr2=0x5589cfb38850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb32b40 0x5589cfb38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08ec9c2640 1 -- 172.21.15.143:0/3203717549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb3c370 msgr2=0x5589cfb37f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb3c370 0x5589cfb37f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08ec9c2640 1 -- 172.21.15.143:0/3203717549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cfb32760 con 0x5589cfb315d0 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb482b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f08d800fa20 tx=0x7f08d8012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe7502f8ee839c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08d2ffd640 1 -- 172.21.15.143:0/3203717549 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8005850 con 0x5589cfb315d0 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08d2ffd640 1 -- 172.21.15.143:0/3203717549 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08d800f440 con 0x5589cfb315d0 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.296+0000 7f08d2ffd640 1 -- 172.21.15.143:0/3203717549 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004490 con 0x5589cfb315d0 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 -- 172.21.15.143:0/3203717549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 msgr2=0x5589cfb482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb482b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f08d800fa20 tx=0x7f08d8012890 comp rx=0 tx=0).stop 2024-09-10T10:46:43.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 -- 172.21.15.143:0/3203717549 shutdown_connections 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb3c370 0x5589cfb37f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb32b40 0x5589cfb38850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 --2- 172.21.15.143:0/3203717549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb315d0 0x5589cfb482b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 -- 172.21.15.143:0/3203717549 >> 172.21.15.143:0/3203717549 conn(0x5589cfafa310 msgr2=0x5589cfb37860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 -- 172.21.15.143:0/3203717549 shutdown_connections 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 -- 172.21.15.143:0/3203717549 wait complete. 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 -- start start 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.297+0000 7f08f1800740 1 Processor -- start 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb315d0 0x5589cfb44600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 0x5589cfb44b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81adde0 con 0x5589cfb32b40 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81ad710 con 0x5589cfb315d0 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81aa950 con 0x5589cfb3c370 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 0x5589cfb44b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08f1800740 10 client.0 _ll_register_callbacks cb 0x7f08d80129a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 0x5589cfb44b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56340/0 (socket says 172.21.15.143:56340) 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 -- 172.21.15.143:0/3317411041 learned_addr learned my addr 172.21.15.143:0/3317411041 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb315d0 0x5589cfb44600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 -- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 msgr2=0x5589cfb357c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 -- 172.21.15.143:0/3317411041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb315d0 msgr2=0x5589cfb44600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb315d0 0x5589cfb44600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 -- 172.21.15.143:0/3317411041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cfb35f00 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.298+0000 7f08ed1c3640 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 0x5589cfb44b40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08e4000c00 tx=0x7f08e4004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc107cfbccbce989 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08d0ff9640 1 -- 172.21.15.143:0/3317411041 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4056060 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08d0ff9640 1 -- 172.21.15.143:0/3317411041 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f08e4005450 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d800f440 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08d0ff9640 1 -- 172.21.15.143:0/3317411041 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08e405d050 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 10 client.15900 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589cfbb6610 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08d0ff9640 1 -- 172.21.15.143:0/3317411041 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08d800f440 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08d0ff9640 1 -- 172.21.15.143:0/3317411041 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f08e4004950 con 0x5589cfb32b40 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 10 client.15900 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:43.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 10 client.15900 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 20 client.15900 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfbb6900 0x5589cfbb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 4 client.15900 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.299+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589cfa9ed70 con 0x5589cfbb6900 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfbb6900 0x5589cfbb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08d0ff9640 10 client.15900 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08ec9c2640 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfbb6900 0x5589cfbb8cf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2fdb51709012d068 server_cookie=5c54ca0e93e97aa5 in_seq=0 out_seq=0 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08d0ff9640 1 -- 172.21.15.143:0/3317411041 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589cfa9ed70 con 0x5589cfbb6900 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08d0ff9640 10 client.15900 handle_command_reply: tid=0 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 client.15900 shutdown 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 msgr2=0x5589cfb44b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 0x5589cfb44b40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08e4000c00 tx=0x7f08e4004550 comp rx=0 tx=0).stop 2024-09-10T10:46:43.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 shutdown_connections 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5589cfbb6900 0x5589cfbb8cf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589cfb3c370 0x5589cfb357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08d0ff9640 0 client.15900 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589cfb32b40 0x5589cfb44b40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 --2- 172.21.15.143:0/3317411041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589cfb315d0 0x5589cfb44600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 >> 172.21.15.143:0/3317411041 conn(0x5589cfafa310 msgr2=0x5589cfb38080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 shutdown_connections 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 1 -- 172.21.15.143:0/3317411041 wait complete. 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.300+0000 7f08f1800740 20 client.15900 trim_cache size 0 max 16384 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08d40a2b80 msgr2=0x7f08d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08d40a2b80 0x7f08d40a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f08e00054c0 tx=0x7f08e0000f40 comp rx=0 tx=0).stop 2024-09-10T10:46:43.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 msgr2=0x7f08e819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 0x7f08e819cc00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f08e000eaa0 tx=0x7f08e000ef60 comp rx=0 tx=0).stop 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 shutdown_connections 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f08d40a2b80 0x7f08d40a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08e8106f20 0x7f08e819cc00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08e81027a0 0x7f08e8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 --2- 172.21.15.143:0/3301779650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08e8101df0 0x7f08e8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 >> 172.21.15.143:0/3301779650 conn(0x7f08e80fd8f0 msgr2=0x7f08e80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 shutdown_connections 2024-09-10T10:46:43.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:43.301+0000 7f08ef427640 1 -- 172.21.15.143:0/3301779650 wait complete. 2024-09-10T10:46:44.446 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:46:44.571 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:44.700 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:44 smithi143 ceph-mon[31057]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T10:46:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:44 smithi178 ceph-mon[34813]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T10:46:44.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:44 smithi144 ceph-mon[34230]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/3865488224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 msgr2=0x7fd2a410bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/3865488224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a410bbe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd294009ca0 tx=0x7fd29402a950 comp rx=0 tx=0).stop 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/3865488224 shutdown_connections 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/3865488224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2a410c1b0 0x7fd2a410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/3865488224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a410bbe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/3865488224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a4102dd0 0x7fd2a41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/3865488224 >> 172.21.15.143:0/3865488224 conn(0x7fd2a40fc7d0 msgr2=0x7fd2a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/3865488224 shutdown_connections 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.186+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/3865488224 wait complete. 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.187+0000 7fd2ab1a0640 1 Processor -- start 2024-09-10T10:46:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.187+0000 7fd2ab1a0640 1 -- start start 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2a4102dd0 0x7fd2a41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a410c1b0 0x7fd2a419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4111e30 con 0x7fd2a41036f0 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4111cb0 con 0x7fd2a410c1b0 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4111fb0 con 0x7fd2a4102dd0 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56360/0 (socket says 172.21.15.143:56360) 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1390258408 learned_addr learned my addr 172.21.15.143:0/1390258408 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a973d640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a410c1b0 0x7fd2a419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2a4102dd0 0x7fd2a41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1390258408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2a4102dd0 msgr2=0x7fd2a41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2a4102dd0 0x7fd2a41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1390258408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a410c1b0 msgr2=0x7fd2a419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a410c1b0 0x7fd2a419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1390258408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a419d380 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a4198910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd294005cc0 tx=0x7fd29402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a1ffb640 1 -- 172.21.15.143:0/1390258408 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294034040 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a1ffb640 1 -- 172.21.15.143:0/1390258408 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2940308f0 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a419d610 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.188+0000 7fd2a1ffb640 1 -- 172.21.15.143:0/1390258408 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2940047c0 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.189+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a41a4f50 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.191+0000 7fd2a1ffb640 1 -- 172.21.15.143:0/1390258408 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd294037020 con 0x7fd2a41036f0 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.191+0000 7fd2ad579740 1 Processor -- start 2024-09-10T10:46:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.191+0000 7fd2ad579740 1 -- start start 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4406b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4411400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4411940 0x55e3e4413d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2ad579740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3e436c2a0 con 0x55e3e4407080 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2ad579740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3e4313b10 con 0x55e3e4406760 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2ad579740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e3e4368260 con 0x55e3e4411940 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4411400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4411400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56362/0 (socket says 172.21.15.143:56362) 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 -- 172.21.15.143:0/3539102651 learned_addr learned my addr 172.21.15.143:0/3539102651 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a973d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4406b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4411940 0x55e3e4413d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a1ffb640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 -- 172.21.15.143:0/3539102651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4411940 msgr2=0x55e3e4413d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4411940 0x55e3e4413d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 -- 172.21.15.143:0/3539102651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 msgr2=0x55e3e4406b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4406b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 -- 172.21.15.143:0/3539102651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e40ddb90 con 0x55e3e4407080 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a973d640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a1ffb640 1 -- 172.21.15.143:0/1390258408 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd2940f0390 con 0x7fd2a41036f0 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.192+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4411400 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00bcb0 tx=0x7fd29c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ae185fe6cfa5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.193+0000 7fd29affd640 1 -- 172.21.15.143:0/3539102651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c015070 con 0x55e3e4407080 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.193+0000 7fd29affd640 1 -- 172.21.15.143:0/3539102651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd29c01a910 con 0x55e3e4407080 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.193+0000 7fd29affd640 1 -- 172.21.15.143:0/3539102651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c011930 con 0x55e3e4407080 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.193+0000 7fd2ad579740 1 -- 172.21.15.143:0/3539102651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 msgr2=0x55e3e4411400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.193+0000 7fd2a973d640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd2900a2b80 0x7fd2900a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005fd0 tx=0x7fd28c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.193+0000 7fd2ad579740 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4411400 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00bcb0 tx=0x7fd29c00f730 comp rx=0 tx=0).stop 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 -- 172.21.15.143:0/3539102651 shutdown_connections 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4411940 0x55e3e4413d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4411400 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 --2- 172.21.15.143:0/3539102651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4406b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 -- 172.21.15.143:0/3539102651 >> 172.21.15.143:0/3539102651 conn(0x55e3e4402cf0 msgr2=0x55e3e44030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 -- 172.21.15.143:0/3539102651 shutdown_connections 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 -- 172.21.15.143:0/3539102651 wait complete. 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 -- start start 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.194+0000 7fd2ad579740 1 Processor -- start 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e440e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4407080 0x55e3e440ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e440f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3e4414510 con 0x55e3e4411940 2024-09-10T10:46:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3e42f46e0 con 0x55e3e4407080 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e3e4414690 con 0x55e3e4406760 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 10 client.0 _ll_register_callbacks cb 0x55e3e4318a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e440f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e440f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56366/0 (socket says 172.21.15.143:56366) 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 -- 172.21.15.143:0/2210225766 learned_addr learned my addr 172.21.15.143:0/2210225766 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e440e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4407080 0x55e3e440ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 -- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 msgr2=0x55e3e440e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e440e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 -- 172.21.15.143:0/2210225766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4407080 msgr2=0x55e3e440ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4407080 0x55e3e440ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 -- 172.21.15.143:0/2210225766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e441da00 con 0x55e3e4411940 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4407080 0x55e3e440ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2a973d640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e440f290 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd28c010af0 tx=0x7fd28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc74c0343828421 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd283fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd283fff640 1 -- 172.21.15.143:0/2210225766 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c028420 con 0x55e3e4411940 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.195+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29c01a910 con 0x55e3e4411940 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd283fff640 1 -- 172.21.15.143:0/2210225766 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd28c0285c0 con 0x55e3e4411940 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd283fff640 1 -- 172.21.15.143:0/2210225766 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd28c030050 con 0x55e3e4411940 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 10 client.15912 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:45.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3e44203d0 con 0x55e3e4411940 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd283fff640 1 -- 172.21.15.143:0/2210225766 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd29c01a910 con 0x55e3e4411940 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd283fff640 1 -- 172.21.15.143:0/2210225766 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd28c01f7f0 con 0x55e3e4411940 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 10 client.15912 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 10 client.15912 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 20 client.15912 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e4421890 0x55e3e4423d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 4 client.15912 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3e43799d0 con 0x55e3e4421890 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.196+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e4421890 0x55e3e4423d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.197+0000 7fd283fff640 10 client.15912 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.197+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e4421890 0x55e3e4423d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3101578e5a31f042 server_cookie=98d05c7c4b90d9c8 in_seq=0 out_seq=0 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd283fff640 1 -- 172.21.15.143:0/2210225766 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3e43799d0 con 0x55e3e4421890 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd283fff640 10 client.15912 handle_command_reply: tid=0 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 client.15912 shutdown 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 msgr2=0x55e3e440f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e440f290 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd28c010af0 tx=0x7fd28c004730 comp rx=0 tx=0).stop 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 shutdown_connections 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e440f290 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4407080 0x55e3e440ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e4421890 0x55e3e4423d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 --2- 172.21.15.143:0/2210225766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e440e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 >> 172.21.15.143:0/2210225766 conn(0x55e3e4402cf0 msgr2=0x55e3e4408a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd283fff640 0 client.15912 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 shutdown_connections 2024-09-10T10:46:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 1 -- 172.21.15.143:0/2210225766 wait complete. 2024-09-10T10:46:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.198+0000 7fd2ad579740 20 client.15912 trim_cache size 0 max 16384 2024-09-10T10:46:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 Processor -- start 2024-09-10T10:46:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 -- start start 2024-09-10T10:46:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e44091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4429650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4411940 0x55e3e4429b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41aeb30 con 0x55e3e4407080 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41b0670 con 0x55e3e4411940 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2ad579740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41af200 con 0x55e3e4406760 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4429650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4429650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56370/0 (socket says 172.21.15.143:56370) 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.215+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1520663849 learned_addr learned my addr 172.21.15.143:0/1520663849 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a973d640 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4411940 0x55e3e4429b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e44091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1520663849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 msgr2=0x55e3e44091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e44091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1520663849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4411940 msgr2=0x55e3e4429b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4411940 0x55e3e4429b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a3fff640 1 -- 172.21.15.143:0/1520663849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e4402b20 con 0x55e3e4407080 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4429650 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2940b8d30 tx=0x7fd29402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11fa83f464191ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd29a7fc640 1 -- 172.21.15.143:0/1520663849 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2940a5e60 con 0x55e3e4407080 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd29a7fc640 1 -- 172.21.15.143:0/1520663849 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2940bcfd0 con 0x55e3e4407080 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.216+0000 7fd29a7fc640 1 -- 172.21.15.143:0/1520663849 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294004040 con 0x55e3e4407080 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 -- 172.21.15.143:0/1520663849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 msgr2=0x55e3e4429650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4429650 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2940b8d30 tx=0x7fd29402a950 comp rx=0 tx=0).stop 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 -- 172.21.15.143:0/1520663849 shutdown_connections 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4411940 0x55e3e4429b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4407080 0x55e3e4429650 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 --2- 172.21.15.143:0/1520663849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4406760 0x55e3e44091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 -- 172.21.15.143:0/1520663849 >> 172.21.15.143:0/1520663849 conn(0x55e3e4402cf0 msgr2=0x55e3e4410890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 -- 172.21.15.143:0/1520663849 shutdown_connections 2024-09-10T10:46:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 -- 172.21.15.143:0/1520663849 wait complete. 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 -- start start 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.217+0000 7fd2ad579740 1 Processor -- start 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4415ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4407080 0x55e3e4416530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e44195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41affa0 con 0x55e3e4411940 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41af8d0 con 0x55e3e4406760 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41acb10 con 0x55e3e4407080 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2ad579740 10 client.0 _ll_register_callbacks cb 0x7fd28c0107c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e44195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e44195a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56376/0 (socket says 172.21.15.143:56376) 2024-09-10T10:46:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 -- 172.21.15.143:0/86656178 learned_addr learned my addr 172.21.15.143:0/86656178 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4415ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a3fff640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4407080 0x55e3e4416530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 -- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4407080 msgr2=0x55e3e4416530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4407080 0x55e3e4416530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 -- 172.21.15.143:0/86656178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 msgr2=0x55e3e4415ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4415ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.218+0000 7fd2a973d640 1 -- 172.21.15.143:0/86656178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e4403980 con 0x55e3e4411940 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2a973d640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e44195a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd28c01fe00 tx=0x7fd28c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f47a74615518abf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd283fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd283fff640 1 -- 172.21.15.143:0/86656178 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c038020 con 0x55e3e4411940 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd283fff640 1 -- 172.21.15.143:0/86656178 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd28c01f620 con 0x55e3e4411940 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2940bcfd0 con 0x55e3e4411940 2024-09-10T10:46:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd283fff640 1 -- 172.21.15.143:0/86656178 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd28c043050 con 0x55e3e4411940 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3e43fe260 con 0x55e3e4411940 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd283fff640 1 -- 172.21.15.143:0/86656178 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2940bcfd0 con 0x55e3e4411940 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd283fff640 1 -- 172.21.15.143:0/86656178 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd28c028420 con 0x55e3e4411940 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 10 client.15918 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 20 client.15918 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e448d400 0x55e3e448f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 4 client.15918 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.219+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55e3e4373d20 con 0x55e3e448d400 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.220+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e448d400 0x55e3e448f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.220+0000 7fd2a8f3c640 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e448d400 0x55e3e448f820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b48612facc3d86 server_cookie=11d884fc74a3fa72 in_seq=0 out_seq=0 2024-09-10T10:46:45.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.220+0000 7fd283fff640 10 client.15918 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd283fff640 1 -- 172.21.15.143:0/86656178 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e3e4373d20 con 0x55e3e448d400 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd283fff640 10 client.15918 handle_command_reply: tid=0 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 client.15918 shutdown 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 msgr2=0x55e3e44195a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e44195a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd28c01fe00 tx=0x7fd28c00f7f0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 shutdown_connections 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3e4411940 0x55e3e44195a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e3e448d400 0x55e3e448f820 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd283fff640 0 client.15918 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3e4407080 0x55e3e4416530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 --2- 172.21.15.143:0/86656178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3e4406760 0x55e3e4415ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.227+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 >> 172.21.15.143:0/86656178 conn(0x55e3e4402cf0 msgr2=0x55e3e4410070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 shutdown_connections 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ad579740 1 -- 172.21.15.143:0/86656178 wait complete. 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ad579740 20 client.15918 trim_cache size 0 max 16384 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 29, 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 6, 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: "218ba9b": { 2024-09-10T10:46:45.235 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 6, 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 150.0, 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:46:45.236 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:45.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd2900a2b80 msgr2=0x7fd2900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd2900a2b80 0x7fd2900a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005fd0 tx=0x7fd28c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 msgr2=0x7fd2a4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a4198910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd294005cc0 tx=0x7fd29402f650 comp rx=0 tx=0).stop 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 shutdown_connections 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a410c1b0 0x7fd2a419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2a41036f0 0x7fd2a4198910 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 --2- 172.21.15.143:0/1390258408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2a4102dd0 0x7fd2a41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 >> 172.21.15.143:0/1390258408 conn(0x7fd2a40fc7d0 msgr2=0x7fd2a40fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.228+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 shutdown_connections 2024-09-10T10:46:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:45.229+0000 7fd2ab1a0640 1 -- 172.21.15.143:0/1390258408 wait complete. 2024-09-10T10:46:45.520 INFO:tasks.workunit.client.0.smithi143.stderr:Updating files: 91% (11621/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T10:46:46.009 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '218ba9b', quiesce took 5.6 seconds. Will release after: 24.4 2024-09-10T10:46:46.088 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T10:46:46.232 INFO:tasks.workunit.client.0.smithi143.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: git switch -c 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr:Or undo this operation with: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: git switch - 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:46:46.233 INFO:tasks.workunit.client.0.smithi143.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:46:46.238 DEBUG:teuthology.orchestra.run.smithi143:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T10:46:46.283 INFO:tasks.workunit.client.0.smithi143.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T10:46:46.285 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:46:46.285 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T10:46:46.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 -- 172.21.15.143:0/3526900425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0101de0 msgr2=0x7f50f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 --2- 172.21.15.143:0/3526900425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0101de0 0x7f50f01021c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009ca0 tx=0x7f50e002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 -- 172.21.15.143:0/3526900425 shutdown_connections 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 --2- 172.21.15.143:0/3526900425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f0106f10 0x7f50f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 --2- 172.21.15.143:0/3526900425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f0102790 0x7f50f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 --2- 172.21.15.143:0/3526900425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0101de0 0x7f50f01021c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 -- 172.21.15.143:0/3526900425 >> 172.21.15.143:0/3526900425 conn(0x7f50f00fd8c0 msgr2=0x7f50f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 -- 172.21.15.143:0/3526900425 shutdown_connections 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.722+0000 7f50f5795640 1 -- 172.21.15.143:0/3526900425 wait complete. 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 Processor -- start 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 -- start start 2024-09-10T10:46:46.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f0101de0 0x7f50f0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 0x7f50f01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f0106f10 0x7f50f019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0111e20 con 0x7f50f0102790 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0111ca0 con 0x7f50f0101de0 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50f5795640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0111fa0 con 0x7f50f0106f10 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 0x7f50f01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 0x7f50f01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34716/0 (socket says 172.21.15.143:34716) 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.723+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1391236104 learned_addr learned my addr 172.21.15.143:0/1391236104 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50eeffd640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f0101de0 0x7f50f0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1391236104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f0106f10 msgr2=0x7f50f019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f0106f10 0x7f50f019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1391236104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f0101de0 msgr2=0x7f50f0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f0101de0 0x7f50f0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1391236104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f019d330 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 0x7f50f01988c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f50e400c970 tx=0x7f50e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50d7fff640 1 -- 172.21.15.143:0/1391236104 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4007bb0 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f019d620 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50d7fff640 1 -- 172.21.15.143:0/1391236104 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e4007d50 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50d7fff640 1 -- 172.21.15.143:0/1391236104 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400f450 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.724+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f01a4f00 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.726+0000 7f50d7fff640 1 -- 172.21.15.143:0/1391236104 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f50e4004f20 con 0x7f50f0102790 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.726+0000 7f50f7b6e740 1 Processor -- start 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 -- start start 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717ce3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717ce3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 0x559717cf0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559717c492f0 con 0x559717cee370 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559717bf0cf0 con 0x559717ce4b40 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559717c452b0 con 0x559717ce35d0 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 0x559717cf0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50d7fff640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f50d80a2b80 0x7f50d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 0x559717cf0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34720/0 (socket says 172.21.15.143:34720) 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3440186597 learned_addr learned my addr 172.21.15.143:0/3440186597 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50eeffd640 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717ce3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ef7fe640 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717ce3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ef7fe640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f50d80a2b80 0x7f50d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50d7fff640 1 -- 172.21.15.143:0/1391236104 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f50e40cf1d0 con 0x7f50f0102790 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3440186597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 msgr2=0x559717ce3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717ce3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3440186597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 msgr2=0x559717ce3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717ce3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.727+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3440186597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559717bd53a0 con 0x559717cee370 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 0x559717cf0760 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f50e409fc40 tx=0x7f50e4097370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6a38b7a7bf36a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50ef7fe640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f50d80a2b80 0x7f50d80a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50e8013720 tx=0x7f50e8005a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50d57fa640 1 -- 172.21.15.143:0/3440186597 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e401d070 con 0x559717cee370 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50d57fa640 1 -- 172.21.15.143:0/3440186597 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e409bdd0 con 0x559717cee370 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50d57fa640 1 -- 172.21.15.143:0/3440186597 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4097d40 con 0x559717cee370 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3440186597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 msgr2=0x559717cf0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 0x559717cf0760 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f50e409fc40 tx=0x7f50e4097370 comp rx=0 tx=0).stop 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3440186597 shutdown_connections 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717cee370 0x559717cf0760 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717ce3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3440186597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717ce3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3440186597 >> 172.21.15.143:0/3440186597 conn(0x559717cac310 msgr2=0x559717cdfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3440186597 shutdown_connections 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.728+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3440186597 wait complete. 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 -- start start 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 Processor -- start 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cedd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ceb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717ceb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559717cf1010 con 0x559717ce4b40 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559717c04060 con 0x559717cee370 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559717cf1190 con 0x559717ce35d0 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50f7b6e740 10 client.0 _ll_register_callbacks cb 0x5597179578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ceb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ceb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34730/0 (socket says 172.21.15.143:34730) 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.729+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3657465505 learned_addr learned my addr 172.21.15.143:0/3657465505 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50eeffd640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cedd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ef7fe640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717ceb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 msgr2=0x559717cedd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cedd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3657465505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 msgr2=0x559717ceb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717ceb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ee7fc640 1 -- 172.21.15.143:0/3657465505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597179ba3c0 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50eeffd640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cedd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ceb000 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f50e40cc9c0 tx=0x7f50e409b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1c7e338ad39218 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50d2ff5640 1 -- 172.21.15.143:0/3657465505 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40b3070 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50d2ff5640 1 -- 172.21.15.143:0/3657465505 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f50e40a3eb0 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e409bdd0 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50d2ff5640 1 -- 172.21.15.143:0/3657465505 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50e401d050 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50f7b6e740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559717cfca90 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50d2ff5640 1 -- 172.21.15.143:0/3657465505 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e409bdd0 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50d2ff5640 1 -- 172.21.15.143:0/3657465505 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f50e409a590 con 0x559717ce4b40 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50f7b6e740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:46.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50f7b6e740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.730+0000 7f50f7b6e740 20 client.15930 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.731+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717cfdde0 0x559717d001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.731+0000 7f50f7b6e740 4 client.15930 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.731+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559717c56860 con 0x559717cfdde0 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.731+0000 7f50eeffd640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717cfdde0 0x559717d001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.731+0000 7f50d2ff5640 10 client.15930 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.731+0000 7f50eeffd640 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717cfdde0 0x559717d001d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bda9fda24ff04836 server_cookie=a0ba47ea48a839b2 in_seq=0 out_seq=0 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50d2ff5640 1 -- 172.21.15.143:0/3657465505 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559717c56860 con 0x559717cfdde0 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50d2ff5640 10 client.15930 handle_command_reply: tid=0 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 client.15930 shutdown 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 msgr2=0x559717ceb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ceb000 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f50e40cc9c0 tx=0x7f50e409b520 comp rx=0 tx=0).stop 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 shutdown_connections 2024-09-10T10:46:46.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717cfdde0 0x559717d001d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717ceb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50d2ff5640 0 client.15930 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ceb000 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/3657465505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cedd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 >> 172.21.15.143:0/3657465505 conn(0x559717cac310 msgr2=0x559717ce5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 shutdown_connections 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 1 -- 172.21.15.143:0/3657465505 wait complete. 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.732+0000 7f50f7b6e740 20 client.15930 trim_cache size 0 max 16384 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 Processor -- start 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 -- start start 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cdfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ce9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717d031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01aeb20 con 0x559717ce4b40 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b0660 con 0x559717cee370 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01af1f0 con 0x559717ce35d0 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ce9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ce9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34740/0 (socket says 172.21.15.143:34740) 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1742756572 learned_addr learned my addr 172.21.15.143:0/1742756572 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50eeffd640 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cdfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1742756572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 msgr2=0x559717cdfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.796 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cdfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1742756572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 msgr2=0x559717d031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717d031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50ee7fc640 1 -- 172.21.15.143:0/1742756572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559717ced970 con 0x559717ce4b40 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.745+0000 7f50eeffd640 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cdfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ce9fe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f50e40980a0 tx=0x7f50e4099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8243c57a560889 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50d4ff9640 1 -- 172.21.15.143:0/1742756572 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40b3070 con 0x559717ce4b40 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50d4ff9640 1 -- 172.21.15.143:0/1742756572 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e409bdd0 con 0x559717ce4b40 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50d4ff9640 1 -- 172.21.15.143:0/1742756572 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4065080 con 0x559717ce4b40 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 -- 172.21.15.143:0/1742756572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 msgr2=0x559717ce9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ce9fe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f50e40980a0 tx=0x7f50e4099be0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 -- 172.21.15.143:0/1742756572 shutdown_connections 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717cee370 0x559717d031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce4b40 0x559717ce9fe0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/1742756572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717ce35d0 0x559717cdfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 -- 172.21.15.143:0/1742756572 >> 172.21.15.143:0/1742756572 conn(0x559717cac310 msgr2=0x559717ce98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 -- 172.21.15.143:0/1742756572 shutdown_connections 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.746+0000 7f50f7b6e740 1 -- 172.21.15.143:0/1742756572 wait complete. 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 -- start start 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 Processor -- start 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 0x559717d007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717d00d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717cee370 0x559717d606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01aff90 con 0x559717ce35d0 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01af8c0 con 0x559717ce4b40 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01acb00 con 0x559717cee370 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50f7b6e740 10 client.0 _ll_register_callbacks cb 0x7f50e800e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 0x559717d007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 0x559717d007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34754/0 (socket says 172.21.15.143:34754) 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 -- 172.21.15.143:0/321561538 learned_addr learned my addr 172.21.15.143:0/321561538 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50ef7fe640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717cee370 0x559717d606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717d00d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 -- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717cee370 msgr2=0x559717d606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717cee370 0x559717d606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 -- 172.21.15.143:0/321561538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 msgr2=0x559717d00d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717d00d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50eeffd640 1 -- 172.21.15.143:0/321561538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559717d60d20 con 0x559717ce35d0 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.747+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717d00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50eeffd640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 0x559717d007d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f50e0006010 tx=0x7f50e00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4ab66ba8fc80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50d2ff5640 1 -- 172.21.15.143:0/321561538 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003d80 con 0x559717ce35d0 2024-09-10T10:46:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50d2ff5640 1 -- 172.21.15.143:0/321561538 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f50e0004040 con 0x559717ce35d0 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e409bdd0 con 0x559717ce35d0 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50d2ff5640 1 -- 172.21.15.143:0/321561538 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50e0031840 con 0x559717ce35d0 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 10 client.15936 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559717d68060 con 0x559717ce35d0 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50d2ff5640 1 -- 172.21.15.143:0/321561538 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50e409bdd0 con 0x559717ce35d0 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50d2ff5640 1 -- 172.21.15.143:0/321561538 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f50e0044070 con 0x559717ce35d0 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 20 client.15936 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717d68760 0x559717d6ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 4 client.15936 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.748+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559717c50d70 con 0x559717d68760 2024-09-10T10:46:46.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717d68760 0x559717d6ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50ee7fc640 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717d68760 0x559717d6ab80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b8bf51934e15431 server_cookie=8acf3a0aaa9c8076 in_seq=0 out_seq=0 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50d2ff5640 10 client.15936 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50d2ff5640 1 -- 172.21.15.143:0/321561538 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559717c50d70 con 0x559717d68760 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50d2ff5640 10 client.15936 handle_command_reply: tid=0 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 client.15936 shutdown 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 msgr2=0x559717d007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 0x559717d007d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f50e0006010 tx=0x7f50e00316d0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 shutdown_connections 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x559717d68760 0x559717d6ab80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559717cee370 0x559717d606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559717ce4b40 0x559717d00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50d2ff5640 0 client.15936 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 --2- 172.21.15.143:0/321561538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559717ce35d0 0x559717d007d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 >> 172.21.15.143:0/321561538 conn(0x559717cac310 msgr2=0x559717cde4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 shutdown_connections 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 1 -- 172.21.15.143:0/321561538 wait complete. 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.749+0000 7f50f7b6e740 20 client.15936 trim_cache size 0 max 16384 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f50d80a2b80 msgr2=0x7f50d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f50d80a2b80 0x7f50d80a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50e8013720 tx=0x7f50e8005a10 comp rx=0 tx=0).stop 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 msgr2=0x7f50f01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 0x7f50f01988c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f50e400c970 tx=0x7f50e400ce30 comp rx=0 tx=0).stop 2024-09-10T10:46:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 shutdown_connections 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f50d80a2b80 0x7f50d80a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f0106f10 0x7f50f019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f0102790 0x7f50f01988c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 --2- 172.21.15.143:0/1391236104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f0101de0 0x7f50f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 >> 172.21.15.143:0/1391236104 conn(0x7f50f00fd8c0 msgr2=0x7f50f00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 shutdown_connections 2024-09-10T10:46:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:46.750+0000 7f50f5795640 1 -- 172.21.15.143:0/1391236104 wait complete. 2024-09-10T10:46:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:46 smithi178 ceph-mon[34813]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:46:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:46 smithi144 ceph-mon[34230]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:46:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:46 smithi143 ceph-mon[31057]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:46:47.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:46:47.930 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:48.132 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T10:46:48.238 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T10:46:48.277 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:46:48.279 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:46:48.279 INFO:tasks.workunit.client.0.smithi143.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T10:46:48.318 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:46:48.321 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:46:48.321 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T10:46:48.337 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-10T10:46:48.338 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-10T10:46:48.338 DEBUG:teuthology.orchestra.run.smithi143:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 -- 172.21.15.143:0/3823637104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44101420 msgr2=0x7fee4410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 --2- 172.21.15.143:0/3823637104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44101420 0x7fee4410d950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009cd0 tx=0x7fee3c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 -- 172.21.15.143:0/3823637104 shutdown_connections 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 --2- 172.21.15.143:0/3823637104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44101420 0x7fee4410d950 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 --2- 172.21.15.143:0/3823637104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee44100a80 0x7fee44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 --2- 172.21.15.143:0/3823637104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44106a80 0x7fee44106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 -- 172.21.15.143:0/3823637104 >> 172.21.15.143:0/3823637104 conn(0x7fee440fc7a0 msgr2=0x7fee440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 -- 172.21.15.143:0/3823637104 shutdown_connections 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 -- 172.21.15.143:0/3823637104 wait complete. 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.461+0000 7fee4c158640 1 Processor -- start 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 -- start start 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44100a80 0x7fee44199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee44101420 0x7fee44199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 0x7fee441a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fee44110d50 con 0x7fee44106a80 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee44110bd0 con 0x7fee44100a80 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4c158640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee44110ed0 con 0x7fee44101420 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 0x7fee441a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 0x7fee441a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34776/0 (socket says 172.21.15.143:34776) 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 -- 172.21.15.143:0/743736438 learned_addr learned my addr 172.21.15.143:0/743736438 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee496f3640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee44101420 0x7fee44199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee49ef4640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44100a80 0x7fee44199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 -- 172.21.15.143:0/743736438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee44101420 msgr2=0x7fee44199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee44101420 0x7fee44199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 -- 172.21.15.143:0/743736438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44100a80 msgr2=0x7fee44199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44100a80 0x7fee44199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 -- 172.21.15.143:0/743736438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee441a4800 con 0x7fee44106a80 2024-09-10T10:46:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee49ef4640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44100a80 0x7fee44199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.462+0000 7fee4a6f5640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 0x7fee441a42c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005cc0 tx=0x7fee3c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.463+0000 7fee3affd640 1 -- 172.21.15.143:0/743736438 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c004240 con 0x7fee44106a80 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.463+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee441a49d0 con 0x7fee44106a80 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.463+0000 7fee3affd640 1 -- 172.21.15.143:0/743736438 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee3c0305a0 con 0x7fee44106a80 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.463+0000 7fee3affd640 1 -- 172.21.15.143:0/743736438 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0466c0 con 0x7fee44106a80 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.463+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee441a4e80 con 0x7fee44106a80 2024-09-10T10:46:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.465+0000 7fee3affd640 1 -- 172.21.15.143:0/743736438 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fee3c030ac0 con 0x7fee44106a80 2024-09-10T10:46:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.465+0000 7fee4e531740 1 Processor -- start 2024-09-10T10:46:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.465+0000 7fee4e531740 1 -- start start 2024-09-10T10:46:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4e531740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ec5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4e531740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec55d0 0x560eb9ec5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4e531740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ed0370 0x560eb9ed2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4e531740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560eb9e2b2f0 con 0x560eb9ec6b40 2024-09-10T10:46:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee3affd640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee300a2b80 0x7fee300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4e531740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560eb9dd2cf0 con 0x560eb9ed0370 2024-09-10T10:46:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4e531740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560eb9e272b0 con 0x560eb9ec55d0 2024-09-10T10:46:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ec5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ec5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34782/0 (socket says 172.21.15.143:34782) 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 -- 172.21.15.143:0/2358944924 learned_addr learned my addr 172.21.15.143:0/2358944924 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4a6f5640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee300a2b80 0x7fee300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee49ef4640 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec55d0 0x560eb9ec5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ed0370 0x560eb9ed2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee3affd640 1 -- 172.21.15.143:0/743736438 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fee3c02e030 con 0x7fee44106a80 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 -- 172.21.15.143:0/2358944924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec55d0 msgr2=0x560eb9ec5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec55d0 0x560eb9ec5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 -- 172.21.15.143:0/2358944924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ed0370 msgr2=0x560eb9ed2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ed0370 0x560eb9ed2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 -- 172.21.15.143:0/2358944924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb9db73a0 con 0x560eb9ec6b40 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee496f3640 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ec5090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fee40006aa0 tx=0x7fee400095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e8c719492f152a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.466+0000 7fee4a6f5640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee300a2b80 0x7fee300a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0b92d0 tx=0x7fee3c0b91e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee2ffff640 1 -- 172.21.15.143:0/2358944924 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40017610 con 0x560eb9ec6b40 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee2ffff640 1 -- 172.21.15.143:0/2358944924 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee4001c070 con 0x560eb9ec6b40 2024-09-10T10:46:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee2ffff640 1 -- 172.21.15.143:0/2358944924 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400158a0 con 0x560eb9ec6b40 2024-09-10T10:46:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 -- 172.21.15.143:0/2358944924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 msgr2=0x560eb9ec5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ec5090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fee40006aa0 tx=0x7fee400095d0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 -- 172.21.15.143:0/2358944924 shutdown_connections 2024-09-10T10:46:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ed0370 0x560eb9ed2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec55d0 0x560eb9ec5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 --2- 172.21.15.143:0/2358944924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ec5090 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 -- 172.21.15.143:0/2358944924 >> 172.21.15.143:0/2358944924 conn(0x560eb9e8e310 msgr2=0x560eb9ec1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 -- 172.21.15.143:0/2358944924 shutdown_connections 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.467+0000 7fee4e531740 1 -- 172.21.15.143:0/2358944924 wait complete. 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 -- start start 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 Processor -- start 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ecfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9eccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ecd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560eb9ed3010 con 0x560eb9ed0370 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560eb9de6060 con 0x560eb9ec55d0 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560eb9ed3190 con 0x560eb9ec6b40 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee4e531740 10 client.0 _ll_register_callbacks cb 0x560eb9b398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee496f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ecd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee496f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ecd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34792/0 (socket says 172.21.15.143:34792) 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.468+0000 7fee496f3640 1 -- 172.21.15.143:0/2255117506 learned_addr learned my addr 172.21.15.143:0/2255117506 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee49ef4640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9eccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ecfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 -- 172.21.15.143:0/2255117506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 msgr2=0x560eb9eccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9eccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 -- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 msgr2=0x560eb9ecfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ecfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 -- 172.21.15.143:0/2255117506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb9b9c3c0 con 0x560eb9ed0370 2024-09-10T10:46:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee496f3640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ecd480 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fee4000a010 tx=0x7fee40015580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a422d8e94a79f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee2d7fa640 1 -- 172.21.15.143:0/2255117506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40015d50 con 0x560eb9ed0370 2024-09-10T10:46:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee2d7fa640 1 -- 172.21.15.143:0/2255117506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fee40017410 con 0x560eb9ed0370 2024-09-10T10:46:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4001c070 con 0x560eb9ed0370 2024-09-10T10:46:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee2d7fa640 1 -- 172.21.15.143:0/2255117506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee40017610 con 0x560eb9ed0370 2024-09-10T10:46:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee4e531740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eb9edea30 con 0x560eb9ed0370 2024-09-10T10:46:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee2d7fa640 1 -- 172.21.15.143:0/2255117506 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee4001c070 con 0x560eb9ed0370 2024-09-10T10:46:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.469+0000 7fee2d7fa640 1 -- 172.21.15.143:0/2255117506 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fee40022900 con 0x560eb9ed0370 2024-09-10T10:46:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee4e531740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee4e531740 10 client.15948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee4e531740 20 client.15948 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee4e531740 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9edfde0 0x560eb9ee21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee4e531740 4 client.15948 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eb9e38860 con 0x560eb9edfde0 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee49ef4640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9edfde0 0x560eb9ee21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee2d7fa640 10 client.15948 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.470+0000 7fee49ef4640 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9edfde0 0x560eb9ee21d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e56eaee7f58b7af server_cookie=bdde64872fbd359e in_seq=0 out_seq=0 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee2d7fa640 1 -- 172.21.15.143:0/2255117506 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560eb9e38860 con 0x560eb9edfde0 2024-09-10T10:46:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee2d7fa640 10 client.15948 handle_command_reply: tid=0 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 client.15948 shutdown 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 msgr2=0x560eb9ecd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ecd480 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fee4000a010 tx=0x7fee40015580 comp rx=0 tx=0).stop 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 shutdown_connections 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ecd480 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9eccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9edfde0 0x560eb9ee21d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 --2- 172.21.15.143:0/2255117506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ecfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 >> 172.21.15.143:0/2255117506 conn(0x560eb9e8e310 msgr2=0x560eb9ec7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee2d7fa640 0 client.15948 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 shutdown_connections 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 1 -- 172.21.15.143:0/2255117506 wait complete. 2024-09-10T10:46:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.471+0000 7fee4e531740 20 client.15948 trim_cache size 0 max 16384 2024-09-10T10:46:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 Processor -- start 2024-09-10T10:46:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 -- start start 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9edc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ee77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ed0370 0x560eb9ee7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fee441aeae0 con 0x560eb9ec6b40 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b0620 con 0x560eb9ec55d0 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee4e531740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee441af1b0 con 0x560eb9ed0370 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.484+0000 7fee49ef4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ee77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ee77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34804/0 (socket says 172.21.15.143:34804) 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 -- 172.21.15.143:0/3477000705 learned_addr learned my addr 172.21.15.143:0/3477000705 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee496f3640 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9edc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee496f3640 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ed0370 0x560eb9ee7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 -- 172.21.15.143:0/3477000705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ed0370 msgr2=0x560eb9ee7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ed0370 0x560eb9ee7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 -- 172.21.15.143:0/3477000705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 msgr2=0x560eb9edc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9edc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 -- 172.21.15.143:0/3477000705 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb9ec8120 con 0x560eb9ec6b40 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee496f3640 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9edc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee49ef4640 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ee77d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x560eb9ecc7d0 tx=0x7fee3400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393ad81f279b559c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee2f7fe640 1 -- 172.21.15.143:0/3477000705 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34021020 con 0x560eb9ec6b40 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee2f7fe640 1 -- 172.21.15.143:0/3477000705 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee3401cad0 con 0x560eb9ec6b40 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.485+0000 7fee2f7fe640 1 -- 172.21.15.143:0/3477000705 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3401bd00 con 0x560eb9ec6b40 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 -- 172.21.15.143:0/3477000705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 msgr2=0x560eb9ee77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ee77d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x560eb9ecc7d0 tx=0x7fee3400f7f0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 -- 172.21.15.143:0/3477000705 shutdown_connections 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ed0370 0x560eb9ee7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ec6b40 0x560eb9ee77d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 --2- 172.21.15.143:0/3477000705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9edc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 -- 172.21.15.143:0/3477000705 >> 172.21.15.143:0/3477000705 conn(0x560eb9e8e310 msgr2=0x560eb9ecc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 -- 172.21.15.143:0/3477000705 shutdown_connections 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 -- 172.21.15.143:0/3477000705 wait complete. 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 -- start start 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.486+0000 7fee4e531740 1 Processor -- start 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ed8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9ed3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ed4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fee441aff50 con 0x560eb9ed0370 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee441af880 con 0x560eb9ec55d0 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee441acac0 con 0x560eb9ec6b40 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 10 client.0 _ll_register_callbacks cb 0x7fee3400b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ed4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ed4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34814/0 (socket says 172.21.15.143:34814) 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 -- 172.21.15.143:0/3617748500 learned_addr learned my addr 172.21.15.143:0/3617748500 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee49ef4640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9ed3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ed8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 -- 172.21.15.143:0/3617748500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 msgr2=0x560eb9ed3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9ed3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 -- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 msgr2=0x560eb9ed8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ed8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 -- 172.21.15.143:0/3617748500 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb9ed4b90 con 0x560eb9ed0370 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee496f3640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ed4360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee40023fd0 tx=0x7fee4002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3058e73d783f27a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee2d7fa640 1 -- 172.21.15.143:0/3617748500 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40015d50 con 0x560eb9ed0370 2024-09-10T10:46:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee2d7fa640 1 -- 172.21.15.143:0/3617748500 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fee400177b0 con 0x560eb9ed0370 2024-09-10T10:46:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.487+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3401cad0 con 0x560eb9ed0370 2024-09-10T10:46:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee2d7fa640 1 -- 172.21.15.143:0/3617748500 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee400179b0 con 0x560eb9ed0370 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eb9f4eae0 con 0x560eb9ed0370 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee2d7fa640 1 -- 172.21.15.143:0/3617748500 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee3401cad0 con 0x560eb9ed0370 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee2d7fa640 1 -- 172.21.15.143:0/3617748500 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fee40022410 con 0x560eb9ed0370 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 10 client.15954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 20 client.15954 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9f4f1d0 0x560eb9f515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 4 client.15954 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560eb9e32d70 con 0x560eb9f4f1d0 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.488+0000 7fee49ef4640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9f4f1d0 0x560eb9f515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee49ef4640 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9f4f1d0 0x560eb9f515f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0e805acfdfc815f server_cookie=96aa2c14a1c1a183 in_seq=0 out_seq=0 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee2d7fa640 10 client.15954 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee2d7fa640 1 -- 172.21.15.143:0/3617748500 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560eb9e32d70 con 0x560eb9f4f1d0 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee2d7fa640 10 client.15954 handle_command_reply: tid=0 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee4e531740 1 client.15954 shutdown 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 msgr2=0x560eb9ed4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.489+0000 7fee4e531740 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ed4360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee40023fd0 tx=0x7fee4002e040 comp rx=0 tx=0).stop 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 shutdown_connections 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560eb9ed0370 0x560eb9ed4360 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560eb9ec6b40 0x560eb9ed3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x560eb9f4f1d0 0x560eb9f515f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 --2- 172.21.15.143:0/3617748500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560eb9ec55d0 0x560eb9ed8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 >> 172.21.15.143:0/3617748500 conn(0x560eb9e8e310 msgr2=0x560eb9ece7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee2d7fa640 0 client.15954 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:46:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 shutdown_connections 2024-09-10T10:46:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 1 -- 172.21.15.143:0/3617748500 wait complete. 2024-09-10T10:46:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4e531740 20 client.15954 trim_cache size 0 max 16384 2024-09-10T10:46:48.512 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee300a2b80 msgr2=0x7fee300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee300a2b80 0x7fee300a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0b92d0 tx=0x7fee3c0b91e0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 msgr2=0x7fee441a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 0x7fee441a42c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005cc0 tx=0x7fee3c003ba0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 shutdown_connections 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fee300a2b80 0x7fee300a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee44106a80 0x7fee441a42c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee44101420 0x7fee44199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 --2- 172.21.15.143:0/743736438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee44100a80 0x7fee44199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 >> 172.21.15.143:0/743736438 conn(0x7fee440fc7a0 msgr2=0x7fee440fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 shutdown_connections 2024-09-10T10:46:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:48.490+0000 7fee4c158640 1 -- 172.21.15.143:0/743736438 wait complete. 2024-09-10T10:46:48.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:48 smithi143 ceph-mon[31057]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:48 smithi178 ceph-mon[34813]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:48.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:48 smithi144 ceph-mon[34230]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:49.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:49.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 -- 172.21.15.143:0/1531966443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec100a80 msgr2=0x7fa6ec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/1531966443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec100a80 0x7fa6ec100ee0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009930 tx=0x7fa6d402a420 comp rx=0 tx=0).stop 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 -- 172.21.15.143:0/1531966443 shutdown_connections 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/1531966443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec101420 0x7fa6ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/1531966443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec100a80 0x7fa6ec100ee0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/1531966443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec106a80 0x7fa6ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 -- 172.21.15.143:0/1531966443 >> 172.21.15.143:0/1531966443 conn(0x7fa6ec0fc7a0 msgr2=0x7fa6ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 -- 172.21.15.143:0/1531966443 shutdown_connections 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.954+0000 7fa6f15f6640 1 -- 172.21.15.143:0/1531966443 wait complete. 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 Processor -- start 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 -- start start 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec100a80 0x7fa6ec1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 0x7fa6ec1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec106a80 0x7fa6ec19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec110d40 con 0x7fa6ec101420 2024-09-10T10:46:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec110bc0 con 0x7fa6ec106a80 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6f15f6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec110ec0 con 0x7fa6ec100a80 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 0x7fa6ec1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 0x7fa6ec1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34836/0 (socket says 172.21.15.143:34836) 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6e27fc640 1 -- 172.21.15.143:0/2098021514 learned_addr learned my addr 172.21.15.143:0/2098021514 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec100a80 0x7fa6ec1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec106a80 0x7fa6ec19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6e27fc640 1 -- 172.21.15.143:0/2098021514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec100a80 msgr2=0x7fa6ec1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.955+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec100a80 0x7fa6ec1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e27fc640 1 -- 172.21.15.143:0/2098021514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec106a80 msgr2=0x7fa6ec19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec106a80 0x7fa6ec19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e27fc640 1 -- 172.21.15.143:0/2098021514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec19d360 con 0x7fa6ec101420 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 0x7fa6ec1988f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4031440 tx=0x7fa6d4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e8ff9640 1 -- 172.21.15.143:0/2098021514 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d403f070 con 0x7fa6ec101420 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e8ff9640 1 -- 172.21.15.143:0/2098021514 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6d4031d60 con 0x7fa6ec101420 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec19d5f0 con 0x7fa6ec101420 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6e8ff9640 1 -- 172.21.15.143:0/2098021514 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4035030 con 0x7fa6ec101420 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.956+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1a4f30 con 0x7fa6ec101420 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.958+0000 7fa6e8ff9640 1 -- 172.21.15.143:0/2098021514 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa6d4042070 con 0x7fa6ec101420 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.958+0000 7fa6f39cf740 1 Processor -- start 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.958+0000 7fa6f39cf740 1 -- start start 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc835090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc835a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc846b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616cc79b2f0 con 0x5616cc8355d0 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616cc742cf0 con 0x5616cc836b40 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5616cc7972b0 con 0x5616cc844780 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6e8ff9640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6d80a2b80 0x7fa6d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc835a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc835a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34848/0 (socket says 172.21.15.143:34848) 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 -- 172.21.15.143:0/2011514993 learned_addr learned my addr 172.21.15.143:0/2011514993 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6d80a2b80 0x7fa6d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6e8ff9640 1 -- 172.21.15.143:0/2098021514 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6d40efc00 con 0x7fa6ec101420 2024-09-10T10:46:49.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc835090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc846b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 -- 172.21.15.143:0/2011514993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 msgr2=0x5616cc846b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc846b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 -- 172.21.15.143:0/2011514993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 msgr2=0x5616cc835090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc835090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6eaffd640 1 -- 172.21.15.143:0/2011514993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616cc7273a0 con 0x5616cc8355d0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.959+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc846b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.960+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc835a10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00ab80 tx=0x7fa6dc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42c3ee718fe1ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.960+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6d80a2b80 0x7fa6d80a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400a780 tx=0x7fa6e4005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.960+0000 7fa6e17fa640 1 -- 172.21.15.143:0/2011514993 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc005220 con 0x5616cc8355d0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.960+0000 7fa6e17fa640 1 -- 172.21.15.143:0/2011514993 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6dc004c50 con 0x5616cc8355d0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.960+0000 7fa6e17fa640 1 -- 172.21.15.143:0/2011514993 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc018730 con 0x5616cc8355d0 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- 172.21.15.143:0/2011514993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 msgr2=0x5616cc835a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc835a10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00ab80 tx=0x7fa6dc00f790 comp rx=0 tx=0).stop 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- 172.21.15.143:0/2011514993 shutdown_connections 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc846b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc835a10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/2011514993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc835090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- 172.21.15.143:0/2011514993 >> 172.21.15.143:0/2011514993 conn(0x5616cc7fe310 msgr2=0x5616cc831c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- 172.21.15.143:0/2011514993 shutdown_connections 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- 172.21.15.143:0/2011514993 wait complete. 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- start start 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 Processor -- start 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc844110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 0x5616cc8413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc841940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616cc847480 con 0x5616cc836b40 2024-09-10T10:46:50.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616cc756060 con 0x5616cc8355d0 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5616cc847600 con 0x5616cc844780 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.961+0000 7fa6f39cf740 10 client.0 _ll_register_callbacks cb 0x5616cc4a98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 0x5616cc8413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 0x5616cc8413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34858/0 (socket says 172.21.15.143:34858) 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 -- 172.21.15.143:0/3234317688 learned_addr learned my addr 172.21.15.143:0/3234317688 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc841940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc844110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 -- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 msgr2=0x5616cc841940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc841940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 -- 172.21.15.143:0/3234317688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 msgr2=0x5616cc844110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc844110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 -- 172.21.15.143:0/3234317688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616cc50c3c0 con 0x5616cc836b40 2024-09-10T10:46:50.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc844110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 0x5616cc8413c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40ed710 tx=0x7fa6d40bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646f75529142d09d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6d27fc640 1 -- 172.21.15.143:0/3234317688 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4049070 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6d27fc640 1 -- 172.21.15.143:0/3234317688 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6d40c0120 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6dc004c50 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6d27fc640 1 -- 172.21.15.143:0/3234317688 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6d40d50c0 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6f39cf740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.962+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616cc852e20 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6d27fc640 1 -- 172.21.15.143:0/3234317688 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6dc004c50 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6d27fc640 1 -- 172.21.15.143:0/3234317688 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa6d403b680 con 0x5616cc836b40 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6f39cf740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6f39cf740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6f39cf740 20 client.15966 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8542d0 0x5616cc8566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6f39cf740 4 client.15966 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616cc7a8860 con 0x5616cc8542d0 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8542d0 0x5616cc8566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8542d0 0x5616cc8566c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4057080d2406fb server_cookie=5839b5d788bd37e3 in_seq=0 out_seq=0 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.963+0000 7fa6d27fc640 10 client.15966 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6d27fc640 1 -- 172.21.15.143:0/3234317688 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616cc7a8860 con 0x5616cc8542d0 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6d27fc640 10 client.15966 handle_command_reply: tid=0 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 client.15966 shutdown 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 msgr2=0x5616cc8413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 0x5616cc8413c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40ed710 tx=0x7fa6d40bc040 comp rx=0 tx=0).stop 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 shutdown_connections 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8542d0 0x5616cc8566c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc841940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc836b40 0x5616cc8413c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/3234317688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc844110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6d27fc640 0 client.15966 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.964+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 >> 172.21.15.143:0/3234317688 conn(0x5616cc7fe310 msgr2=0x5616cc837930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.965+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 shutdown_connections 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.965+0000 7fa6f39cf740 1 -- 172.21.15.143:0/3234317688 wait complete. 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.965+0000 7fa6f39cf740 20 client.15966 trim_cache size 0 max 16384 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 Processor -- start 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 -- start start 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc841390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc850720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc8316e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ac880 con 0x5616cc8355d0 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ae3c0 con 0x5616cc836b40 2024-09-10T10:46:50.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1acf50 con 0x5616cc844780 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc841390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc841390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34870/0 (socket says 172.21.15.143:34870) 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 -- 172.21.15.143:0/881920873 learned_addr learned my addr 172.21.15.143:0/881920873 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc8316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc850720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 -- 172.21.15.143:0/881920873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 msgr2=0x5616cc8316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc8316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 -- 172.21.15.143:0/881920873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 msgr2=0x5616cc850720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.978+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc850720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6eaffd640 1 -- 172.21.15.143:0/881920873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616cc836760 con 0x5616cc8355d0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc841390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00aad0 tx=0x7fa6dc00a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9e9debcea9cc4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6e0ff9640 1 -- 172.21.15.143:0/881920873 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc034070 con 0x5616cc8355d0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6e0ff9640 1 -- 172.21.15.143:0/881920873 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6dc005470 con 0x5616cc8355d0 2024-09-10T10:46:50.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6e0ff9640 1 -- 172.21.15.143:0/881920873 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc004cc0 con 0x5616cc8355d0 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6f39cf740 1 -- 172.21.15.143:0/881920873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 msgr2=0x5616cc841390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.979+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc841390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00aad0 tx=0x7fa6dc00a5e0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- 172.21.15.143:0/881920873 shutdown_connections 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc844780 0x5616cc8316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc836b40 0x5616cc850720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/881920873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc8355d0 0x5616cc841390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- 172.21.15.143:0/881920873 >> 172.21.15.143:0/881920873 conn(0x5616cc7fe310 msgr2=0x5616cc840c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- 172.21.15.143:0/881920873 shutdown_connections 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- 172.21.15.143:0/881920873 wait complete. 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- start start 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 Processor -- start 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc84c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc836b40 0x5616cc84cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 0x5616cc847df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1adcf0 con 0x5616cc844780 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ad620 con 0x5616cc8355d0 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1aa880 con 0x5616cc836b40 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6f39cf740 10 client.0 _ll_register_callbacks cb 0x7fa6d4005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 0x5616cc847df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6eb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 0x5616cc847df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34880/0 (socket says 172.21.15.143:34880) 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6eb7fe640 1 -- 172.21.15.143:0/4042593604 learned_addr learned my addr 172.21.15.143:0/4042593604 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.980+0000 7fa6e27fc640 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc836b40 0x5616cc84cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6eb7fe640 1 -- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc836b40 msgr2=0x5616cc84cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc836b40 0x5616cc84cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6eb7fe640 1 -- 172.21.15.143:0/4042593604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 msgr2=0x5616cc84c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc84c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6eb7fe640 1 -- 172.21.15.143:0/4042593604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616cc8485d0 con 0x5616cc844780 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6eb7fe640 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 0x5616cc847df0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400fd80 tx=0x7fa6e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de980a17a520a7c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6d27fc640 1 -- 172.21.15.143:0/4042593604 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e402e060 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6d27fc640 1 -- 172.21.15.143:0/4042593604 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6e400fdb0 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6dc005470 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6d27fc640 1 -- 172.21.15.143:0/4042593604 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6e4035050 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616cc8bea60 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6d27fc640 1 -- 172.21.15.143:0/4042593604 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6dc005470 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6d27fc640 1 -- 172.21.15.143:0/4042593604 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa6e401d050 con 0x5616cc844780 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 20 client.15972 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8bf1e0 0x5616cc8c1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 4 client.15972 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:46:50.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.981+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616cc7a2d70 con 0x5616cc8bf1e0 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8bf1e0 0x5616cc8c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6eaffd640 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8bf1e0 0x5616cc8c1620 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b829962283dc9c9 server_cookie=203da9655f824a90 in_seq=0 out_seq=0 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6d27fc640 10 client.15972 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6d27fc640 1 -- 172.21.15.143:0/4042593604 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616cc7a2d70 con 0x5616cc8bf1e0 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6d27fc640 10 client.15972 handle_command_reply: tid=0 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 client.15972 shutdown 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 msgr2=0x5616cc847df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 0x5616cc847df0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400fd80 tx=0x7fa6e4004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 shutdown_connections 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616cc844780 0x5616cc847df0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616cc8bf1e0 0x5616cc8c1620 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616cc836b40 0x5616cc84cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6d27fc640 0 client.15972 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:50.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 --2- 172.21.15.143:0/4042593604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616cc8355d0 0x5616cc84c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.982+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 >> 172.21.15.143:0/4042593604 conn(0x5616cc7fe310 msgr2=0x5616cc831640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 shutdown_connections 2024-09-10T10:46:50.010 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:46:50.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f39cf740 1 -- 172.21.15.143:0/4042593604 wait complete. 2024-09-10T10:46:50.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f39cf740 20 client.15972 trim_cache size 0 max 16384 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6d80a2b80 msgr2=0x7fa6d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6d80a2b80 0x7fa6d80a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400a780 tx=0x7fa6e4005f60 comp rx=0 tx=0).stop 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 msgr2=0x7fa6ec1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 0x7fa6ec1988f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4031440 tx=0x7fa6d4003d50 comp rx=0 tx=0).stop 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 shutdown_connections 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6d80a2b80 0x7fa6d80a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec106a80 0x7fa6ec19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6ec101420 0x7fa6ec1988f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 --2- 172.21.15.143:0/2098021514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ec100a80 0x7fa6ec1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 >> 172.21.15.143:0/2098021514 conn(0x7fa6ec0fc7a0 msgr2=0x7fa6ec06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 shutdown_connections 2024-09-10T10:46:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:49.983+0000 7fa6f15f6640 1 -- 172.21.15.143:0/2098021514 wait complete. 2024-09-10T10:46:50.741 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:50 smithi143 ceph-mon[31057]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:46:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:50 smithi178 ceph-mon[34813]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:46:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:50 smithi144 ceph-mon[34230]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:46:51.778 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:46:51.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:46:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:52.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 -- 172.21.15.143:0/3259478083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8106a80 msgr2=0x7f19a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 --2- 172.21.15.143:0/3259478083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8106a80 0x7f19a8106e60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1998009930 tx=0x7f199802a420 comp rx=0 tx=0).stop 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 -- 172.21.15.143:0/3259478083 shutdown_connections 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 --2- 172.21.15.143:0/3259478083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19a8101420 0x7f19a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 --2- 172.21.15.143:0/3259478083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8100a80 0x7f19a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 --2- 172.21.15.143:0/3259478083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8106a80 0x7f19a8106e60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 -- 172.21.15.143:0/3259478083 >> 172.21.15.143:0/3259478083 conn(0x7f19a80fc7a0 msgr2=0x7f19a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 -- 172.21.15.143:0/3259478083 shutdown_connections 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 -- 172.21.15.143:0/3259478083 wait complete. 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.545+0000 7f19ae785640 1 Processor -- start 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 -- start start 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 0x7f19a81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8101420 0x7f19a8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19a8106a80 0x7f19a819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19a8110d50 con 0x7f19a8100a80 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19a8110bd0 con 0x7f19a8101420 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19ae785640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19a8110ed0 con 0x7f19a8106a80 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 0x7f19a81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 0x7f19a81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34892/0 (socket says 172.21.15.143:34892) 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 -- 172.21.15.143:0/3873259492 learned_addr learned my addr 172.21.15.143:0/3873259492 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 -- 172.21.15.143:0/3873259492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19a8106a80 msgr2=0x7f19a819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19a8106a80 0x7f19a819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 -- 172.21.15.143:0/3873259492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8101420 msgr2=0x7f19a8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a77fe640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8101420 0x7f19a8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8101420 0x7f19a8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.546+0000 7f19a7fff640 1 -- 172.21.15.143:0/3873259492 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a819d380 con 0x7f19a8100a80 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.547+0000 7f19a7fff640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 0x7f19a81983d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1998005940 tx=0x7f19980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:52.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.547+0000 7f19a57fa640 1 -- 172.21.15.143:0/3873259492 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199803f030 con 0x7f19a8100a80 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.547+0000 7f19a57fa640 1 -- 172.21.15.143:0/3873259492 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1998003cb0 con 0x7f19a8100a80 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.547+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a819d610 con 0x7f19a8100a80 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.547+0000 7f19a57fa640 1 -- 172.21.15.143:0/3873259492 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998035070 con 0x7f19a8100a80 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.547+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a81a4f50 con 0x7f19a8100a80 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.549+0000 7f19a57fa640 1 -- 172.21.15.143:0/3873259492 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1998042030 con 0x7f19a8100a80 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.549+0000 7f19b0b5e740 1 Processor -- start 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.549+0000 7f19b0b5e740 1 -- start start 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.549+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f0f70 0x556e111f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.549+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f1890 0x556e111fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.549+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 0x556e111fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556e11152280 con 0x556e111fc150 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e110f9b10 con 0x556e111f1890 2024-09-10T10:46:52.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556e1114e240 con 0x556e111f0f70 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a57fa640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f19900a2c00 0x7f19900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 0x556e111fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19acd22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f0f70 0x556e111f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19acd22640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f0f70 0x556e111f1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:50744/0 (socket says 172.21.15.143:50744) 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 0x556e111fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34900/0 (socket says 172.21.15.143:34900) 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19acd22640 1 -- 172.21.15.143:0/297220364 learned_addr learned my addr 172.21.15.143:0/297220364 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a7fff640 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f1890 0x556e111fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19acd22640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f19900a2c00 0x7f19900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a57fa640 1 -- 172.21.15.143:0/3873259492 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f19980ef920 con 0x7f19a8100a80 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 -- 172.21.15.143:0/297220364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f0f70 msgr2=0x556e111f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f0f70 0x556e111f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 -- 172.21.15.143:0/297220364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f1890 msgr2=0x556e111fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f1890 0x556e111fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 -- 172.21.15.143:0/297220364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e111061f0 con 0x556e111fc150 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19a77fe640 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 0x556e111fe540 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f19a8105270 tx=0x7f199c00acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9558860cf831cb7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f19acd22640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f19900a2c00 0x7f19900a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f19a0003b10 tx=0x7f19a0005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.550+0000 7f198e7fc640 1 -- 172.21.15.143:0/297220364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c015070 con 0x556e111fc150 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f198e7fc640 1 -- 172.21.15.143:0/297220364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f199c0113f0 con 0x556e111fc150 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f198e7fc640 1 -- 172.21.15.143:0/297220364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c018560 con 0x556e111fc150 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 -- 172.21.15.143:0/297220364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 msgr2=0x556e111fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 0x556e111fe540 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f19a8105270 tx=0x7f199c00acc0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 -- 172.21.15.143:0/297220364 shutdown_connections 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111fc150 0x556e111fe540 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f1890 0x556e111fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/297220364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f0f70 0x556e111f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 -- 172.21.15.143:0/297220364 >> 172.21.15.143:0/297220364 conn(0x556e111ed500 msgr2=0x556e111ed8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 -- 172.21.15.143:0/297220364 shutdown_connections 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.551+0000 7f19b0b5e740 1 -- 172.21.15.143:0/297220364 wait complete. 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 -- start start 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 Processor -- start 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 0x556e111f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f1890 0x556e111f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111fc150 0x556e111f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556e111fed20 con 0x556e111f0f70 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e1110cf10 con 0x556e111fc150 2024-09-10T10:46:52.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556e111feea0 con 0x556e111f1890 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19b0b5e740 10 client.0 _ll_register_callbacks cb 0x556e110d5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 0x556e111f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 0x556e111f8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34908/0 (socket says 172.21.15.143:34908) 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19a7fff640 1 -- 172.21.15.143:0/2941007048 learned_addr learned my addr 172.21.15.143:0/2941007048 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.552+0000 7f19a77fe640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f1890 0x556e111f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19acd22640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111fc150 0x556e111f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19a7fff640 1 -- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f1890 msgr2=0x556e111f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19a7fff640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f1890 0x556e111f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19a7fff640 1 -- 172.21.15.143:0/2941007048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111fc150 msgr2=0x556e111f9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19a7fff640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111fc150 0x556e111f9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19a7fff640 1 -- 172.21.15.143:0/2941007048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e10ec36a0 con 0x556e111f0f70 2024-09-10T10:46:52.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19a7fff640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 0x556e111f8f90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f19980b8280 tx=0x7f199802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2a79d87ea3d35c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19877fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19877fe640 1 -- 172.21.15.143:0/2941007048 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998049070 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19877fe640 1 -- 172.21.15.143:0/2941007048 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f199803b460 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199c0113f0 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19877fe640 1 -- 172.21.15.143:0/2941007048 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19980ea080 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19b0b5e740 10 client.15984 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e1120a890 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19877fe640 1 -- 172.21.15.143:0/2941007048 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f199c0113f0 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19877fe640 1 -- 172.21.15.143:0/2941007048 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f199803bc60 con 0x556e111f0f70 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19b0b5e740 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.553+0000 7f19b0b5e740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19b0b5e740 20 client.15984 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:52.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e1120bbe0 0x556e1120dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19b0b5e740 4 client.15984 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1115f7f0 con 0x556e1120bbe0 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19a77fe640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e1120bbe0 0x556e1120dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19877fe640 10 client.15984 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.554+0000 7f19a77fe640 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e1120bbe0 0x556e1120dff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b0cef5f3066e4f server_cookie=725ba230f27db2b6 in_seq=0 out_seq=0 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19877fe640 1 -- 172.21.15.143:0/2941007048 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e1115f7f0 con 0x556e1120bbe0 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19877fe640 10 client.15984 handle_command_reply: tid=0 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 client.15984 shutdown 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 msgr2=0x556e111f8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 0x556e111f8f90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f19980b8280 tx=0x7f199802a420 comp rx=0 tx=0).stop 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 shutdown_connections 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111fc150 0x556e111f9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e1120bbe0 0x556e1120dff0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111f1890 0x556e111f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/2941007048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f0f70 0x556e111f8f90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19877fe640 0 client.15984 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 >> 172.21.15.143:0/2941007048 conn(0x556e111ed500 msgr2=0x556e111e8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 shutdown_connections 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 1 -- 172.21.15.143:0/2941007048 wait complete. 2024-09-10T10:46:52.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.555+0000 7f19b0b5e740 20 client.15984 trim_cache size 0 max 16384 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.572+0000 7f19b0b5e740 1 Processor -- start 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 -- start start 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e11213860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11213da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81ac940 con 0x556e111f1890 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81ae480 con 0x556e111f0f70 2024-09-10T10:46:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81ad010 con 0x556e111fc150 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e11213860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e11213860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34922/0 (socket says 172.21.15.143:34922) 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 -- 172.21.15.143:0/3753422857 learned_addr learned my addr 172.21.15.143:0/3753422857 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 -- 172.21.15.143:0/3753422857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 msgr2=0x556e11213da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11213da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a7fff640 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 -- 172.21.15.143:0/3753422857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 msgr2=0x556e111f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 -- 172.21.15.143:0/3753422857 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e111e4830 con 0x556e111f1890 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f19a77fe640 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e11213860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c520 tx=0x7f199c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f63f78d9acc36d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.573+0000 7f198dffb640 1 -- 172.21.15.143:0/3753422857 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c015040 con 0x556e111f1890 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f198dffb640 1 -- 172.21.15.143:0/3753422857 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f199c00ce20 con 0x556e111f1890 2024-09-10T10:46:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f198dffb640 1 -- 172.21.15.143:0/3753422857 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c01a7d0 con 0x556e111f1890 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 -- 172.21.15.143:0/3753422857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 msgr2=0x556e11213860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e11213860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c520 tx=0x7f199c0047d0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 -- 172.21.15.143:0/3753422857 shutdown_connections 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11213da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e11213860 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/3753422857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 -- 172.21.15.143:0/3753422857 >> 172.21.15.143:0/3753422857 conn(0x556e111ed500 msgr2=0x556e111f29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 -- 172.21.15.143:0/3753422857 shutdown_connections 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.574+0000 7f19b0b5e740 1 -- 172.21.15.143:0/3753422857 wait complete. 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 -- start start 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 Processor -- start 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e112004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11200a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81addb0 con 0x556e111f1890 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81ad6e0 con 0x556e111f0f70 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81aa920 con 0x556e111fc150 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19b0b5e740 10 client.0 _ll_register_callbacks cb 0x7f19980b8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e112004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19a77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e112004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34936/0 (socket says 172.21.15.143:34936) 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19a77fe640 1 -- 172.21.15.143:0/1999030609 learned_addr learned my addr 172.21.15.143:0/1999030609 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19acd22640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11200a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.575+0000 7f19a7fff640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19a77fe640 1 -- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 msgr2=0x556e11200a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19a77fe640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11200a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19a77fe640 1 -- 172.21.15.143:0/1999030609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 msgr2=0x556e111fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19a77fe640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19a77fe640 1 -- 172.21.15.143:0/1999030609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e11275990 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19a77fe640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e112004e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f199c011f20 tx=0x7f199c00a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea693e071d2d459b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19877fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19877fe640 1 -- 172.21.15.143:0/1999030609 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c02f070 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19877fe640 1 -- 172.21.15.143:0/1999030609 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f199c011600 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199c00ce20 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19877fe640 1 -- 172.21.15.143:0/1999030609 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f199c015050 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19b0b5e740 10 client.15990 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e11276c90 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19877fe640 1 -- 172.21.15.143:0/1999030609 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f199c00ce20 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19877fe640 1 -- 172.21.15.143:0/1999030609 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f199c025030 con 0x556e111f1890 2024-09-10T10:46:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19b0b5e740 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19b0b5e740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.576+0000 7f19b0b5e740 20 client.15990 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.577+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e11276f20 0x556e11279310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.577+0000 7f19b0b5e740 4 client.15990 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.577+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e11159d00 con 0x556e11276f20 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.577+0000 7f19a7fff640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e11276f20 0x556e11279310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.577+0000 7f19a7fff640 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e11276f20 0x556e11279310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f31e2ad8b8768f server_cookie=19e06abcd17dedd4 in_seq=0 out_seq=0 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.577+0000 7f19877fe640 10 client.15990 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19877fe640 1 -- 172.21.15.143:0/1999030609 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e11159d00 con 0x556e11276f20 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19877fe640 10 client.15990 handle_command_reply: tid=0 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 client.15990 shutdown 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 msgr2=0x556e112004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e112004e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f199c011f20 tx=0x7f199c00a580 comp rx=0 tx=0).stop 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 shutdown_connections 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x556e11276f20 0x556e11279310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e111fc150 0x556e11200a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e111f1890 0x556e112004e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 --2- 172.21.15.143:0/1999030609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e111f0f70 0x556e111fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 >> 172.21.15.143:0/1999030609 conn(0x556e111ed500 msgr2=0x556e111e8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.578+0000 7f19877fe640 0 client.15990 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 shutdown_connections 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19b0b5e740 1 -- 172.21.15.143:0/1999030609 wait complete. 2024-09-10T10:46:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19b0b5e740 20 client.15990 trim_cache size 0 max 16384 2024-09-10T10:46:52.593 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:52.593 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:46:52.593 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "0d37f11e-f839-4f8f-8443-659dc4bd52ec", 2024-09-10T10:46:52.593 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:46:52.593 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f19900a2c00 msgr2=0x7f19900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f19900a2c00 0x7f19900a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f19a0003b10 tx=0x7f19a0005e90 comp rx=0 tx=0).stop 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 msgr2=0x7f19a81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 0x7f19a81983d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1998005940 tx=0x7f19980042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 shutdown_connections 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f19900a2c00 0x7f19900a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19a8106a80 0x7f19a819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19a8101420 0x7f19a8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 --2- 172.21.15.143:0/3873259492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a8100a80 0x7f19a81983d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 >> 172.21.15.143:0/3873259492 conn(0x7f19a80fc7a0 msgr2=0x7f19a806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 shutdown_connections 2024-09-10T10:46:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:52.579+0000 7f19ae785640 1 -- 172.21.15.143:0/3873259492 wait complete. 2024-09-10T10:46:52.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:52 smithi143 ceph-mon[31057]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T10:46:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:52 smithi178 ceph-mon[34813]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T10:46:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:52 smithi144 ceph-mon[34230]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T10:46:53.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:46:53.527 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:46:53.913 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:53 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:46:53.913 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:53 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:53.913 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:53 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:46:53.914 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:53 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:53.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:53 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:46:53.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:53 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:53.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:53 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:46:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:53 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:53 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:46:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:53 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:53 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:46:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:53 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 -- 172.21.15.143:0/2791036632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440101420 msgr2=0x7f644010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 --2- 172.21.15.143:0/2791036632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440101420 0x7f644010d950 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6438009930 tx=0x7f643802a420 comp rx=0 tx=0).stop 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 -- 172.21.15.143:0/2791036632 shutdown_connections 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 --2- 172.21.15.143:0/2791036632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440101420 0x7f644010d950 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 --2- 172.21.15.143:0/2791036632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6440100a80 0x7f6440100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 --2- 172.21.15.143:0/2791036632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440106a80 0x7f6440106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.994+0000 7f64460af640 1 -- 172.21.15.143:0/2791036632 >> 172.21.15.143:0/2791036632 conn(0x7f64400fc7a0 msgr2=0x7f64400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.995+0000 7f64460af640 1 -- 172.21.15.143:0/2791036632 shutdown_connections 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.995+0000 7f64460af640 1 -- 172.21.15.143:0/2791036632 wait complete. 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.995+0000 7f64460af640 1 Processor -- start 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.995+0000 7f64460af640 1 -- start start 2024-09-10T10:46:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6440100a80 0x7f64401983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440101420 0x7f6440198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 0x7f644019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6440110d50 con 0x7f6440106a80 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6440110bd0 con 0x7f6440101420 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6440110ed0 con 0x7f6440100a80 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 0x7f644019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6440100a80 0x7f64401983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 0x7f644019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34952/0 (socket says 172.21.15.143:34952) 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 -- 172.21.15.143:0/1054017682 learned_addr learned my addr 172.21.15.143:0/1054017682 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643effd640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440101420 0x7f6440198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 -- 172.21.15.143:0/1054017682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6440100a80 msgr2=0x7f64401983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6440100a80 0x7f64401983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 -- 172.21.15.143:0/1054017682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440101420 msgr2=0x7f6440198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440101420 0x7f6440198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 -- 172.21.15.143:0/1054017682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019d380 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643ffff640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 0x7f644019cca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6438009900 tx=0x7f6438004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643cff9640 1 -- 172.21.15.143:0/1054017682 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643803f030 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643cff9640 1 -- 172.21.15.143:0/1054017682 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64380035f0 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f643cff9640 1 -- 172.21.15.143:0/1054017682 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438035070 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.996+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019d610 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.997+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64401a4f50 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f643cff9640 1 -- 172.21.15.143:0/1054017682 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6438003b10 con 0x7f6440106a80 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 Processor -- start 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 -- start start 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409da97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56409d9fd2e0 con 0x56409da974a0 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56409d9a4ce0 con 0x56409daa2680 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f6448488740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56409d9f92a0 con 0x56409da97dc0 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:53.999+0000 7f643cff9640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64280a2c00 0x7f64280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409da97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409da97880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34968/0 (socket says 172.21.15.143:34968) 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 -- 172.21.15.143:0/1066846161 learned_addr learned my addr 172.21.15.143:0/1066846161 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643ffff640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64280a2c00 0x7f64280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643cff9640 1 -- 172.21.15.143:0/1054017682 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64380efe30 con 0x7f6440106a80 2024-09-10T10:46:54.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 -- 172.21.15.143:0/1066846161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 msgr2=0x56409daa2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 -- 172.21.15.143:0/1066846161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 msgr2=0x56409daa4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 -- 172.21.15.143:0/1066846161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409d989440 con 0x56409da974a0 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643effd640 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409da97880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6434009fd0 tx=0x7f64340096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86fdf530fa0d15e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f643ffff640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64280a2c00 0x7f64280a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64380b8840 tx=0x7f6438087060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f6425ffb640 1 -- 172.21.15.143:0/1066846161 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340107c0 con 0x56409da974a0 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f6425ffb640 1 -- 172.21.15.143:0/1066846161 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6434010e10 con 0x56409da974a0 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.000+0000 7f6425ffb640 1 -- 172.21.15.143:0/1066846161 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434019c40 con 0x56409da974a0 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 -- 172.21.15.143:0/1066846161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 msgr2=0x56409da97880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409da97880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6434009fd0 tx=0x7f64340096d0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 -- 172.21.15.143:0/1066846161 shutdown_connections 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 --2- 172.21.15.143:0/1066846161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409da97880 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 -- 172.21.15.143:0/1066846161 >> 172.21.15.143:0/1066846161 conn(0x56409da93a30 msgr2=0x56409da93e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 -- 172.21.15.143:0/1066846161 shutdown_connections 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 -- 172.21.15.143:0/1066846161 wait complete. 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.001+0000 7f6448488740 1 -- start start 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 Processor -- start 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da974a0 0x56409daa1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da97dc0 0x56409da9f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409da9f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56409daa5250 con 0x56409daa2680 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56409d9b8050 con 0x56409da97dc0 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56409daa53d0 con 0x56409da974a0 2024-09-10T10:46:54.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f6448488740 10 client.0 _ll_register_callbacks cb 0x56409d9f8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409da9f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409da9f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34972/0 (socket says 172.21.15.143:34972) 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 -- 172.21.15.143:0/1366797719 learned_addr learned my addr 172.21.15.143:0/1366797719 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da974a0 0x56409daa1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da97dc0 0x56409da9f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 -- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da974a0 msgr2=0x56409daa1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da974a0 0x56409daa1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 -- 172.21.15.143:0/1366797719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da97dc0 msgr2=0x56409da9f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da97dc0 0x56409da9f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 -- 172.21.15.143:0/1366797719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409d76e3c0 con 0x56409daa2680 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f643effd640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409da9f7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6434009950 tx=0x7f643400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc0028350cdd9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.002+0000 7f641effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f641effd640 1 -- 172.21.15.143:0/1366797719 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643402d060 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f641effd640 1 -- 172.21.15.143:0/1366797719 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6434010bf0 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6434010e10 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f641effd640 1 -- 172.21.15.143:0/1366797719 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6434034050 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56409dab0c20 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f641effd640 1 -- 172.21.15.143:0/1366797719 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6434010e10 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f641effd640 1 -- 172.21.15.143:0/1366797719 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6434025070 con 0x56409daa2680 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 20 client.16002 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409dab1fc0 0x56409dab43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 4 client.16002 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.003+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56409da0a850 con 0x56409dab1fc0 2024-09-10T10:46:54.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.004+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409dab1fc0 0x56409dab43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.004+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409dab1fc0 0x56409dab43b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa763335c297829 server_cookie=27f5be78cd81649e in_seq=0 out_seq=0 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.004+0000 7f641effd640 10 client.16002 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f641effd640 1 -- 172.21.15.143:0/1366797719 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56409da0a850 con 0x56409dab1fc0 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f641effd640 10 client.16002 handle_command_reply: tid=0 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 client.16002 shutdown 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 msgr2=0x56409da9f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409da9f7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6434009950 tx=0x7f643400c040 comp rx=0 tx=0).stop 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 shutdown_connections 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409da9f7e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da97dc0 0x56409da9f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409dab1fc0 0x56409dab43b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 --2- 172.21.15.143:0/1366797719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da974a0 0x56409daa1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f641effd640 0 client.16002 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 >> 172.21.15.143:0/1366797719 conn(0x56409da93a30 msgr2=0x56409da99760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 shutdown_connections 2024-09-10T10:46:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 1 -- 172.21.15.143:0/1366797719 wait complete. 2024-09-10T10:46:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.005+0000 7f6448488740 20 client.16002 trim_cache size 0 max 16384 2024-09-10T10:46:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 Processor -- start 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 -- start start 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da974a0 0x56409daae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409dab9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409daba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f64401aeac0 con 0x56409daa2680 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64401b0600 con 0x56409da974a0 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f6448488740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64401af190 con 0x56409da97dc0 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409daba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409daba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34978/0 (socket says 172.21.15.143:34978) 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f643effd640 1 -- 172.21.15.143:0/3275367787 learned_addr learned my addr 172.21.15.143:0/3275367787 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f643f7fe640 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409dab9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.022+0000 7f643effd640 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da974a0 0x56409daae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643effd640 1 -- 172.21.15.143:0/3275367787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 msgr2=0x56409dab9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643effd640 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409dab9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643effd640 1 -- 172.21.15.143:0/3275367787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da974a0 msgr2=0x56409daae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643effd640 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da974a0 0x56409daae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643effd640 1 -- 172.21.15.143:0/3275367787 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409daa2340 con 0x56409daa2680 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643f7fe640 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409dab9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f643effd640 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409daba140 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f643401d040 tx=0x7f643400a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab82e96b9157c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f64257fa640 1 -- 172.21.15.143:0/3275367787 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643401f040 con 0x56409daa2680 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f64257fa640 1 -- 172.21.15.143:0/3275367787 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6434009ad0 con 0x56409daa2680 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f64257fa640 1 -- 172.21.15.143:0/3275367787 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643401a410 con 0x56409daa2680 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f6448488740 1 -- 172.21.15.143:0/3275367787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 msgr2=0x56409daba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.023+0000 7f6448488740 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409daba140 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f643401d040 tx=0x7f643400a610 comp rx=0 tx=0).stop 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- 172.21.15.143:0/3275367787 shutdown_connections 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409daa2680 0x56409daba140 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409dab9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 --2- 172.21.15.143:0/3275367787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409da974a0 0x56409daae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- 172.21.15.143:0/3275367787 >> 172.21.15.143:0/3275367787 conn(0x56409da93a30 msgr2=0x56409da9db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- 172.21.15.143:0/3275367787 shutdown_connections 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- 172.21.15.143:0/3275367787 wait complete. 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- start start 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 Processor -- start 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409daaad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f64401aff30 con 0x56409da974a0 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64401af860 con 0x56409daa2680 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.024+0000 7f6448488740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64401acaa0 con 0x56409da97dc0 2024-09-10T10:46:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f6448488740 10 client.0 _ll_register_callbacks cb 0x7f643000a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409daaad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409daaad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34988/0 (socket says 172.21.15.143:34988) 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 -- 172.21.15.143:0/1581733372 learned_addr learned my addr 172.21.15.143:0/1581733372 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 -- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 msgr2=0x56409daa6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 -- 172.21.15.143:0/1581733372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 msgr2=0x56409daa65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 -- 172.21.15.143:0/1581733372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409daa6da0 con 0x56409da974a0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f643effd640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409daaad30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f643400a550 tx=0x7f6434019470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23664303901400c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f641effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:46:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f641effd640 1 -- 172.21.15.143:0/1581733372 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643403f020 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f641effd640 1 -- 172.21.15.143:0/1581733372 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f64340197a0 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6434009ad0 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f641effd640 1 -- 172.21.15.143:0/1581733372 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6434047050 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f6448488740 10 client.16008 fetch_fsmap learned FSMap version 29 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.025+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56409db1cbb0 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f641effd640 1 -- 172.21.15.143:0/1581733372 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6434009ad0 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f641effd640 1 -- 172.21.15.143:0/1581733372 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6434025030 con 0x56409da974a0 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f6448488740 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f6448488740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f6448488740 20 client.16008 populate_metadata read hostname 'smithi143' 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f6448488740 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409db1d160 0x56409db1f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f6448488740 4 client.16008 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:46:54.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56409da04d60 con 0x56409db1d160 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.026+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409db1d160 0x56409db1f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f641effd640 10 client.16008 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f643f7fe640 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409db1d160 0x56409db1f550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9127dddedc119fe6 server_cookie=92543d2067a3ae89 in_seq=0 out_seq=0 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f641effd640 1 -- 172.21.15.143:0/1581733372 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56409da04d60 con 0x56409db1d160 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f641effd640 10 client.16008 handle_command_reply: tid=0 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 client.16008 shutdown 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 msgr2=0x56409daaad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409daaad30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f643400a550 tx=0x7f6434019470 comp rx=0 tx=0).stop 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 shutdown_connections 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56409daa2680 0x56409daa65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56409db1d160 0x56409db1f550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56409da97dc0 0x56409daa6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 --2- 172.21.15.143:0/1581733372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56409da974a0 0x56409daaad30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f641effd640 0 client.16008 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 >> 172.21.15.143:0/1581733372 conn(0x56409da93a30 msgr2=0x56409daa0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 shutdown_connections 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 1 -- 172.21.15.143:0/1581733372 wait complete. 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.027+0000 7f6448488740 20 client.16008 trim_cache size 0 max 16384 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "0d37f11e-f839-4f8f-8443-659dc4bd52ec": { 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:46:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "0d37f11e-f839-4f8f-8443-659dc4bd52ec", 2024-09-10T10:46:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:46:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:46:54.036 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64280a2c00 msgr2=0x7f64280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64280a2c00 0x7f64280a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64380b8840 tx=0x7f6438087060 comp rx=0 tx=0).stop 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 msgr2=0x7f644019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 0x7f644019cca0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6438009900 tx=0x7f6438004270 comp rx=0 tx=0).stop 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 shutdown_connections 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f64280a2c00 0x7f64280a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6440106a80 0x7f644019cca0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440101420 0x7f6440198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 --2- 172.21.15.143:0/1054017682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6440100a80 0x7f64401983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 >> 172.21.15.143:0/1054017682 conn(0x7f64400fc7a0 msgr2=0x7f644006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 shutdown_connections 2024-09-10T10:46:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:46:54.028+0000 7f64460af640 1 -- 172.21.15.143:0/1054017682 wait complete. 2024-09-10T10:46:54.755 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:54 smithi143 ceph-mon[31057]: pgmap v406: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T10:46:54.755 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:54 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:46:54.786 INFO:tasks.cephfs.filesystem:scrub status for tag:0d37f11e-f839-4f8f-8443-659dc4bd52ec - {'path': '/', 'tag': '0d37f11e-f839-4f8f-8443-659dc4bd52ec', 'options': 'recursive,force'} 2024-09-10T10:46:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:54 smithi178 ceph-mon[34813]: pgmap v406: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T10:46:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:54 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:46:54.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:54 smithi144 ceph-mon[34230]: pgmap v406: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T10:46:54.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:54 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:46:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:56 smithi178 ceph-mon[34813]: pgmap v407: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:56 smithi144 ceph-mon[34230]: pgmap v407: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:57.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:56 smithi143 ceph-mon[31057]: pgmap v407: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:46:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:46:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:57 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:46:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:57 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:58.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:46:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:57 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:58 smithi178 ceph-mon[34813]: pgmap v408: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:46:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:46:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:46:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:58 smithi144 ceph-mon[34230]: pgmap v408: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:46:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:46:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:46:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:46:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:58 smithi143 ceph-mon[31057]: pgmap v408: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:46:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:46:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:46:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:46:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:47:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:00 smithi178 ceph-mon[34813]: pgmap v409: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T10:47:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:00 smithi144 ceph-mon[34230]: pgmap v409: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T10:47:01.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:00 smithi143 ceph-mon[31057]: pgmap v409: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T10:47:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:02 smithi178 ceph-mon[34813]: pgmap v410: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-10T10:47:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:02 smithi144 ceph-mon[34230]: pgmap v410: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-10T10:47:03.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:02 smithi143 ceph-mon[31057]: pgmap v410: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-10T10:47:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:04 smithi178 ceph-mon[34813]: pgmap v411: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T10:47:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:04 smithi144 ceph-mon[34230]: pgmap v411: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T10:47:05.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:04 smithi143 ceph-mon[31057]: pgmap v411: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T10:47:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:06 smithi178 ceph-mon[34813]: pgmap v412: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:47:06.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:06 smithi144 ceph-mon[34230]: pgmap v412: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:47:07.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:06 smithi143 ceph-mon[31057]: pgmap v412: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T10:47:08.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:08 smithi178 ceph-mon[34813]: pgmap v413: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:47:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:08 smithi144 ceph-mon[34230]: pgmap v413: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:47:09.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:08 smithi143 ceph-mon[31057]: pgmap v413: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T10:47:10.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:47:10.672 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:10.793 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:10 smithi143 ceph-mon[31057]: pgmap v414: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:47:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:10 smithi178 ceph-mon[34813]: pgmap v414: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:47:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:10 smithi144 ceph-mon[34230]: pgmap v414: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:47:11.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 -- 172.21.15.143:0/944376415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 msgr2=0x7f00e8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 --2- 172.21.15.143:0/944376415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8103110 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009930 tx=0x7f00d802a440 comp rx=0 tx=0).stop 2024-09-10T10:47:11.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 -- 172.21.15.143:0/944376415 shutdown_connections 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 --2- 172.21.15.143:0/944376415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00e81082a0 0x7f00e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 --2- 172.21.15.143:0/944376415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e81036e0 0x7f00e8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 --2- 172.21.15.143:0/944376415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8103110 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 -- 172.21.15.143:0/944376415 >> 172.21.15.143:0/944376415 conn(0x7f00e80fc7d0 msgr2=0x7f00e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 -- 172.21.15.143:0/944376415 shutdown_connections 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 -- 172.21.15.143:0/944376415 wait complete. 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.162+0000 7f00ed574640 1 Processor -- start 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 -- start start 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00e81036e0 0x7f00e8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e81082a0 0x7f00e819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111f90 con 0x7f00e8102d30 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111e10 con 0x7f00e81036e0 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00ed574640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112110 con 0x7f00e81082a0 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38022/0 (socket says 172.21.15.143:38022) 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 -- 172.21.15.143:0/1709039378 learned_addr learned my addr 172.21.15.143:0/1709039378 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 -- 172.21.15.143:0/1709039378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e81082a0 msgr2=0x7f00e819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e81082a0 0x7f00e819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 -- 172.21.15.143:0/1709039378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00e81036e0 msgr2=0x7f00e8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00dffff640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00e81036e0 0x7f00e8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00e81036e0 0x7f00e8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 -- 172.21.15.143:0/1709039378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e819d500 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e6ffd640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8198550 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004770 tx=0x7f00d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d803f070 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00d803b460 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.163+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8035070 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.164+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e819d790 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.164+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a50d0 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.164+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8103110 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.166+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f00d8003a00 con 0x7f00e8102d30 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.167+0000 7f00e4ff9640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00d40a2c00 0x7f00d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.167+0000 7f00dffff640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00d40a2c00 0x7f00d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.167+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f00d80efa60 con 0x7f00e8102d30 2024-09-10T10:47:11.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.168+0000 7f00dffff640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00d40a2c00 0x7f00d40a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f00e80fee80 tx=0x7f00d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.170+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00d80bc660 con 0x7f00e8102d30 2024-09-10T10:47:11.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.372+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00e81998b0 con 0x7f00e8102d30 2024-09-10T10:47:11.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.372+0000 7f00e4ff9640 1 -- 172.21.15.143:0/1709039378 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11998 (secure 0 0 0) 0x7f00d80bbf60 con 0x7f00e8102d30 2024-09-10T10:47:11.374 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:47:11.375 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"btime":"2024-09-10T10:45:17:572085+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:45:16.857221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00d40a2c00 msgr2=0x7f00d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00d40a2c00 0x7f00d40a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f00e80fee80 tx=0x7f00d0008040 comp rx=0 tx=0).stop 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 msgr2=0x7f00e8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8198550 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004770 tx=0x7f00d80042f0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 shutdown_connections 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f00d40a2c00 0x7f00d40a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.374+0000 7f00ed574640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e81082a0 0x7f00e819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.375+0000 7f00ed574640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00e81036e0 0x7f00e8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.375+0000 7f00ed574640 1 --2- 172.21.15.143:0/1709039378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00e8102d30 0x7f00e8198550 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.375+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 >> 172.21.15.143:0/1709039378 conn(0x7f00e80fc7d0 msgr2=0x7f00e8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.375+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 shutdown_connections 2024-09-10T10:47:11.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:11.375+0000 7f00ed574640 1 -- 172.21.15.143:0/1709039378 wait complete. 2024-09-10T10:47:11.747 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:11 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1709039378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:47:11.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:11 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1709039378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:47:11.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:11 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1709039378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:47:12.211 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 218ba9b --release --await' 2024-09-10T10:47:12.211 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 218ba9b --release --await 2024-09-10T10:47:12.456 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:12.887 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:12 smithi143 ceph-mon[31057]: pgmap v415: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T10:47:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:12 smithi178 ceph-mon[34813]: pgmap v415: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T10:47:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:12 smithi144 ceph-mon[34230]: pgmap v415: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T10:47:12.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 -- 172.21.15.143:0/2971506466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 msgr2=0x7fb8dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/2971506466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc10ea90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009ca0 tx=0x7fb8d402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 -- 172.21.15.143:0/2971506466 shutdown_connections 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/2971506466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc10ea90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/2971506466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8dc102790 0x7fb8dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/2971506466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8dc101de0 0x7fb8dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 -- 172.21.15.143:0/2971506466 >> 172.21.15.143:0/2971506466 conn(0x7fb8dc0fd8c0 msgr2=0x7fb8dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 -- 172.21.15.143:0/2971506466 shutdown_connections 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.973+0000 7fb8e08cb640 1 -- 172.21.15.143:0/2971506466 wait complete. 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.974+0000 7fb8e08cb640 1 Processor -- start 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.974+0000 7fb8e08cb640 1 -- start start 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.974+0000 7fb8e08cb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8dc101de0 0x7fb8dc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.974+0000 7fb8e08cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8dc102790 0x7fb8dc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8e08cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8e08cb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc111e40 con 0x7fb8dc106f10 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8e08cb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc111cc0 con 0x7fb8dc101de0 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8e08cb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc111fc0 con 0x7fb8dc102790 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38048/0 (socket says 172.21.15.143:38048) 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 -- 172.21.15.143:0/242499469 learned_addr learned my addr 172.21.15.143:0/242499469 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8dc102790 0x7fb8dc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8da59c640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8dc101de0 0x7fb8dc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 -- 172.21.15.143:0/242499469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8dc102790 msgr2=0x7fb8dc198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8dc102790 0x7fb8dc198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 -- 172.21.15.143:0/242499469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8dc101de0 msgr2=0x7fb8dc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8dc101de0 0x7fb8dc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 -- 172.21.15.143:0/242499469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc19d390 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc19ccb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009dc0 tx=0x7fb8d4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/242499469 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4003cf0 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/242499469 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d40044f0 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc19d620 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/242499469 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4030b80 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.975+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1a4f60 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.977+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/242499469 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb8d40046a0 con 0x7fb8dc106f10 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 Processor -- start 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 -- start start 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea6455440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea6462630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8cb7fe640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8c40a2b80 0x7fb8c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555ea63ba620 con 0x555ea6455060 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ea6361e50 con 0x555ea6455980 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.978+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ea63b65e0 con 0x555ea6460240 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea6455440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea6455440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38064/0 (socket says 172.21.15.143:38064) 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2633065065 learned_addr learned my addr 172.21.15.143:0/2633065065 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8c40a2b80 0x7fb8c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8cb7fe640 1 -- 172.21.15.143:0/242499469 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb8d40f0870 con 0x7fb8dc106f10 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea6462630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2633065065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 msgr2=0x555ea6462630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea6462630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2633065065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 msgr2=0x555ea645fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/2633065065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea6352d40 con 0x555ea6455060 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea6462630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea6455440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00159a0 tx=0x7fb8d0015e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe032b0d8bf672bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8c40a2b80 0x7fb8c40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00177b0 tx=0x7fb8d001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8c8ff9640 1 -- 172.21.15.143:0/2633065065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00153a0 con 0x555ea6455060 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8c8ff9640 1 -- 172.21.15.143:0/2633065065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d001c070 con 0x555ea6455060 2024-09-10T10:47:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.979+0000 7fb8c8ff9640 1 -- 172.21.15.143:0/2633065065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00208c0 con 0x555ea6455060 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2633065065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 msgr2=0x555ea6455440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea6455440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00159a0 tx=0x7fb8d0015e60 comp rx=0 tx=0).stop 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2633065065 shutdown_connections 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea6462630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2633065065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea6455440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2633065065 >> 172.21.15.143:0/2633065065 conn(0x555ea64515f0 msgr2=0x555ea64519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2633065065 shutdown_connections 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.980+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2633065065 wait complete. 2024-09-10T10:47:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 -- start start 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 Processor -- start 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea645dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555ea6467220 con 0x555ea6455060 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ea6342f10 con 0x555ea6455980 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555ea64673a0 con 0x555ea6460240 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8e2ca4740 10 client.0 _ll_register_callbacks cb 0x555ea6366db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38070/0 (socket says 172.21.15.143:38070) 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 -- 172.21.15.143:0/475316727 learned_addr learned my addr 172.21.15.143:0/475316727 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 -- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 msgr2=0x555ea645dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea645dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 -- 172.21.15.143:0/475316727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 msgr2=0x555ea645d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.981+0000 7fb8da59c640 1 -- 172.21.15.143:0/475316727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea60c9580 con 0x555ea6455060 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8da59c640 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00baa0 tx=0x7fb8cc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ab7531988af83e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/475316727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc01b640 con 0x555ea6455060 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/475316727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb8cc011440 con 0x555ea6455060 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d001c070 con 0x555ea6455060 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/475316727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc01b7e0 con 0x555ea6455060 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 10 client.16026 fetch_fsmap learned FSMap version 29 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ea6193960 con 0x555ea6455060 2024-09-10T10:47:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/475316727 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d001c070 con 0x555ea6455060 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/475316727 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb8cc019020 con 0x555ea6455060 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 10 client.16026 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 20 client.16026 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 4 client.16026 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.982+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea63c7b90 con 0x555ea6474240 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.983+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.983+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64766e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f576a8a3ac9c7954 server_cookie=5c92b681d5d0ba6 in_seq=0 out_seq=0 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.983+0000 7fb8c1ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/475316727 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ea63c7b90 con 0x555ea6474240 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8c1ffb640 10 client.16026 handle_command_reply: tid=0 2024-09-10T10:47:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 client.16026 shutdown 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 msgr2=0x555ea645d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00baa0 tx=0x7fb8cc00bf60 comp rx=0 tx=0).stop 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 shutdown_connections 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64766e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea645dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8c1ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/475316727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d190 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 >> 172.21.15.143:0/475316727 conn(0x555ea64515f0 msgr2=0x555ea6450750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 shutdown_connections 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/475316727 wait complete. 2024-09-10T10:47:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:12.984+0000 7fb8e2ca4740 20 client.16026 trim_cache size 0 max 16384 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 Processor -- start 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 -- start start 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea647bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1ac900 con 0x555ea6455060 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1ae440 con 0x555ea6455980 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1acfd0 con 0x555ea6460240 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38076/0 (socket says 172.21.15.143:38076) 2024-09-10T10:47:13.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 -- 172.21.15.143:0/2396359222 learned_addr learned my addr 172.21.15.143:0/2396359222 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea647bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 -- 172.21.15.143:0/2396359222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 msgr2=0x555ea647bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea647bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 -- 172.21.15.143:0/2396359222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 msgr2=0x555ea645dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8da59c640 1 -- 172.21.15.143:0/2396359222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea6454e90 con 0x555ea6455060 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.001+0000 7fb8dad9d640 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea647bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8da59c640 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d860 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc010540 tx=0x7fb8cc005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feb26246e7f0ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8c3fff640 1 -- 172.21.15.143:0/2396359222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc02c020 con 0x555ea6455060 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8c3fff640 1 -- 172.21.15.143:0/2396359222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8cc01be30 con 0x555ea6455060 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8c3fff640 1 -- 172.21.15.143:0/2396359222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc005060 con 0x555ea6455060 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2396359222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 msgr2=0x555ea645d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d860 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc010540 tx=0x7fb8cc005b90 comp rx=0 tx=0).stop 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2396359222 shutdown_connections 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea647bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455980 0x555ea645dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/2396359222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455060 0x555ea645d860 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2396359222 >> 172.21.15.143:0/2396359222 conn(0x555ea64515f0 msgr2=0x555ea645d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2396359222 shutdown_connections 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.002+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/2396359222 wait complete. 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 -- start start 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 Processor -- start 2024-09-10T10:47:13.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455060 0x555ea6468cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 0x555ea6469200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea646c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1add70 con 0x555ea6455980 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1ad6a0 con 0x555ea6455060 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1aa8e0 con 0x555ea6460240 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8e2ca4740 10 client.0 _ll_register_callbacks cb 0x7fb8cc004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 0x555ea6469200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 0x555ea6469200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38090/0 (socket says 172.21.15.143:38090) 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/3174616360 learned_addr learned my addr 172.21.15.143:0/3174616360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 msgr2=0x555ea646c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea646c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/3174616360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455060 msgr2=0x555ea6468cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455060 0x555ea6468cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.003+0000 7fb8d9d9b640 1 -- 172.21.15.143:0/3174616360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea646c960 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8d9d9b640 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 0x555ea6469200 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x555ea645ca20 tx=0x7fb8d0009160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c356eeadf433e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/3174616360 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0018a60 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/3174616360 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb8d0018c00 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc01be30 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/3174616360 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8d0018e00 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ea64dfab0 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/3174616360 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8cc01be30 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/3174616360 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb8d0033020 con 0x555ea6455980 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 10 client.16032 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:47:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 20 client.16032 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 4 client.16032 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "218ba9b", "release": true, "await": true}] 2024-09-10T10:47:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.004+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "set_id": "218ba9b", "release": true, "await": true}) -- 0x555ea63c20a0 con 0x555ea6474240 2024-09-10T10:47:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.005+0000 7fb8da59c640 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.005+0000 7fb8c1ffb640 10 client.16032 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.005+0000 7fb8da59c640 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64e1fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adddbf56bb055209 server_cookie=4cc2af8c501cf973 in_seq=0 out_seq=0 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8c1ffb640 1 -- 172.21.15.143:0/3174616360 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x555ea63c20a0 con 0x555ea6474240 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8c1ffb640 10 client.16032 handle_command_reply: tid=0 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 client.16032 shutdown 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 msgr2=0x555ea6469200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 0x555ea6469200 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x555ea645ca20 tx=0x7fb8d0009160 comp rx=0 tx=0).stop 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 shutdown_connections 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555ea6474240 0x555ea64e1fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ea6460240 0x555ea646c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555ea6455980 0x555ea6469200 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 --2- 172.21.15.143:0/3174616360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ea6455060 0x555ea6468cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 >> 172.21.15.143:0/3174616360 conn(0x555ea64515f0 msgr2=0x555ea64572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8c1ffb640 0 client.16032 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 shutdown_connections 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 1 -- 172.21.15.143:0/3174616360 wait complete. 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e2ca4740 20 client.16032 trim_cache size 0 max 16384 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 29, 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 8, 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:47:13.012 INFO:teuthology.orchestra.run.smithi143.stdout: "218ba9b": { 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 8, 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 150.0, 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:47:13.013 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:47:13.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.009+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8c40a2b80 msgr2=0x7fb8c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8c40a2b80 0x7fb8c40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00177b0 tx=0x7fb8d001b000 comp rx=0 tx=0).stop 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 msgr2=0x7fb8dc19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc19ccb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009dc0 tx=0x7fb8d4032ff0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 shutdown_connections 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8c40a2b80 0x7fb8c40a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc106f10 0x7fb8dc19ccb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8dc102790 0x7fb8dc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 --2- 172.21.15.143:0/242499469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8dc101de0 0x7fb8dc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 >> 172.21.15.143:0/242499469 conn(0x7fb8dc0fd8c0 msgr2=0x7fb8dc0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 shutdown_connections 2024-09-10T10:47:13.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:13.010+0000 7fb8e08cb640 1 -- 172.21.15.143:0/242499469 wait complete. 2024-09-10T10:47:13.022 INFO:tasks.workunit.client.0.smithi143.stdout:getting blogbench 2024-09-10T10:47:13.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'getting blogbench' 2024-09-10T10:47:13.022 INFO:tasks.workunit.client.0.smithi143.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-10T10:47:13.105 INFO:tasks.workunit.client.0.smithi143.stderr:--2024-09-10 10:47:13-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-10T10:47:13.106 INFO:tasks.workunit.client.0.smithi143.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-10T10:47:13.127 INFO:tasks.workunit.client.0.smithi143.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-10T10:47:13.151 INFO:tasks.workunit.client.0.smithi143.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-10T10:47:13.151 INFO:tasks.workunit.client.0.smithi143.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-10T10:47:13.151 INFO:tasks.workunit.client.0.smithi143.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-10T10:47:13.153 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:47:13.190 INFO:tasks.workunit.client.0.smithi143.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2024-09-10T10:47:13.210 INFO:tasks.workunit.client.0.smithi143.stderr: 50K .......... .......... .......... .......... .......... 99% 2.40M 0s 2024-09-10T10:47:13.210 INFO:tasks.workunit.client.0.smithi143.stderr: 100K 100% 1.74T=0.06s 2024-09-10T10:47:13.210 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:47:13.210 INFO:tasks.workunit.client.0.smithi143.stderr:2024-09-10 10:47:13 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-10T10:47:13.210 INFO:tasks.workunit.client.0.smithi143.stderr: 2024-09-10T10:47:13.211 INFO:tasks.workunit.client.0.smithi143.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-10T10:47:13.270 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/ 2024-09-10T10:47:13.271 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/m4/ 2024-09-10T10:47:13.273 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-10T10:47:13.275 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/man/ 2024-09-10T10:47:13.277 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/man/blogbench.8 2024-09-10T10:47:13.279 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/man/Makefile.am 2024-09-10T10:47:13.282 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/man/Makefile.in 2024-09-10T10:47:13.284 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/ 2024-09-10T10:47:13.286 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-10T10:47:13.288 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-10T10:47:13.290 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/blogbench.c 2024-09-10T10:47:13.292 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/blogbench.h 2024-09-10T10:47:13.294 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-10T10:47:13.297 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-10T10:47:13.299 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-10T10:47:13.302 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-10T10:47:13.304 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-10T10:47:13.307 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/Makefile.am 2024-09-10T10:47:13.308 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/Makefile.in 2024-09-10T10:47:13.310 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/process.c 2024-09-10T10:47:13.313 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/globals.h 2024-09-10T10:47:13.315 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/commenter.c 2024-09-10T10:47:13.317 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/rewriter.c 2024-09-10T10:47:13.319 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/reader.c 2024-09-10T10:47:13.321 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/writer.c 2024-09-10T10:47:13.323 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/src/helpers.c 2024-09-10T10:47:13.325 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/NEWS 2024-09-10T10:47:13.327 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/compile 2024-09-10T10:47:13.329 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/depcomp 2024-09-10T10:47:13.332 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/aclocal.m4 2024-09-10T10:47:13.334 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/README 2024-09-10T10:47:13.336 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/configure 2024-09-10T10:47:13.343 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/configure.ac 2024-09-10T10:47:13.345 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/config.guess 2024-09-10T10:47:13.348 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/install-sh 2024-09-10T10:47:13.351 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/config.sub 2024-09-10T10:47:13.353 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/missing 2024-09-10T10:47:13.355 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/mkinstalldirs 2024-09-10T10:47:13.357 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/Makefile.am 2024-09-10T10:47:13.359 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/Makefile.in 2024-09-10T10:47:13.361 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/config.h.in 2024-09-10T10:47:13.363 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/AUTHORS 2024-09-10T10:47:13.365 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/INSTALL 2024-09-10T10:47:13.368 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/ChangeLog 2024-09-10T10:47:13.370 INFO:tasks.workunit.client.0.smithi143.stdout:blogbench-1.0/COPYING 2024-09-10T10:47:13.373 INFO:tasks.workunit.client.0.smithi143.stderr:+ cd blogbench-1.0/ 2024-09-10T10:47:13.375 INFO:tasks.workunit.client.0.smithi143.stdout:making blogbench 2024-09-10T10:47:13.375 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'making blogbench' 2024-09-10T10:47:13.375 INFO:tasks.workunit.client.0.smithi143.stderr:+ ./configure 2024-09-10T10:47:13.517 INFO:tasks.workunit.client.0.smithi143.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T10:47:13.762 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '218ba9b', total seconds elapsed: 33.3 2024-09-10T10:47:14.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:13 smithi178 ceph-mon[34813]: pgmap v416: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T10:47:14.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:13 smithi144 ceph-mon[34230]: pgmap v416: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T10:47:14.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:13 smithi143 ceph-mon[31057]: pgmap v416: 129 pgs: 129 active+clean; 666 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T10:47:14.526 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether build environment is sane... yes 2024-09-10T10:47:14.539 INFO:tasks.workunit.client.0.smithi143.stdout:checking for gawk... gawk 2024-09-10T10:47:14.561 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T10:47:14.574 INFO:tasks.workunit.client.0.smithi143.stdout:checking for gcc... gcc 2024-09-10T10:47:14.645 INFO:tasks.workunit.client.0.smithi143.stdout:checking for C compiler default output file name... a.out 2024-09-10T10:47:14.647 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether the C compiler works... yes 2024-09-10T10:47:14.651 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether we are cross compiling... no 2024-09-10T10:47:14.699 INFO:tasks.workunit.client.0.smithi143.stdout:checking for suffix of executables... 2024-09-10T10:47:14.743 INFO:tasks.workunit.client.0.smithi143.stdout:checking for suffix of object files... o 2024-09-10T10:47:14.786 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T10:47:14.834 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether gcc accepts -g... yes 2024-09-10T10:47:14.942 INFO:tasks.workunit.client.0.smithi143.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-10T10:47:15.000 INFO:tasks.workunit.client.0.smithi143.stdout:checking for style of include used by make... GNU 2024-09-10T10:47:15.140 INFO:tasks.workunit.client.0.smithi143.stdout:checking dependency style of gcc... gcc3 2024-09-10T10:47:15.140 INFO:tasks.workunit.client.0.smithi143.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T10:47:15.225 INFO:tasks.workunit.client.0.smithi143.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T10:47:15.283 INFO:tasks.workunit.client.0.smithi143.stdout:checking for egrep... grep -E 2024-09-10T10:47:15.296 INFO:tasks.workunit.client.0.smithi143.stdout:checking for AIX... no 2024-09-10T10:47:15.390 INFO:tasks.workunit.client.0.smithi143.stdout:checking for library containing strerror... none required 2024-09-10T10:47:15.602 INFO:tasks.workunit.client.0.smithi143.stdout:checking for ANSI C header files... yes 2024-09-10T10:47:15.663 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T10:47:15.724 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/types.h... yes 2024-09-10T10:47:15.788 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/stat.h... yes 2024-09-10T10:47:15.852 INFO:tasks.workunit.client.0.smithi143.stdout:checking for stdlib.h... yes 2024-09-10T10:47:15.917 INFO:tasks.workunit.client.0.smithi143.stdout:checking for string.h... yes 2024-09-10T10:47:15.992 INFO:tasks.workunit.client.0.smithi143.stdout:checking for memory.h... yes 2024-09-10T10:47:16.056 INFO:tasks.workunit.client.0.smithi143.stdout:checking for strings.h... yes 2024-09-10T10:47:16.121 INFO:tasks.workunit.client.0.smithi143.stdout:checking for inttypes.h... yes 2024-09-10T10:47:16.187 INFO:tasks.workunit.client.0.smithi143.stdout:checking for stdint.h... yes 2024-09-10T10:47:16.258 INFO:tasks.workunit.client.0.smithi143.stdout:checking for unistd.h... yes 2024-09-10T10:47:16.276 INFO:tasks.workunit.client.0.smithi143.stdout:checking for memory.h... (cached) yes 2024-09-10T10:47:16.286 INFO:tasks.workunit.client.0.smithi143.stdout:checking for strings.h... (cached) yes 2024-09-10T10:47:16.299 INFO:tasks.workunit.client.0.smithi143.stdout:checking for unistd.h... (cached) yes 2024-09-10T10:47:16.371 INFO:tasks.workunit.client.0.smithi143.stdout:checking locale.h usability... yes 2024-09-10T10:47:16.416 INFO:tasks.workunit.client.0.smithi143.stdout:checking locale.h presence... yes 2024-09-10T10:47:16.418 INFO:tasks.workunit.client.0.smithi143.stdout:checking for locale.h... yes 2024-09-10T10:47:16.447 INFO:tasks.workunit.client.0.smithi143.stdout:checking for stdlib.h... (cached) yes 2024-09-10T10:47:16.457 INFO:tasks.workunit.client.0.smithi143.stdout:checking for string.h... (cached) yes 2024-09-10T10:47:16.552 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/socket.h usability... yes 2024-09-10T10:47:16.582 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/socket.h presence... yes 2024-09-10T10:47:16.583 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/socket.h... yes 2024-09-10T10:47:16.653 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/time.h usability... yes 2024-09-10T10:47:16.683 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/time.h presence... yes 2024-09-10T10:47:16.683 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/time.h... yes 2024-09-10T10:47:16.755 INFO:tasks.workunit.client.0.smithi143.stdout:checking fcntl.h usability... yes 2024-09-10T10:47:16.783 INFO:tasks.workunit.client.0.smithi143.stdout:checking fcntl.h presence... yes 2024-09-10T10:47:16.784 INFO:tasks.workunit.client.0.smithi143.stdout:checking for fcntl.h... yes 2024-09-10T10:47:16.861 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/fcntl.h usability... yes 2024-09-10T10:47:16.888 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/fcntl.h presence... yes 2024-09-10T10:47:16.889 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/fcntl.h... yes 2024-09-10T10:47:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:16 smithi178 ceph-mon[34813]: pgmap v417: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-10T10:47:16.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:16 smithi144 ceph-mon[34230]: pgmap v417: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-10T10:47:17.029 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/param.h usability... yes 2024-09-10T10:47:17.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:16 smithi143 ceph-mon[31057]: pgmap v417: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-09-10T10:47:17.059 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/param.h presence... yes 2024-09-10T10:47:17.060 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/param.h... yes 2024-09-10T10:47:17.125 INFO:tasks.workunit.client.0.smithi143.stdout:checking ioctl.h usability... no 2024-09-10T10:47:17.155 INFO:tasks.workunit.client.0.smithi143.stdout:checking ioctl.h presence... no 2024-09-10T10:47:17.156 INFO:tasks.workunit.client.0.smithi143.stdout:checking for ioctl.h... no 2024-09-10T10:47:17.221 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/ioctl.h usability... yes 2024-09-10T10:47:17.248 INFO:tasks.workunit.client.0.smithi143.stdout:checking sys/ioctl.h presence... yes 2024-09-10T10:47:17.249 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sys/ioctl.h... yes 2024-09-10T10:47:17.315 INFO:tasks.workunit.client.0.smithi143.stdout:checking return type of signal handlers... void 2024-09-10T10:47:17.374 INFO:tasks.workunit.client.0.smithi143.stdout:checking for size_t... yes 2024-09-10T10:47:17.424 INFO:tasks.workunit.client.0.smithi143.stdout:checking for dev_t... yes 2024-09-10T10:47:17.479 INFO:tasks.workunit.client.0.smithi143.stdout:checking for short... yes 2024-09-10T10:47:17.586 INFO:tasks.workunit.client.0.smithi143.stdout:checking size of short... 2 2024-09-10T10:47:17.647 INFO:tasks.workunit.client.0.smithi143.stdout:checking for int... yes 2024-09-10T10:47:17.736 INFO:tasks.workunit.client.0.smithi143.stdout:checking size of int... 4 2024-09-10T10:47:17.793 INFO:tasks.workunit.client.0.smithi143.stdout:checking for long... yes 2024-09-10T10:47:17.884 INFO:tasks.workunit.client.0.smithi143.stdout:checking size of long... 8 2024-09-10T10:47:17.944 INFO:tasks.workunit.client.0.smithi143.stdout:checking for long long... yes 2024-09-10T10:47:18.031 INFO:tasks.workunit.client.0.smithi143.stdout:checking size of long long... 8 2024-09-10T10:47:18.089 INFO:tasks.workunit.client.0.smithi143.stdout:checking for ssize_t... yes 2024-09-10T10:47:18.139 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sig_atomic_t... yes 2024-09-10T10:47:18.206 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether gcc needs -traditional... no 2024-09-10T10:47:18.252 INFO:tasks.workunit.client.0.smithi143.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T10:47:18.285 INFO:tasks.workunit.client.0.smithi143.stdout:checking for inline... inline 2024-09-10T10:47:18.371 INFO:tasks.workunit.client.0.smithi143.stdout:checking for working alloca.h... yes 2024-09-10T10:47:18.453 INFO:tasks.workunit.client.0.smithi143.stdout:checking for alloca... yes 2024-09-10T10:47:18.545 INFO:tasks.workunit.client.0.smithi143.stdout:checking for working memcmp... yes 2024-09-10T10:47:18.646 INFO:tasks.workunit.client.0.smithi143.stdout:checking for strftime... yes 2024-09-10T10:47:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:18 smithi178 ceph-mon[34813]: pgmap v418: 129 pgs: 129 active+clean; 2.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 174 KiB/s wr, 26 op/s 2024-09-10T10:47:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:18 smithi144 ceph-mon[34230]: pgmap v418: 129 pgs: 129 active+clean; 2.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 174 KiB/s wr, 26 op/s 2024-09-10T10:47:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:18 smithi143 ceph-mon[31057]: pgmap v418: 129 pgs: 129 active+clean; 2.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 174 KiB/s wr, 26 op/s 2024-09-10T10:47:19.321 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-10T10:47:19.491 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether stat accepts an empty string... no 2024-09-10T10:47:19.595 INFO:tasks.workunit.client.0.smithi143.stdout:checking for vprintf... yes 2024-09-10T10:47:19.687 INFO:tasks.workunit.client.0.smithi143.stdout:checking for _doprnt... no 2024-09-10T10:47:19.715 INFO:tasks.workunit.client.0.smithi143.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T10:47:19.730 INFO:tasks.workunit.client.0.smithi143.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T10:47:20.003 INFO:tasks.workunit.client.0.smithi143.stdout:checking for the pthreads library -lpthreads... no 2024-09-10T10:47:20.076 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether pthreads work without any flags... yes 2024-09-10T10:47:20.140 INFO:tasks.workunit.client.0.smithi143.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-10T10:47:20.140 INFO:tasks.workunit.client.0.smithi143.stdout:checking if more special flags are required for pthreads... no 2024-09-10T10:47:20.140 INFO:tasks.workunit.client.0.smithi143.stdout:checking for cc_r... gcc 2024-09-10T10:47:20.222 INFO:tasks.workunit.client.0.smithi143.stdout:checking for setrlimit... yes 2024-09-10T10:47:20.303 INFO:tasks.workunit.client.0.smithi143.stdout:checking for setproctitle... no 2024-09-10T10:47:20.386 INFO:tasks.workunit.client.0.smithi143.stdout:checking for getopt_long... yes 2024-09-10T10:47:20.480 INFO:tasks.workunit.client.0.smithi143.stdout:checking for strtoull... yes 2024-09-10T10:47:20.563 INFO:tasks.workunit.client.0.smithi143.stdout:checking for strtoq... yes 2024-09-10T10:47:20.648 INFO:tasks.workunit.client.0.smithi143.stdout:checking for snprintf... yes 2024-09-10T10:47:20.733 INFO:tasks.workunit.client.0.smithi143.stdout:checking for vsnprintf... yes 2024-09-10T10:47:20.819 INFO:tasks.workunit.client.0.smithi143.stdout:checking for vfprintf... yes 2024-09-10T10:47:20.901 INFO:tasks.workunit.client.0.smithi143.stdout:checking for gethostname... yes 2024-09-10T10:47:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:20 smithi178 ceph-mon[34813]: pgmap v419: 129 pgs: 129 active+clean; 2.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 202 KiB/s wr, 48 op/s 2024-09-10T10:47:20.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:20 smithi144 ceph-mon[34230]: pgmap v419: 129 pgs: 129 active+clean; 2.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 202 KiB/s wr, 48 op/s 2024-09-10T10:47:20.982 INFO:tasks.workunit.client.0.smithi143.stdout:checking for setlocale... yes 2024-09-10T10:47:21.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:20 smithi143 ceph-mon[31057]: pgmap v419: 129 pgs: 129 active+clean; 2.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 202 KiB/s wr, 48 op/s 2024-09-10T10:47:21.063 INFO:tasks.workunit.client.0.smithi143.stdout:checking for sync... yes 2024-09-10T10:47:21.149 INFO:tasks.workunit.client.0.smithi143.stdout:checking whether snprintf is C99 conformant... done 2024-09-10T10:47:21.165 INFO:tasks.workunit.client.0.smithi143.stdout:configure: creating ./config.status 2024-09-10T10:47:21.396 INFO:tasks.workunit.client.0.smithi143.stdout:config.status: creating Makefile 2024-09-10T10:47:21.418 INFO:tasks.workunit.client.0.smithi143.stdout:config.status: creating src/Makefile 2024-09-10T10:47:21.444 INFO:tasks.workunit.client.0.smithi143.stdout:config.status: creating man/Makefile 2024-09-10T10:47:21.459 INFO:tasks.workunit.client.0.smithi143.stdout:config.status: creating config.h 2024-09-10T10:47:21.528 INFO:tasks.workunit.client.0.smithi143.stdout:config.status: executing depfiles commands 2024-09-10T10:47:21.702 INFO:tasks.workunit.client.0.smithi143.stderr:+ make 2024-09-10T10:47:21.716 INFO:tasks.workunit.client.0.smithi143.stdout:make all-recursive 2024-09-10T10:47:21.723 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T10:47:21.727 INFO:tasks.workunit.client.0.smithi143.stdout:Making all in src 2024-09-10T10:47:21.750 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-10T10:47:21.750 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-10T10:47:21.751 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-10T10:47:21.859 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-10T10:47:21.859 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-10T10:47:21.953 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-10T10:47:21.953 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-10T10:47:22.035 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-10T10:47:22.035 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-10T10:47:22.114 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-10T10:47:22.114 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-10T10:47:22.188 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-10T10:47:22.188 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-10T10:47:22.265 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-10T10:47:22.265 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-10T10:47:22.368 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-10T10:47:22.368 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-10T10:47:22.449 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-10T10:47:22.449 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-10T10:47:22.512 INFO:tasks.workunit.client.0.smithi143.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-10T10:47:22.512 INFO:tasks.workunit.client.0.smithi143.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-10T10:47:22.599 INFO:tasks.workunit.client.0.smithi143.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-10T10:47:22.655 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-10T10:47:22.656 INFO:tasks.workunit.client.0.smithi143.stdout:Making all in man 2024-09-10T10:47:22.667 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-10T10:47:22.667 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Nothing to be done for 'all'. 2024-09-10T10:47:22.667 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-10T10:47:22.675 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T10:47:22.677 INFO:tasks.workunit.client.0.smithi143.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T10:47:22.678 INFO:tasks.workunit.client.0.smithi143.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T10:47:22.679 INFO:tasks.workunit.client.0.smithi143.stderr:+ cd src 2024-09-10T10:47:22.680 INFO:tasks.workunit.client.0.smithi143.stderr:+ mkdir blogtest_in 2024-09-10T10:47:22.682 INFO:tasks.workunit.client.0.smithi143.stdout:running blogbench 2024-09-10T10:47:22.682 INFO:tasks.workunit.client.0.smithi143.stderr:+ echo 'running blogbench' 2024-09-10T10:47:22.682 INFO:tasks.workunit.client.0.smithi143.stderr:+ ./blogbench -d blogtest_in 2024-09-10T10:47:22.683 INFO:tasks.workunit.client.0.smithi143.stdout: 2024-09-10T10:47:22.683 INFO:tasks.workunit.client.0.smithi143.stdout:Frequency = 10 secs 2024-09-10T10:47:22.683 INFO:tasks.workunit.client.0.smithi143.stdout:Scratch dir = [blogtest_in] 2024-09-10T10:47:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:22 smithi178 ceph-mon[34813]: pgmap v420: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 233 KiB/s wr, 76 op/s 2024-09-10T10:47:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:22 smithi144 ceph-mon[34230]: pgmap v420: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 233 KiB/s wr, 76 op/s 2024-09-10T10:47:23.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:22 smithi143 ceph-mon[31057]: pgmap v420: 129 pgs: 129 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 233 KiB/s wr, 76 op/s 2024-09-10T10:47:23.587 INFO:tasks.workunit.client.0.smithi143.stdout:Spawning 3 writers... 2024-09-10T10:47:23.588 INFO:tasks.workunit.client.0.smithi143.stdout:Spawning 1 rewriters... 2024-09-10T10:47:23.588 INFO:tasks.workunit.client.0.smithi143.stdout:Spawning 5 commenters... 2024-09-10T10:47:23.588 INFO:tasks.workunit.client.0.smithi143.stdout:Spawning 100 readers... 2024-09-10T10:47:23.590 INFO:tasks.workunit.client.0.smithi143.stdout:Benchmarking for 30 iterations. 2024-09-10T10:47:23.590 INFO:tasks.workunit.client.0.smithi143.stdout:The test will run during 5 minutes. 2024-09-10T10:47:23.590 INFO:tasks.workunit.client.0.smithi143.stdout: 2024-09-10T10:47:23.590 INFO:tasks.workunit.client.0.smithi143.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-10T10:47:24.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:47:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:24 smithi178 ceph-mon[34813]: pgmap v421: 129 pgs: 129 active+clean; 5.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 469 KiB/s wr, 110 op/s 2024-09-10T10:47:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:24 smithi144 ceph-mon[34230]: pgmap v421: 129 pgs: 129 active+clean; 5.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 469 KiB/s wr, 110 op/s 2024-09-10T10:47:25.034 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:24 smithi143 ceph-mon[31057]: pgmap v421: 129 pgs: 129 active+clean; 5.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 469 KiB/s wr, 110 op/s 2024-09-10T10:47:25.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 -- 172.21.15.143:0/1295635586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 msgr2=0x7f6560101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 --2- 172.21.15.143:0/1295635586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560101200 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6550009900 tx=0x7f655002a480 comp rx=0 tx=0).stop 2024-09-10T10:47:25.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 -- 172.21.15.143:0/1295635586 shutdown_connections 2024-09-10T10:47:25.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 --2- 172.21.15.143:0/1295635586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560102ef0 0x7f656010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 --2- 172.21.15.143:0/1295635586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6560102570 0x7f65601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 --2- 172.21.15.143:0/1295635586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560101200 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 -- 172.21.15.143:0/1295635586 >> 172.21.15.143:0/1295635586 conn(0x7f65600fc7d0 msgr2=0x7f65600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 -- 172.21.15.143:0/1295635586 shutdown_connections 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.534+0000 7f6566205640 1 -- 172.21.15.143:0/1295635586 wait complete. 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 Processor -- start 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 -- start start 2024-09-10T10:47:25.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6560102570 0x7f6560198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560102ef0 0x7f656019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6560110e30 con 0x7f6560100e20 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6560110cb0 con 0x7f6560102ef0 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6566205640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6560110fb0 con 0x7f6560102570 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60350/0 (socket says 172.21.15.143:60350) 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 -- 172.21.15.143:0/4070920408 learned_addr learned my addr 172.21.15.143:0/4070920408 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f6557fff640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6560102570 0x7f6560198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655ffff640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560102ef0 0x7f656019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 -- 172.21.15.143:0/4070920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6560102570 msgr2=0x7f6560198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6560102570 0x7f6560198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 -- 172.21.15.143:0/4070920408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560102ef0 msgr2=0x7f656019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560102ef0 0x7f656019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.535+0000 7f655f7fe640 1 -- 172.21.15.143:0/4070920408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656019d3e0 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.536+0000 7f655f7fe640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560198430 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6550006010 tx=0x7f6550004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.536+0000 7f655d7fa640 1 -- 172.21.15.143:0/4070920408 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655003f030 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.536+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656019d670 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.536+0000 7f655d7fa640 1 -- 172.21.15.143:0/4070920408 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6550003aa0 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.536+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601a4fb0 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.536+0000 7f655d7fa640 1 -- 172.21.15.143:0/4070920408 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550049060 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.538+0000 7f655d7fa640 1 -- 172.21.15.143:0/4070920408 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6550042030 con 0x7f6560100e20 2024-09-10T10:47:25.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.538+0000 7f65685de740 1 Processor -- start 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.538+0000 7f65685de740 1 -- start start 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f65685de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad3fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f65685de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad405140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f65685de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad407a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655d7fa640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f654c0a2b80 0x7f654c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f65685de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eaad3602e0 con 0x55eaad3fadc0 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f65685de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eaad307ce0 con 0x55eaad405680 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f65685de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eaad35c2a0 con 0x55eaad3fa4a0 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad405140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad407a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad3fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad405140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60360/0 (socket says 172.21.15.143:60360) 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655f7fe640 1 -- 172.21.15.143:0/1999266181 learned_addr learned my addr 172.21.15.143:0/1999266181 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655d7fa640 1 -- 172.21.15.143:0/4070920408 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f65500efda0 con 0x7f6560100e20 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655ffff640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f654c0a2b80 0x7f654c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 -- 172.21.15.143:0/1999266181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 msgr2=0x55eaad3fa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad3fa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 -- 172.21.15.143:0/1999266181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 msgr2=0x55eaad405140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad405140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 -- 172.21.15.143:0/1999266181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaad2ec440 con 0x55eaad405680 2024-09-10T10:47:25.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6557fff640 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad407a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f654800bbb0 tx=0x7f654800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd517c9dec56b5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f655ffff640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f654c0a2b80 0x7f654c0a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6558009270 tx=0x7f6558009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.539+0000 7f6555ffb640 1 -- 172.21.15.143:0/1999266181 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548015070 con 0x55eaad405680 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f6555ffb640 1 -- 172.21.15.143:0/1999266181 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65480054d0 con 0x55eaad405680 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f6555ffb640 1 -- 172.21.15.143:0/1999266181 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548011440 con 0x55eaad405680 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 -- 172.21.15.143:0/1999266181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 msgr2=0x55eaad407a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad407a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f654800bbb0 tx=0x7f654800f730 comp rx=0 tx=0).stop 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 -- 172.21.15.143:0/1999266181 shutdown_connections 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad407a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad405140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 --2- 172.21.15.143:0/1999266181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad3fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 -- 172.21.15.143:0/1999266181 >> 172.21.15.143:0/1999266181 conn(0x55eaad3f6a30 msgr2=0x55eaad3f6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 -- 172.21.15.143:0/1999266181 shutdown_connections 2024-09-10T10:47:25.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.540+0000 7f65685de740 1 -- 172.21.15.143:0/1999266181 wait complete. 2024-09-10T10:47:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 -- start start 2024-09-10T10:47:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 Processor -- start 2024-09-10T10:47:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad404f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad402220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad402760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eaad408250 con 0x55eaad3fadc0 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eaad31b050 con 0x55eaad405680 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eaad4083d0 con 0x55eaad3fa4a0 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f65685de740 10 client.0 _ll_register_callbacks cb 0x55eaad35b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad402220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad402220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60364/0 (socket says 172.21.15.143:60364) 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 -- 172.21.15.143:0/2871030974 learned_addr learned my addr 172.21.15.143:0/2871030974 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f655ffff640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad402760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f655f7fe640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad404f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 -- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 msgr2=0x55eaad404f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad404f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 -- 172.21.15.143:0/2871030974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 msgr2=0x55eaad402760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.541+0000 7f6557fff640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad402760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f6557fff640 1 -- 172.21.15.143:0/2871030974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaad0d13c0 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f655f7fe640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad404f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f6557fff640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad402220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6548004990 tx=0x7f65480049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7965f2644665bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f653effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f653effd640 1 -- 172.21.15.143:0/2871030974 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548015070 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65480054d0 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f653effd640 1 -- 172.21.15.143:0/2871030974 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6548004c60 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f653effd640 1 -- 172.21.15.143:0/2871030974 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f654800b7f0 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eaad413bc0 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f653effd640 1 -- 172.21.15.143:0/2871030974 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65480054d0 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f653effd640 1 -- 172.21.15.143:0/2871030974 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f65480119e0 con 0x55eaad3fadc0 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:25.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 20 client.16044 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad415080 0x55eaad417470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 4 client.16044 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.542+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaad36d850 con 0x55eaad415080 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.543+0000 7f655f7fe640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad415080 0x55eaad417470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.543+0000 7f655f7fe640 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad415080 0x55eaad417470 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce493def1be0982c server_cookie=47d83771e8e0ad3f in_seq=0 out_seq=0 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.543+0000 7f653effd640 10 client.16044 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f653effd640 1 -- 172.21.15.143:0/2871030974 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eaad36d850 con 0x55eaad415080 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f653effd640 10 client.16044 handle_command_reply: tid=0 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 client.16044 shutdown 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 msgr2=0x55eaad402220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad402220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6548004990 tx=0x7f65480049c0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 shutdown_connections 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad402760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad402220 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad415080 0x55eaad417470 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 --2- 172.21.15.143:0/2871030974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad404f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f653effd640 0 client.16044 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 >> 172.21.15.143:0/2871030974 conn(0x55eaad3f6a30 msgr2=0x55eaad3fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 shutdown_connections 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.544+0000 7f65685de740 1 -- 172.21.15.143:0/2871030974 wait complete. 2024-09-10T10:47:25.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.545+0000 7f65685de740 20 client.16044 trim_cache size 0 max 16384 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.561+0000 7f65685de740 1 Processor -- start 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 -- start start 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad4114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad3f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad3fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65601ac810 con 0x55eaad3fadc0 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65601ae380 con 0x55eaad405680 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65685de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65601acee0 con 0x55eaad3fa4a0 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad3f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad3f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60370/0 (socket says 172.21.15.143:60370) 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 -- 172.21.15.143:0/1733256030 learned_addr learned my addr 172.21.15.143:0/1733256030 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f655f7fe640 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad4114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 -- 172.21.15.143:0/1733256030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 msgr2=0x55eaad4114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f655ffff640 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad3fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad4114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 -- 172.21.15.143:0/1733256030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 msgr2=0x55eaad3fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad3fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 -- 172.21.15.143:0/1733256030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaad4011d0 con 0x55eaad3fadc0 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f655f7fe640 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad4114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f6557fff640 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad3f7040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f65480048f0 tx=0x7f6548005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f4038a765aa6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.562+0000 7f65557fa640 1 -- 172.21.15.143:0/1733256030 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654800b7b0 con 0x55eaad3fadc0 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65557fa640 1 -- 172.21.15.143:0/1733256030 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65480118f0 con 0x55eaad3fadc0 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65557fa640 1 -- 172.21.15.143:0/1733256030 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548005330 con 0x55eaad3fadc0 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 -- 172.21.15.143:0/1733256030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 msgr2=0x55eaad3f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad3f7040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f65480048f0 tx=0x7f6548005c10 comp rx=0 tx=0).stop 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 -- 172.21.15.143:0/1733256030 shutdown_connections 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad405680 0x55eaad3fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fadc0 0x55eaad3f7040 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 --2- 172.21.15.143:0/1733256030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3fa4a0 0x55eaad4114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 -- 172.21.15.143:0/1733256030 >> 172.21.15.143:0/1733256030 conn(0x55eaad3f6a30 msgr2=0x55eaad3fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 -- 172.21.15.143:0/1733256030 shutdown_connections 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.563+0000 7f65685de740 1 -- 172.21.15.143:0/1733256030 wait complete. 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 -- start start 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 Processor -- start 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 0x55eaad408720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad3fadc0 0x55eaad408c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad405680 0x55eaad47dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65601adc80 con 0x55eaad3fa4a0 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65601ad5b0 con 0x55eaad3fadc0 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65601aa7f0 con 0x55eaad405680 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f65685de740 10 client.0 _ll_register_callbacks cb 0x7f6550005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 0x55eaad408720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 0x55eaad408720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60386/0 (socket says 172.21.15.143:60386) 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 -- 172.21.15.143:0/3662312009 learned_addr learned my addr 172.21.15.143:0/3662312009 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f6557fff640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad3fadc0 0x55eaad408c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655ffff640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad405680 0x55eaad47dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 -- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad405680 msgr2=0x55eaad47dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad405680 0x55eaad47dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 -- 172.21.15.143:0/3662312009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad3fadc0 msgr2=0x55eaad408c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad3fadc0 0x55eaad408c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f655f7fe640 1 -- 172.21.15.143:0/3662312009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaad47e180 con 0x55eaad3fa4a0 2024-09-10T10:47:25.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.564+0000 7f6557fff640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad3fadc0 0x55eaad408c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f655f7fe640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 0x55eaad408720 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f65500ed590 tx=0x7f655008c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2760d6c7f2c401 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f653effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f653effd640 1 -- 172.21.15.143:0/3662312009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550035040 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f653effd640 1 -- 172.21.15.143:0/3662312009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f65500b8910 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65480118f0 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f653effd640 1 -- 172.21.15.143:0/3662312009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65500bbc00 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 10 client.16050 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eaad47f310 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f653effd640 1 -- 172.21.15.143:0/3662312009 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65480118f0 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f653effd640 1 -- 172.21.15.143:0/3662312009 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f655003bcb0 con 0x55eaad3fa4a0 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:25.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 20 client.16050 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad47fab0 0x55eaad481ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 4 client.16050 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.565+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eaad367d60 con 0x55eaad47fab0 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.566+0000 7f6557fff640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad47fab0 0x55eaad481ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.566+0000 7f653effd640 10 client.16050 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.566+0000 7f6557fff640 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad47fab0 0x55eaad481ef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f125034d256724bf server_cookie=5be330e7d1b753c in_seq=0 out_seq=0 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.566+0000 7f653effd640 1 -- 172.21.15.143:0/3662312009 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eaad367d60 con 0x55eaad47fab0 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.566+0000 7f653effd640 10 client.16050 handle_command_reply: tid=0 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.566+0000 7f65685de740 1 client.16050 shutdown 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 msgr2=0x55eaad408720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 0x55eaad408720 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f65500ed590 tx=0x7f655008c150 comp rx=0 tx=0).stop 2024-09-10T10:47:25.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 shutdown_connections 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eaad47fab0 0x55eaad481ef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad405680 0x55eaad47dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eaad3fadc0 0x55eaad408c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 --2- 172.21.15.143:0/3662312009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eaad3fa4a0 0x55eaad408720 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 >> 172.21.15.143:0/3662312009 conn(0x55eaad3f6a30 msgr2=0x55eaad403a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f653effd640 0 client.16050 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 shutdown_connections 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 1 -- 172.21.15.143:0/3662312009 wait complete. 2024-09-10T10:47:25.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.567+0000 7f65685de740 20 client.16050 trim_cache size 0 max 16384 2024-09-10T10:47:25.585 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:47:25.585 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:47:25.585 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:47:25.585 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f654c0a2b80 msgr2=0x7f654c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f654c0a2b80 0x7f654c0a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6558009270 tx=0x7f6558009500 comp rx=0 tx=0).stop 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 msgr2=0x7f6560198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560198430 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6550006010 tx=0x7f6550004270 comp rx=0 tx=0).stop 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 shutdown_connections 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f654c0a2b80 0x7f654c0a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560102ef0 0x7f656019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6560102570 0x7f6560198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 --2- 172.21.15.143:0/4070920408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560100e20 0x7f6560198430 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:25.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 >> 172.21.15.143:0/4070920408 conn(0x7f65600fc7d0 msgr2=0x7f656006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:25.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 shutdown_connections 2024-09-10T10:47:25.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:25.568+0000 7f6566205640 1 -- 172.21.15.143:0/4070920408 wait complete. 2024-09-10T10:47:25.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:25 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:47:25.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:25 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:47:25.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:25 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:47:26.409 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:47:26.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:47:26.654 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:26.776 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:26 smithi143 ceph-mon[31057]: pgmap v422: 129 pgs: 129 active+clean; 5.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 537 KiB/s wr, 127 op/s 2024-09-10T10:47:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:26 smithi178 ceph-mon[34813]: pgmap v422: 129 pgs: 129 active+clean; 5.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 537 KiB/s wr, 127 op/s 2024-09-10T10:47:26.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:26 smithi144 ceph-mon[34230]: pgmap v422: 129 pgs: 129 active+clean; 5.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 537 KiB/s wr, 127 op/s 2024-09-10T10:47:27.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.123+0000 7f2a67aa2640 1 -- 172.21.15.143:0/2964324703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60101400 msgr2=0x7f2a6010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.123+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/2964324703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60101400 0x7f2a6010d930 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009cd0 tx=0x7f2a5802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- 172.21.15.143:0/2964324703 shutdown_connections 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/2964324703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60101400 0x7f2a6010d930 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/2964324703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a60100a60 0x7f2a60100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/2964324703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a60106a60 0x7f2a60106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- 172.21.15.143:0/2964324703 >> 172.21.15.143:0/2964324703 conn(0x7f2a600fc780 msgr2=0x7f2a600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- 172.21.15.143:0/2964324703 shutdown_connections 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- 172.21.15.143:0/2964324703 wait complete. 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 Processor -- start 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- start start 2024-09-10T10:47:27.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 0x7f2a60196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a60101400 0x7f2a60197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a60106a60 0x7f2a60197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60110d10 con 0x7f2a60100a60 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60110b90 con 0x7f2a60101400 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.124+0000 7f2a67aa2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60110e90 con 0x7f2a60106a60 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 0x7f2a60196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 0x7f2a60196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54322/0 (socket says 172.21.15.143:54322) 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 -- 172.21.15.143:0/3746123173 learned_addr learned my addr 172.21.15.143:0/3746123173 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6603f640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a60106a60 0x7f2a60197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6503d640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a60101400 0x7f2a60197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 -- 172.21.15.143:0/3746123173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a60106a60 msgr2=0x7f2a60197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:27.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a60106a60 0x7f2a60197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 -- 172.21.15.143:0/3746123173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a60101400 msgr2=0x7f2a60197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a60101400 0x7f2a60197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 -- 172.21.15.143:0/3746123173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a601a6860 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a6583e640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 0x7f2a60196ff0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00d770 tx=0x7f2a5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017070 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a601a6b50 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a5c0054d0 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a601a7090 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.125+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c004cf0 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.126+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a60100ec0 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.127+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2a5c004e90 con 0x7f2a60100a60 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.128+0000 7f2a56ffd640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2a4c0a2b80 0x7f2a4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:27.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.128+0000 7f2a6503d640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2a4c0a2b80 0x7f2a4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:27.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.128+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2a5c0cf180 con 0x7f2a60100a60 2024-09-10T10:47:27.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.128+0000 7f2a6503d640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2a4c0a2b80 0x7f2a4c0a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a50005fd0 tx=0x7f2a50009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:27.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.131+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a5c09bd80 con 0x7f2a60100a60 2024-09-10T10:47:27.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.332+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a60109030 con 0x7f2a60100a60 2024-09-10T10:47:27.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.332+0000 7f2a56ffd640 1 -- 172.21.15.143:0/3746123173 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11999 (secure 0 0 0) 0x7f2a5c09b680 con 0x7f2a60100a60 2024-09-10T10:47:27.335 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:47:27.335 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":30,"btime":"2024-09-10T10:47:24:562279+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:47:24.105795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:47:27.335 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 30 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2a4c0a2b80 msgr2=0x7f2a4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2a4c0a2b80 0x7f2a4c0a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a50005fd0 tx=0x7f2a50009470 comp rx=0 tx=0).stop 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 msgr2=0x7f2a60196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 0x7f2a60196ff0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00d770 tx=0x7f2a5c00dc30 comp rx=0 tx=0).stop 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 shutdown_connections 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2a4c0a2b80 0x7f2a4c0a5040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a60106a60 0x7f2a60197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a60101400 0x7f2a60197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 --2- 172.21.15.143:0/3746123173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a60100a60 0x7f2a60196ff0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:27.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 >> 172.21.15.143:0/3746123173 conn(0x7f2a600fc780 msgr2=0x7f2a600fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:27.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 shutdown_connections 2024-09-10T10:47:27.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:27.335+0000 7f2a67aa2640 1 -- 172.21.15.143:0/3746123173 wait complete. 2024-09-10T10:47:27.702 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:27 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3746123173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:47:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:27 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3746123173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:47:27.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:27 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3746123173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:47:28.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:47:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:28.734 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:28 smithi143 ceph-mon[31057]: pgmap v423: 129 pgs: 129 active+clean; 9.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 845 KiB/s wr, 149 op/s 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 -- 172.21.15.143:0/1050149995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100d00 msgr2=0x7f3540106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 --2- 172.21.15.143:0/1050149995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100d00 0x7f3540106840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3534009930 tx=0x7f353402a420 comp rx=0 tx=0).stop 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 -- 172.21.15.143:0/1050149995 shutdown_connections 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 --2- 172.21.15.143:0/1050149995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540106f50 0x7f354010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 --2- 172.21.15.143:0/1050149995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100d00 0x7f3540106840 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 --2- 172.21.15.143:0/1050149995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540100350 0x7f3540100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 -- 172.21.15.143:0/1050149995 >> 172.21.15.143:0/1050149995 conn(0x7f3540077210 msgr2=0x7f35400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 -- 172.21.15.143:0/1050149995 shutdown_connections 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 -- 172.21.15.143:0/1050149995 wait complete. 2024-09-10T10:47:28.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 Processor -- start 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.814+0000 7f35465b1640 1 -- start start 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f35465b1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 0x7f354019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f35465b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540100d00 0x7f354019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f35465b1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540106f50 0x7f354019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f35465b1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3540111e50 con 0x7f3540100350 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f35465b1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3540111cd0 con 0x7f3540106f50 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f35465b1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3540111fd0 con 0x7f3540100d00 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 0x7f354019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 0x7f354019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54336/0 (socket says 172.21.15.143:54336) 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 -- 172.21.15.143:0/3819469835 learned_addr learned my addr 172.21.15.143:0/3819469835 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353f7fe640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540100d00 0x7f354019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f3544b4e640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540106f50 0x7f354019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 -- 172.21.15.143:0/3819469835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540100d00 msgr2=0x7f354019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540100d00 0x7f354019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 -- 172.21.15.143:0/3819469835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540106f50 msgr2=0x7f354019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540106f50 0x7f354019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 -- 172.21.15.143:0/3819469835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354019f640 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353ffff640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 0x7f354019a690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f353000d950 tx=0x7f353000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.815+0000 7f353d7fa640 1 -- 172.21.15.143:0/3819469835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530017070 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.816+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354019f930 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.816+0000 7f353d7fa640 1 -- 172.21.15.143:0/3819469835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3530009c30 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.816+0000 7f353d7fa640 1 -- 172.21.15.143:0/3819469835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353000f620 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.816+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401a7210 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.818+0000 7f353d7fa640 1 -- 172.21.15.143:0/3819469835 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f35300109a0 con 0x7f3540100350 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 Processor -- start 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 -- start start 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561623761b40 0x561623760090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623760a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 0x56162376d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616236c62f0 con 0x56162376b370 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56162366dcf0 con 0x5616237605d0 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f354898a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5616236c22b0 con 0x561623761b40 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 0x56162376d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 0x56162376d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54344/0 (socket says 172.21.15.143:54344) 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 -- 172.21.15.143:0/3533241159 learned_addr learned my addr 172.21.15.143:0/3533241159 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f3544b4e640 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561623761b40 0x561623760090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353ffff640 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623760a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353d7fa640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35280a2b80 0x7f35280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 -- 172.21.15.143:0/3533241159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561623761b40 msgr2=0x561623760090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561623761b40 0x561623760090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 -- 172.21.15.143:0/3533241159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 msgr2=0x561623760a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623760a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353f7fe640 1 -- 172.21.15.143:0/3533241159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616236523a0 con 0x56162376b370 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f353ffff640 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623760a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.819+0000 7f3544b4e640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35280a2b80 0x7f35280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f353d7fa640 1 -- 172.21.15.143:0/3819469835 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f35300cef40 con 0x7f3540100350 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f353f7fe640 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 0x56162376d760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3534009900 tx=0x7f3534005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44072571523af20a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f35267fc640 1 -- 172.21.15.143:0/3533241159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534040070 con 0x56162376b370 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f35267fc640 1 -- 172.21.15.143:0/3533241159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35340316a0 con 0x56162376b370 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f35267fc640 1 -- 172.21.15.143:0/3533241159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534035040 con 0x56162376b370 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 -- 172.21.15.143:0/3533241159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 msgr2=0x56162376d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f3544b4e640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35280a2b80 0x7f35280a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3538005fd0 tx=0x7f3538004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 0x56162376d760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3534009900 tx=0x7f3534005c00 comp rx=0 tx=0).stop 2024-09-10T10:47:28.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 -- 172.21.15.143:0/3533241159 shutdown_connections 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56162376b370 0x56162376d760 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623760a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 --2- 172.21.15.143:0/3533241159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561623761b40 0x561623760090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.820+0000 7f354898a740 1 -- 172.21.15.143:0/3533241159 >> 172.21.15.143:0/3533241159 conn(0x561623729310 msgr2=0x56162375cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 -- 172.21.15.143:0/3533241159 shutdown_connections 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 -- 172.21.15.143:0/3533241159 wait complete. 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 -- start start 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 Processor -- start 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616237605d0 0x56162376ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623768020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56162376b370 0x561623768610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56162376e010 con 0x561623761b40 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561623681060 con 0x56162376b370 2024-09-10T10:47:28.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56162376e190 con 0x5616237605d0 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f354898a740 10 client.0 _ll_register_callbacks cb 0x5616233d48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.821+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623768020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623768020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54348/0 (socket says 172.21.15.143:54348) 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 -- 172.21.15.143:0/631585778 learned_addr learned my addr 172.21.15.143:0/631585778 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353ffff640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616237605d0 0x56162376ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f3544b4e640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56162376b370 0x561623768610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 -- 172.21.15.143:0/631585778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616237605d0 msgr2=0x56162376ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616237605d0 0x56162376ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 -- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56162376b370 msgr2=0x561623768610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56162376b370 0x561623768610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 -- 172.21.15.143:0/631585778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616234373c0 con 0x561623761b40 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f353f7fe640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623768020 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3534005020 tx=0x7f3534005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1064a39e3dc765 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:28.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f351f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f351f7fe640 1 -- 172.21.15.143:0/631585778 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353404b020 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f351f7fe640 1 -- 172.21.15.143:0/631585778 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3534005570 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35340316a0 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f351f7fe640 1 -- 172.21.15.143:0/631585778 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3534056050 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f354898a740 10 client.16068 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561623779ab0 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.822+0000 7f351f7fe640 1 -- 172.21.15.143:0/631585778 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35340316a0 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f351f7fe640 1 -- 172.21.15.143:0/631585778 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f353403c6a0 con 0x561623761b40 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f354898a740 10 client.16068 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f354898a740 10 client.16068 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f354898a740 20 client.16068 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f354898a740 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56162377ae00 0x56162377d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f354898a740 4 client.16068 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616236d3860 con 0x56162377ae00 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f353ffff640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56162377ae00 0x56162377d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f351f7fe640 10 client.16068 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.823+0000 7f353ffff640 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56162377ae00 0x56162377d1f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1cb9c9dd600111a5 server_cookie=34f687c1f6bc3db4 in_seq=0 out_seq=0 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f351f7fe640 1 -- 172.21.15.143:0/631585778 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616236d3860 con 0x56162377ae00 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f351f7fe640 10 client.16068 handle_command_reply: tid=0 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 client.16068 shutdown 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 msgr2=0x561623768020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623768020 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3534005020 tx=0x7f3534005050 comp rx=0 tx=0).stop 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 shutdown_connections 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x56162377ae00 0x56162377d1f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56162376b370 0x561623768610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623768020 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f351f7fe640 0 client.16068 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:47:28.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 --2- 172.21.15.143:0/631585778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616237605d0 0x56162376ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 >> 172.21.15.143:0/631585778 conn(0x561623729310 msgr2=0x561623762960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:28.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 shutdown_connections 2024-09-10T10:47:28.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.824+0000 7f354898a740 1 -- 172.21.15.143:0/631585778 wait complete. 2024-09-10T10:47:28.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.825+0000 7f354898a740 20 client.16068 trim_cache size 0 max 16384 2024-09-10T10:47:28.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.837+0000 7f354898a740 1 Processor -- start 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 -- start start 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x5616237772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623763120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x561623763660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f35401b0d30 con 0x561623761b40 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35401b28a0 con 0x5616237605d0 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f354898a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35401b1400 con 0x56162376b370 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623763120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623763120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54354/0 (socket says 172.21.15.143:54354) 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 -- 172.21.15.143:0/1290870389 learned_addr learned my addr 172.21.15.143:0/1290870389 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353ffff640 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x5616237772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f3544b4e640 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x561623763660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 -- 172.21.15.143:0/1290870389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 msgr2=0x561623763660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x561623763660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 -- 172.21.15.143:0/1290870389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 msgr2=0x5616237772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x5616237772b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 -- 172.21.15.143:0/1290870389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162376a990 con 0x561623761b40 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f353f7fe640 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623763120 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3534031440 tx=0x7f3534005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86368491319ada20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.838+0000 7f3525ffb640 1 -- 172.21.15.143:0/1290870389 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353404b070 con 0x561623761b40 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f3525ffb640 1 -- 172.21.15.143:0/1290870389 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3534009520 con 0x561623761b40 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f3525ffb640 1 -- 172.21.15.143:0/1290870389 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534035070 con 0x561623761b40 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 -- 172.21.15.143:0/1290870389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 msgr2=0x561623763120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623763120 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3534031440 tx=0x7f3534005fd0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 -- 172.21.15.143:0/1290870389 shutdown_connections 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x561623763660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x561623763120 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 --2- 172.21.15.143:0/1290870389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x5616237772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 -- 172.21.15.143:0/1290870389 >> 172.21.15.143:0/1290870389 conn(0x561623729310 msgr2=0x561623767110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 -- 172.21.15.143:0/1290870389 shutdown_connections 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 -- 172.21.15.143:0/1290870389 wait complete. 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.839+0000 7f354898a740 1 -- start start 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 Processor -- start 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623773c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x56162376ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x56162376f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f35401b21a0 con 0x561623761b40 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35401b1ad0 con 0x5616237605d0 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35401aed10 con 0x56162376b370 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f354898a740 10 client.0 _ll_register_callbacks cb 0x7f3538009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:28.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x56162376ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x56162376ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54368/0 (socket says 172.21.15.143:54368) 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 -- 172.21.15.143:0/2016847417 learned_addr learned my addr 172.21.15.143:0/2016847417 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353ffff640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623773c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f3544b4e640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x56162376f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 -- 172.21.15.143:0/2016847417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 msgr2=0x56162376f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x56162376f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 -- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 msgr2=0x561623773c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623773c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353f7fe640 1 -- 172.21.15.143:0/2016847417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162376fd00 con 0x561623761b40 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.840+0000 7f353ffff640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623773c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f353f7fe640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x56162376ef50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f353403cac0 tx=0x7f3534005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb40cebc4a9f54b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f351f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f351f7fe640 1 -- 172.21.15.143:0/2016847417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353405e030 con 0x561623761b40 2024-09-10T10:47:28.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3534009520 con 0x561623761b40 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f351f7fe640 1 -- 172.21.15.143:0/2016847417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f353403c440 con 0x561623761b40 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f351f7fe640 1 -- 172.21.15.143:0/2016847417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f353404b070 con 0x561623761b40 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 10 client.16074 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5616237e5830 con 0x561623761b40 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f351f7fe640 1 -- 172.21.15.143:0/2016847417 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3534009520 con 0x561623761b40 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f351f7fe640 1 -- 172.21.15.143:0/2016847417 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f35340444a0 con 0x561623761b40 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 10 client.16074 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 20 client.16074 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5616237e5db0 0x5616237e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 4 client.16074 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.841+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616236cdd70 con 0x5616237e5db0 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f353ffff640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5616237e5db0 0x5616237e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:28.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f351f7fe640 10 client.16074 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f353ffff640 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5616237e5db0 0x5616237e81d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25ecfe0abc8178bc server_cookie=dc99fe11918a19fa in_seq=0 out_seq=0 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f351f7fe640 1 -- 172.21.15.143:0/2016847417 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616236cdd70 con 0x5616237e5db0 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f351f7fe640 10 client.16074 handle_command_reply: tid=0 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 client.16074 shutdown 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 msgr2=0x56162376ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x56162376ef50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f353403cac0 tx=0x7f3534005570 comp rx=0 tx=0).stop 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 shutdown_connections 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5616237e5db0 0x5616237e81d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56162376b370 0x56162376f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561623761b40 0x56162376ef50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 --2- 172.21.15.143:0/2016847417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616237605d0 0x561623773c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f351f7fe640 0 client.16074 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:47:28.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 >> 172.21.15.143:0/2016847417 conn(0x561623729310 msgr2=0x56162375b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:28.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 shutdown_connections 2024-09-10T10:47:28.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.842+0000 7f354898a740 1 -- 172.21.15.143:0/2016847417 wait complete. 2024-09-10T10:47:28.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f354898a740 20 client.16074 trim_cache size 0 max 16384 2024-09-10T10:47:28.857 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35280a2b80 msgr2=0x7f35280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35280a2b80 0x7f35280a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3538005fd0 tx=0x7f3538004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 msgr2=0x7f354019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 0x7f354019a690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f353000d950 tx=0x7f353000de10 comp rx=0 tx=0).stop 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 shutdown_connections 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f35280a2b80 0x7f35280a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540106f50 0x7f354019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540100d00 0x7f354019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 --2- 172.21.15.143:0/3819469835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540100350 0x7f354019a690 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 >> 172.21.15.143:0/3819469835 conn(0x7f3540077210 msgr2=0x7f3540105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:28.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 shutdown_connections 2024-09-10T10:47:28.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:28.843+0000 7f35465b1640 1 -- 172.21.15.143:0/3819469835 wait complete. 2024-09-10T10:47:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:28 smithi178 ceph-mon[34813]: pgmap v423: 129 pgs: 129 active+clean; 9.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 845 KiB/s wr, 149 op/s 2024-09-10T10:47:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:28 smithi144 ceph-mon[34230]: pgmap v423: 129 pgs: 129 active+clean; 9.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 845 KiB/s wr, 149 op/s 2024-09-10T10:47:29.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:47:29.971 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 -- 172.21.15.143:0/4152185053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c0074ee0 msgr2=0x7f68c010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 --2- 172.21.15.143:0/4152185053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c0074ee0 0x7f68c010dee0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009900 tx=0x7f68b002a480 comp rx=0 tx=0).stop 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 -- 172.21.15.143:0/4152185053 shutdown_connections 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 --2- 172.21.15.143:0/4152185053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c010e420 0x7f68c0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 --2- 172.21.15.143:0/4152185053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c0074ee0 0x7f68c010dee0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 --2- 172.21.15.143:0/4152185053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 0x7f68c00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 -- 172.21.15.143:0/4152185053 >> 172.21.15.143:0/4152185053 conn(0x7f68c00fd8e0 msgr2=0x7f68c00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 -- 172.21.15.143:0/4152185053 shutdown_connections 2024-09-10T10:47:30.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.521+0000 7f68c7e55640 1 -- 172.21.15.143:0/4152185053 wait complete. 2024-09-10T10:47:30.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.522+0000 7f68c7e55640 1 Processor -- start 2024-09-10T10:47:30.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.522+0000 7f68c7e55640 1 -- start start 2024-09-10T10:47:30.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c7e55640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 0x7f68c019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c7e55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 0x7f68c019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c7e55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c010e420 0x7f68c019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c7e55640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0114060 con 0x7f68c010e420 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c7e55640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0113ee0 con 0x7f68c0074ee0 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c7e55640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01141e0 con 0x7f68c00745c0 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c53f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 0x7f68c019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c5bf1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 0x7f68c019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c53f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 0x7f68c019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42376/0 (socket says 172.21.15.143:42376) 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c53f0640 1 -- 172.21.15.143:0/3213127019 learned_addr learned my addr 172.21.15.143:0/3213127019 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:30.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c63f2640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c010e420 0x7f68c019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c5bf1640 1 -- 172.21.15.143:0/3213127019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 msgr2=0x7f68c019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 0x7f68c019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c5bf1640 1 -- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c010e420 msgr2=0x7f68c019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c010e420 0x7f68c019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c5bf1640 1 -- 172.21.15.143:0/3213127019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c019f570 con 0x7f68c00745c0 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c53f0640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 0x7f68c019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.523+0000 7f68c63f2640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c010e420 0x7f68c019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.524+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 0x7f68c019a5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00e9f0 tx=0x7f68bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.524+0000 7f68b6ffd640 1 -- 172.21.15.143:0/3213127019 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00ccc0 con 0x7f68c00745c0 2024-09-10T10:47:30.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.524+0000 7f68b6ffd640 1 -- 172.21.15.143:0/3213127019 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68bc0108e0 con 0x7f68c00745c0 2024-09-10T10:47:30.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.524+0000 7f68b6ffd640 1 -- 172.21.15.143:0/3213127019 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc018880 con 0x7f68c00745c0 2024-09-10T10:47:30.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.524+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c019f860 con 0x7f68c00745c0 2024-09-10T10:47:30.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.524+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c01a7140 con 0x7f68c00745c0 2024-09-10T10:47:30.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.527+0000 7f68b6ffd640 1 -- 172.21.15.143:0/3213127019 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f68bc010a90 con 0x7f68c00745c0 2024-09-10T10:47:30.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 Processor -- start 2024-09-10T10:47:30.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 -- start start 2024-09-10T10:47:30.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6cfb40 0x55f77f6ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6d9370 0x55f77f6db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f77f6342f0 con 0x55f77f6ce5d0 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f77f5dbcf0 con 0x55f77f6cfb40 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68ca22e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f77f6302b0 con 0x55f77f6d9370 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68c5bf1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68c5bf1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54402/0 (socket says 172.21.15.143:54402) 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.528+0000 7f68c5bf1640 1 -- 172.21.15.143:0/721975850 learned_addr learned my addr 172.21.15.143:0/721975850 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c63f2640 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6cfb40 0x55f77f6ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6d9370 0x55f77f6db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68b6ffd640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f68ac0a2b80 0x7f68ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 -- 172.21.15.143:0/721975850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6cfb40 msgr2=0x55f77f6ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6cfb40 0x55f77f6ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 -- 172.21.15.143:0/721975850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 msgr2=0x55f77f6cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68b6ffd640 1 -- 172.21.15.143:0/3213127019 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f68bc0cf6f0 con 0x7f68c00745c0 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c63f2640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f68ac0a2b80 0x7f68ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 -- 172.21.15.143:0/721975850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77f5c03a0 con 0x55f77f6d9370 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.529+0000 7f68c53f0640 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6d9370 0x55f77f6db760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f68b0000c00 tx=0x7f68b0004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb9cf04d8a93aca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.530+0000 7f68c63f2640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f68ac0a2b80 0x7f68ac0a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68c006aa00 tx=0x7f68b8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.530+0000 7f68abfff640 1 -- 172.21.15.143:0/721975850 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0040070 con 0x55f77f6d9370 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.530+0000 7f68abfff640 1 -- 172.21.15.143:0/721975850 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b00047a0 con 0x55f77f6d9370 2024-09-10T10:47:30.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.530+0000 7f68abfff640 1 -- 172.21.15.143:0/721975850 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0035040 con 0x55f77f6d9370 2024-09-10T10:47:30.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 -- 172.21.15.143:0/721975850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6d9370 msgr2=0x55f77f6db760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6d9370 0x55f77f6db760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f68b0000c00 tx=0x7f68b0004560 comp rx=0 tx=0).stop 2024-09-10T10:47:30.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 -- 172.21.15.143:0/721975850 shutdown_connections 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6d9370 0x55f77f6db760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 --2- 172.21.15.143:0/721975850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6cfb40 0x55f77f6ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 -- 172.21.15.143:0/721975850 >> 172.21.15.143:0/721975850 conn(0x55f77f697310 msgr2=0x55f77f6cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 -- 172.21.15.143:0/721975850 shutdown_connections 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 -- 172.21.15.143:0/721975850 wait complete. 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.531+0000 7f68ca22e740 1 -- start start 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 Processor -- start 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6d9370 0x55f77f6d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f77f6dc010 con 0x55f77f6ce5d0 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f77f5ef060 con 0x55f77f6d9370 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68c53f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f77f6dc190 con 0x55f77f6cfb40 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68c53f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53092/0 (socket says 172.21.15.143:53092) 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68c53f0640 1 -- 172.21.15.143:0/2148938913 learned_addr learned my addr 172.21.15.143:0/2148938913 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68ca22e740 10 client.0 _ll_register_callbacks cb 0x55f77f3428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68c63f2640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6d9370 0x55f77f6d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.532+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c53f0640 1 -- 172.21.15.143:0/2148938913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6d9370 msgr2=0x55f77f6d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6d9370 0x55f77f6d6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c53f0640 1 -- 172.21.15.143:0/2148938913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 msgr2=0x55f77f6d8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c53f0640 1 -- 172.21.15.143:0/2148938913 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77f3a53c0 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c63f2640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6d9370 0x55f77f6d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d6040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68b002aa70 tx=0x7f68b0004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad5d1732bfe09a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68a97fa640 1 -- 172.21.15.143:0/2148938913 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0040070 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68a97fa640 1 -- 172.21.15.143:0/2148938913 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f68b0003cc0 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b00047a0 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68a97fa640 1 -- 172.21.15.143:0/2148938913 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68b0056050 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68ca22e740 10 client.25513 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.533+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f77f6dc010 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68a97fa640 1 -- 172.21.15.143:0/2148938913 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b00047a0 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68a97fa640 1 -- 172.21.15.143:0/2148938913 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f68b0031710 con 0x55f77f6cfb40 2024-09-10T10:47:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68ca22e740 10 client.25513 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68ca22e740 10 client.25513 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:47:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68ca22e740 20 client.25513 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f6f1450 0x55f77f6f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68ca22e740 4 client.25513 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f77f641860 con 0x55f77f6f1450 2024-09-10T10:47:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.534+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f6f1450 0x55f77f6f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.535+0000 7f68a97fa640 10 client.25513 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.535+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f6f1450 0x55f77f6f3840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=206ca0407567b70e server_cookie=913a4db3509fb0e6 in_seq=0 out_seq=0 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.536+0000 7f68a97fa640 1 -- 172.21.15.143:0/2148938913 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f77f641860 con 0x55f77f6f1450 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.536+0000 7f68a97fa640 10 client.25513 handle_command_reply: tid=0 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 client.25513 shutdown 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 msgr2=0x55f77f6d6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d6040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68b002aa70 tx=0x7f68b0004b80 comp rx=0 tx=0).stop 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 shutdown_connections 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f6f1450 0x55f77f6f3840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6d9370 0x55f77f6d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d6040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2148938913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 >> 172.21.15.143:0/2148938913 conn(0x55f77f697310 msgr2=0x55f77f6d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68a97fa640 0 client.25513 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:47:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 shutdown_connections 2024-09-10T10:47:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 1 -- 172.21.15.143:0/2148938913 wait complete. 2024-09-10T10:47:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.537+0000 7f68ca22e740 20 client.25513 trim_cache size 0 max 16384 2024-09-10T10:47:30.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 Processor -- start 2024-09-10T10:47:30.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 -- start start 2024-09-10T10:47:30.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ac9d0 con 0x55f77f6d9370 2024-09-10T10:47:30.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ae540 con 0x55f77f6ce5d0 2024-09-10T10:47:30.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.554+0000 7f68ca22e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ad0a0 con 0x55f77f6cfb40 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c63f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c63f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54416/0 (socket says 172.21.15.143:54416) 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c63f2640 1 -- 172.21.15.143:0/956859062 learned_addr learned my addr 172.21.15.143:0/956859062 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 -- 172.21.15.143:0/956859062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 msgr2=0x55f77f6edb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 -- 172.21.15.143:0/956859062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 msgr2=0x55f77f6f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 -- 172.21.15.143:0/956859062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77f6d4590 con 0x55f77f6cfb40 2024-09-10T10:47:30.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c63f2640 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68c53f0640 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d5020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005ab0 tx=0x7f68b0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b948a66abc2ba72d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68ab7fe640 1 -- 172.21.15.143:0/956859062 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b005f070 con 0x55f77f6cfb40 2024-09-10T10:47:30.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68ab7fe640 1 -- 172.21.15.143:0/956859062 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b004d070 con 0x55f77f6cfb40 2024-09-10T10:47:30.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.555+0000 7f68ab7fe640 1 -- 172.21.15.143:0/956859062 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0040070 con 0x55f77f6cfb40 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 -- 172.21.15.143:0/956859062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 msgr2=0x55f77f6d5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d5020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005ab0 tx=0x7f68b0005a20 comp rx=0 tx=0).stop 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 -- 172.21.15.143:0/956859062 shutdown_connections 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d5020 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 --2- 172.21.15.143:0/956859062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 -- 172.21.15.143:0/956859062 >> 172.21.15.143:0/956859062 conn(0x55f77f697310 msgr2=0x55f77f6d4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.556+0000 7f68ca22e740 1 -- 172.21.15.143:0/956859062 shutdown_connections 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.557+0000 7f68ca22e740 1 -- 172.21.15.143:0/956859062 wait complete. 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.557+0000 7f68ca22e740 1 -- start start 2024-09-10T10:47:30.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.557+0000 7f68ca22e740 1 Processor -- start 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68ca22e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68c5bf1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68c5bf1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42424/0 (socket says 172.21.15.143:42424) 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68c5bf1640 1 -- 172.21.15.143:0/2848715723 learned_addr learned my addr 172.21.15.143:0/2848715723 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ade40 con 0x55f77f6d9370 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ad770 con 0x55f77f6ce5d0 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.558+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01aa9d0 con 0x55f77f6cfb40 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c63f2640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68ca22e740 10 client.0 _ll_register_callbacks cb 0x7f68b8009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c53f0640 1 -- 172.21.15.143:0/2848715723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 msgr2=0x55f77f6d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c53f0640 1 -- 172.21.15.143:0/2848715723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 msgr2=0x55f77f6d3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6d3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c53f0640 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77f75aea0 con 0x55f77f6cfb40 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c63f2640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6d3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68c53f0640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d3510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005c40 tx=0x7f68b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0263657b8dfad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.559+0000 7f68a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68a97fa640 1 -- 172.21.15.143:0/2848715723 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0040070 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68a97fa640 1 -- 172.21.15.143:0/2848715723 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f68b0004f40 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b004d070 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68a97fa640 1 -- 172.21.15.143:0/2848715723 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68b005f070 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 10 client.25519 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f77f75c0e0 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68a97fa640 1 -- 172.21.15.143:0/2848715723 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b00047a0 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68a97fa640 1 -- 172.21.15.143:0/2848715723 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f68b003c5b0 con 0x55f77f6cfb40 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 10 client.25519 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 10 client.25519 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 20 client.25519 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:30.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f75c740 0x55f77f75eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 4 client.25519 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.560+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f68b004d070 con 0x55f77f75c740 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.561+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f75c740 0x55f77f75eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.561+0000 7f68c5bf1640 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f75c740 0x55f77f75eb60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9f40c53a2dd5837 server_cookie=e22ff3d9eba88384 in_seq=0 out_seq=0 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.561+0000 7f68a97fa640 10 client.25519 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.561+0000 7f68a97fa640 1 -- 172.21.15.143:0/2848715723 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f68b004d070 con 0x55f77f75c740 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.561+0000 7f68a97fa640 10 client.25519 handle_command_reply: tid=0 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.561+0000 7f68ca22e740 1 client.25519 shutdown 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 msgr2=0x55f77f6d3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d3510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f68b0005c40 tx=0x7f68b0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 shutdown_connections 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55f77f75c740 0x55f77f75eb60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f77f6d9370 0x55f77f6d3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f77f6cfb40 0x55f77f6d3510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 --2- 172.21.15.143:0/2848715723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77f6ce5d0 0x55f77f6d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 >> 172.21.15.143:0/2848715723 conn(0x55f77f697310 msgr2=0x55f77f6d78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68a97fa640 0 client.25519 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 shutdown_connections 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 1 -- 172.21.15.143:0/2848715723 wait complete. 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.562+0000 7f68ca22e740 20 client.25519 trim_cache size 0 max 16384 2024-09-10T10:47:30.567 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f68ac0a2b80 msgr2=0x7f68ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f68ac0a2b80 0x7f68ac0a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68c006aa00 tx=0x7f68b8004ab0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 msgr2=0x7f68c019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 0x7f68c019a5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00e9f0 tx=0x7f68bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 shutdown_connections 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f68ac0a2b80 0x7f68ac0a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68c010e420 0x7f68c019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68c0074ee0 0x7f68c019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 --2- 172.21.15.143:0/3213127019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68c00745c0 0x7f68c019a5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 >> 172.21.15.143:0/3213127019 conn(0x7f68c00fd8e0 msgr2=0x7f68c00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 shutdown_connections 2024-09-10T10:47:30.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:30.563+0000 7f68c7e55640 1 -- 172.21.15.143:0/3213127019 wait complete. 2024-09-10T10:47:30.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:30 smithi143 ceph-mon[31057]: pgmap v424: 129 pgs: 129 active+clean; 11 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 834 KiB/s wr, 143 op/s 2024-09-10T10:47:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:30 smithi178 ceph-mon[34813]: pgmap v424: 129 pgs: 129 active+clean; 11 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 834 KiB/s wr, 143 op/s 2024-09-10T10:47:30.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:30 smithi144 ceph-mon[34230]: pgmap v424: 129 pgs: 129 active+clean; 11 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 834 KiB/s wr, 143 op/s 2024-09-10T10:47:31.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:47:31.653 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:32.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.215+0000 7f96e959a640 1 -- 172.21.15.143:0/4044701953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc031830 con 0x7f96ec100e20 2024-09-10T10:47:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 -- 172.21.15.143:0/4044701953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec100e20 msgr2=0x7f96ec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 --2- 172.21.15.143:0/4044701953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec100e20 0x7f96ec101200 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009900 tx=0x7f96dc02a480 comp rx=0 tx=0).stop 2024-09-10T10:47:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 -- 172.21.15.143:0/4044701953 shutdown_connections 2024-09-10T10:47:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 --2- 172.21.15.143:0/4044701953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec102ef0 0x7f96ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 --2- 172.21.15.143:0/4044701953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ec102570 0x7f96ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 --2- 172.21.15.143:0/4044701953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec100e20 0x7f96ec101200 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 -- 172.21.15.143:0/4044701953 >> 172.21.15.143:0/4044701953 conn(0x7f96ec0fc7d0 msgr2=0x7f96ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:32.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 -- 172.21.15.143:0/4044701953 shutdown_connections 2024-09-10T10:47:32.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.216+0000 7f96f0c09640 1 -- 172.21.15.143:0/4044701953 wait complete. 2024-09-10T10:47:32.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.217+0000 7f96f0c09640 1 Processor -- start 2024-09-10T10:47:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 -- start start 2024-09-10T10:47:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec100e20 0x7f96ec10b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 0x7f96ec10bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ec102ef0 0x7f96ec10cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec10f300 con 0x7f96ec102570 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec10f180 con 0x7f96ec102ef0 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96f0c09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec10f480 con 0x7f96ec100e20 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 0x7f96ec10bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 0x7f96ec10bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54434/0 (socket says 172.21.15.143:54434) 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2293054269 learned_addr learned my addr 172.21.15.143:0/2293054269 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96ea59c640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec100e20 0x7f96ec10b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96ead9d640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ec102ef0 0x7f96ec10cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2293054269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec100e20 msgr2=0x7f96ec10b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec100e20 0x7f96ec10b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2293054269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ec102ef0 msgr2=0x7f96ec10cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.218+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ec102ef0 0x7f96ec10cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2293054269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec10d6f0 con 0x7f96ec102570 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96ea59c640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec100e20 0x7f96ec10b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 0x7f96ec10bcc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f96e00097d0 tx=0x7f96e0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96db7fe640 1 -- 172.21.15.143:0/2293054269 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e000eab0 con 0x7f96ec102570 2024-09-10T10:47:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec10d9e0 con 0x7f96ec102570 2024-09-10T10:47:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec1ab6f0 con 0x7f96ec102570 2024-09-10T10:47:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96db7fe640 1 -- 172.21.15.143:0/2293054269 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e0019070 con 0x7f96ec102570 2024-09-10T10:47:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.219+0000 7f96db7fe640 1 -- 172.21.15.143:0/2293054269 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0013380 con 0x7f96ec102570 2024-09-10T10:47:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.222+0000 7f96db7fe640 1 -- 172.21.15.143:0/2293054269 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f96e000ec50 con 0x7f96ec102570 2024-09-10T10:47:32.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.222+0000 7f96f2fe2740 1 Processor -- start 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 -- start start 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96db7fe640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96d40a2c00 0x7f96d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0865a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b0865aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b08665370 0x560b08667760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560b085c02f0 con 0x560b0865a5d0 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b08567cf0 con 0x560b0865bb40 2024-09-10T10:47:32.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b085bc2b0 con 0x560b08665370 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96db7fe640 1 -- 172.21.15.143:0/2293054269 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f96e00cf100 con 0x7f96ec102570 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b0865aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b0865aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54446/0 (socket says 172.21.15.143:54446) 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96e9d9b640 1 -- 172.21.15.143:0/558169007 learned_addr learned my addr 172.21.15.143:0/558169007 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ead9d640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b08665370 0x560b08667760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ea59c640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0865a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ea59c640 1 -- 172.21.15.143:0/558169007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b08665370 msgr2=0x560b08667760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ea59c640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b08665370 0x560b08667760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ea59c640 1 -- 172.21.15.143:0/558169007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 msgr2=0x560b0865aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ea59c640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b0865aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96ea59c640 1 -- 172.21.15.143:0/558169007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0854c3a0 con 0x560b0865bb40 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.223+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b0865aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96ead9d640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b08665370 0x560b08667760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96ead9d640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96d40a2c00 0x7f96d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96ea59c640 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0865a090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96dc005940 tx=0x7f96dc032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994b53755b7da940 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96d8ff9640 1 -- 172.21.15.143:0/558169007 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc035040 con 0x560b0865bb40 2024-09-10T10:47:32.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96d8ff9640 1 -- 172.21.15.143:0/558169007 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96dc0316b0 con 0x560b0865bb40 2024-09-10T10:47:32.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96d8ff9640 1 -- 172.21.15.143:0/558169007 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc003dc0 con 0x560b0865bb40 2024-09-10T10:47:32.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.224+0000 7f96ead9d640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96d40a2c00 0x7f96d40a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96e4005c40 tx=0x7f96e4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:32.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 -- 172.21.15.143:0/558169007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 msgr2=0x560b0865a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0865a090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96dc005940 tx=0x7f96dc032e60 comp rx=0 tx=0).stop 2024-09-10T10:47:32.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 -- 172.21.15.143:0/558169007 shutdown_connections 2024-09-10T10:47:32.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b08665370 0x560b08667760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b0865aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/558169007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0865a090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 -- 172.21.15.143:0/558169007 >> 172.21.15.143:0/558169007 conn(0x560b08623310 msgr2=0x560b08656c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:32.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 -- 172.21.15.143:0/558169007 shutdown_connections 2024-09-10T10:47:32.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 -- 172.21.15.143:0/558169007 wait complete. 2024-09-10T10:47:32.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.225+0000 7f96f2fe2740 1 -- start start 2024-09-10T10:47:32.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 Processor -- start 2024-09-10T10:47:32.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b08664ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 0x560b08661f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08665370 0x560b08662490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560b08668010 con 0x560b0865bb40 2024-09-10T10:47:32.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560b0857b060 con 0x560b08665370 2024-09-10T10:47:32.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b08668190 con 0x560b0865a5d0 2024-09-10T10:47:32.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96f2fe2740 10 client.0 _ll_register_callbacks cb 0x560b082ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:32.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 0x560b08661f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 0x560b08661f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54448/0 (socket says 172.21.15.143:54448) 2024-09-10T10:47:32.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96e9d9b640 1 -- 172.21.15.143:0/3902973228 learned_addr learned my addr 172.21.15.143:0/3902973228 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:32.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.226+0000 7f96ea59c640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b08664ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96ead9d640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08665370 0x560b08662490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96e9d9b640 1 -- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 msgr2=0x560b08664ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b08664ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96e9d9b640 1 -- 172.21.15.143:0/3902973228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08665370 msgr2=0x560b08662490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08665370 0x560b08662490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96e9d9b640 1 -- 172.21.15.143:0/3902973228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b083313c0 con 0x560b0865bb40 2024-09-10T10:47:32.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 0x560b08661f50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f96e00cc8f0 tx=0x7f96e009b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f683d48bb7e2e378 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:32.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:32.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96d1ffb640 1 -- 172.21.15.143:0/3902973228 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00b4060 con 0x560b0865bb40 2024-09-10T10:47:32.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96d1ffb640 1 -- 172.21.15.143:0/3902973228 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f96e0084b90 con 0x560b0865bb40 2024-09-10T10:47:32.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96dc0316b0 con 0x560b0865bb40 2024-09-10T10:47:32.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96d1ffb640 1 -- 172.21.15.143:0/3902973228 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96e009f9b0 con 0x560b0865bb40 2024-09-10T10:47:32.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96f2fe2740 10 client.16101 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:32.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.227+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b08673a60 con 0x560b0865bb40 2024-09-10T10:47:32.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96d1ffb640 1 -- 172.21.15.143:0/3902973228 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e00a4270 con 0x560b0865bb40 2024-09-10T10:47:32.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96d1ffb640 1 -- 172.21.15.143:0/3902973228 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f96e0097350 con 0x560b0865bb40 2024-09-10T10:47:32.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96f2fe2740 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:32.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96f2fe2740 10 client.16101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:47:32.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96f2fe2740 20 client.16101 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:32.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08674ec0 0x560b08677300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96f2fe2740 4 client.16101 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:32.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f96dc0316b0 con 0x560b08674ec0 2024-09-10T10:47:32.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.228+0000 7f96ea59c640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08674ec0 0x560b08677300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.229+0000 7f96d1ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:47:32.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.229+0000 7f96ea59c640 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08674ec0 0x560b08677300 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba54f050d389b6bc server_cookie=c6862a728a8438b9 in_seq=0 out_seq=0 2024-09-10T10:47:32.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.229+0000 7f96d1ffb640 1 -- 172.21.15.143:0/3902973228 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f96dc0316b0 con 0x560b08674ec0 2024-09-10T10:47:32.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.229+0000 7f96d1ffb640 10 client.16101 handle_command_reply: tid=0 2024-09-10T10:47:32.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.229+0000 7f96f2fe2740 1 client.16101 shutdown 2024-09-10T10:47:32.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 msgr2=0x560b08661f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 0x560b08661f50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f96e00cc8f0 tx=0x7f96e009b3d0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 shutdown_connections 2024-09-10T10:47:32.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08674ec0 0x560b08677300 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08665370 0x560b08662490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865bb40 0x560b08661f50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96d1ffb640 0 client.16101 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:47:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/3902973228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b08664ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 >> 172.21.15.143:0/3902973228 conn(0x560b08623310 msgr2=0x560b0865c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 shutdown_connections 2024-09-10T10:47:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 1 -- 172.21.15.143:0/3902973228 wait complete. 2024-09-10T10:47:32.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.230+0000 7f96f2fe2740 20 client.16101 trim_cache size 0 max 16384 2024-09-10T10:47:32.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 Processor -- start 2024-09-10T10:47:32.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 -- start start 2024-09-10T10:47:32.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08623310 0x560b08661810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b08663ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865bb40 0x560b0865d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc00fa50 con 0x560b0865a5d0 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc0115c0 con 0x560b08623310 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc010120 con 0x560b0865bb40 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b08663ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08623310 0x560b08661810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b08663ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54450/0 (socket says 172.21.15.143:54450) 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96e9d9b640 1 -- 172.21.15.143:0/894827036 learned_addr learned my addr 172.21.15.143:0/894827036 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.247+0000 7f96ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08623310 0x560b08661810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42488/0 (socket says 172.21.15.143:42488) 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96ead9d640 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865bb40 0x560b0865d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96e9d9b640 1 -- 172.21.15.143:0/894827036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865bb40 msgr2=0x560b0865d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865bb40 0x560b0865d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96e9d9b640 1 -- 172.21.15.143:0/894827036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08623310 msgr2=0x560b08661810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08623310 0x560b08661810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96e9d9b640 1 -- 172.21.15.143:0/894827036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0865b300 con 0x560b0865a5d0 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b08663ec0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f96e0000c00 tx=0x7f96e0099b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356502b29f94d92a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96d3fff640 1 -- 172.21.15.143:0/894827036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00b4040 con 0x560b0865a5d0 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96d3fff640 1 -- 172.21.15.143:0/894827036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e00a4270 con 0x560b0865a5d0 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.248+0000 7f96d3fff640 1 -- 172.21.15.143:0/894827036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0084b90 con 0x560b0865a5d0 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 -- 172.21.15.143:0/894827036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 msgr2=0x560b08663ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b08663ec0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f96e0000c00 tx=0x7f96e0099b10 comp rx=0 tx=0).stop 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 -- 172.21.15.143:0/894827036 shutdown_connections 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865bb40 0x560b0865d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b0865a5d0 0x560b08663ec0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/894827036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b08623310 0x560b08661810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 -- 172.21.15.143:0/894827036 >> 172.21.15.143:0/894827036 conn(0x560b086625f0 msgr2=0x560b086637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 -- 172.21.15.143:0/894827036 shutdown_connections 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.249+0000 7f96f2fe2740 1 -- 172.21.15.143:0/894827036 wait complete. 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 -- start start 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 Processor -- start 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b08623310 0x560b0866d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b0866db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0866a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc010ec0 con 0x560b08623310 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc0107f0 con 0x560b0865bb40 2024-09-10T10:47:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc00da30 con 0x560b0865a5d0 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96f2fe2740 10 client.0 _ll_register_callbacks cb 0x7f96e009b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b0866db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b0866db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53186/0 (socket says 172.21.15.143:53186) 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.250+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2018486740 learned_addr learned my addr 172.21.15.143:0/2018486740 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96ea59c640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b08623310 0x560b0866d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96ead9d640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0866a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2018486740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 msgr2=0x560b0866a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0866a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2018486740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b08623310 msgr2=0x560b0866d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b08623310 0x560b0866d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96e9d9b640 1 -- 172.21.15.143:0/2018486740 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0866ab10 con 0x560b0865a5d0 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96ea59c640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b08623310 0x560b0866d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96e9d9b640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b0866db70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96e00a3d40 tx=0x7f96e009b500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b91cebcd4f091cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96d1ffb640 1 -- 172.21.15.143:0/2018486740 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00b4040 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e00a4270 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96d1ffb640 1 -- 172.21.15.143:0/2018486740 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f96e00847b0 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96d1ffb640 1 -- 172.21.15.143:0/2018486740 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96e0084b90 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96f2fe2740 10 client.25537 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.251+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b0866ab10 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96d1ffb640 1 -- 172.21.15.143:0/2018486740 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e00a4270 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96d1ffb640 1 -- 172.21.15.143:0/2018486740 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f96e00b41e0 con 0x560b0865a5d0 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96f2fe2740 10 client.25537 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96f2fe2740 10 client.25537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96f2fe2740 20 client.25537 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08665370 0x560b086e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96f2fe2740 4 client.25537 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b085cd860 con 0x560b08665370 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96ea59c640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08665370 0x560b086e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96ea59c640 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08665370 0x560b086e2850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6540ed4e7e35744c server_cookie=adb049066a562ebb in_seq=0 out_seq=0 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.252+0000 7f96d1ffb640 10 client.25537 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96d1ffb640 1 -- 172.21.15.143:0/2018486740 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b085cd860 con 0x560b08665370 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96d1ffb640 10 client.25537 handle_command_reply: tid=0 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 client.25537 shutdown 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 msgr2=0x560b0866db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b0866db70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96e00a3d40 tx=0x7f96e009b500 comp rx=0 tx=0).stop 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 shutdown_connections 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x560b08665370 0x560b086e2850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b0865bb40 0x560b0866a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b0865a5d0 0x560b0866db70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 --2- 172.21.15.143:0/2018486740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560b08623310 0x560b0866d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 >> 172.21.15.143:0/2018486740 conn(0x560b086625f0 msgr2=0x560b086637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96d1ffb640 0 client.25537 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 shutdown_connections 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 1 -- 172.21.15.143:0/2018486740 wait complete. 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.253+0000 7f96f2fe2740 20 client.25537 trim_cache size 0 max 16384 2024-09-10T10:47:32.261 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:47:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.254+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96d40a2c00 msgr2=0x7f96d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.254+0000 7f96f0c09640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96d40a2c00 0x7f96d40a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96e4005c40 tx=0x7f96e4004ab0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.254+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 msgr2=0x7f96ec10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.254+0000 7f96f0c09640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 0x7f96ec10bcc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f96e00097d0 tx=0x7f96e0009c90 comp rx=0 tx=0).stop 2024-09-10T10:47:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 shutdown_connections 2024-09-10T10:47:32.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96d40a2c00 0x7f96d40a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ec102ef0 0x7f96ec10cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96ec102570 0x7f96ec10bcc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 --2- 172.21.15.143:0/2293054269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec100e20 0x7f96ec10b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 >> 172.21.15.143:0/2293054269 conn(0x7f96ec0fc7d0 msgr2=0x7f96ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 shutdown_connections 2024-09-10T10:47:32.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:32.255+0000 7f96f0c09640 1 -- 172.21.15.143:0/2293054269 wait complete. 2024-09-10T10:47:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:32 smithi178 ceph-mon[34813]: pgmap v425: 129 pgs: 129 active+clean; 14 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T10:47:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:32 smithi144 ceph-mon[34230]: pgmap v425: 129 pgs: 129 active+clean; 14 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T10:47:32.982 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:32 smithi143 ceph-mon[31057]: pgmap v425: 129 pgs: 129 active+clean; 14 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T10:47:33.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:47:33.372 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:33.590 INFO:tasks.workunit.client.0.smithi143.stdout: 3 1953 413 1543 177 4510 233 2024-09-10T10:47:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 -- 172.21.15.143:0/1576857673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c106f10 msgr2=0x7ff24c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 --2- 172.21.15.143:0/1576857673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c106f10 0x7ff24c10ea90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009900 tx=0x7ff23c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 -- 172.21.15.143:0/1576857673 shutdown_connections 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 --2- 172.21.15.143:0/1576857673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c106f10 0x7ff24c10ea90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 --2- 172.21.15.143:0/1576857673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c102790 0x7ff24c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 --2- 172.21.15.143:0/1576857673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c101de0 0x7ff24c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.918+0000 7ff252270640 1 -- 172.21.15.143:0/1576857673 >> 172.21.15.143:0/1576857673 conn(0x7ff24c0fd880 msgr2=0x7ff24c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.919+0000 7ff252270640 1 -- 172.21.15.143:0/1576857673 shutdown_connections 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.919+0000 7ff252270640 1 -- 172.21.15.143:0/1576857673 wait complete. 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.919+0000 7ff252270640 1 Processor -- start 2024-09-10T10:47:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.919+0000 7ff252270640 1 -- start start 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff252270640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c101de0 0x7ff24c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff252270640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 0x7ff24c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff252270640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c106f10 0x7ff24c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff252270640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c111e20 con 0x7ff24c101de0 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff252270640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c111ca0 con 0x7ff24c106f10 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff252270640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c111fa0 con 0x7ff24c102790 2024-09-10T10:47:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 0x7ff24c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 0x7ff24c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53206/0 (socket says 172.21.15.143:53206) 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff24b7fe640 1 -- 172.21.15.143:0/3284025230 learned_addr learned my addr 172.21.15.143:0/3284025230 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff24bfff640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c101de0 0x7ff24c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.920+0000 7ff25080d640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c106f10 0x7ff24c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff24b7fe640 1 -- 172.21.15.143:0/3284025230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c106f10 msgr2=0x7ff24c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff24b7fe640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c106f10 0x7ff24c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff24b7fe640 1 -- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c101de0 msgr2=0x7ff24c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff24b7fe640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c101de0 0x7ff24c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff24b7fe640 1 -- 172.21.15.143:0/3284025230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19d290 con 0x7ff24c102790 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff25080d640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c106f10 0x7ff24c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff24b7fe640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 0x7ff24c198820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff24000e9f0 tx=0x7ff24000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff2497fa640 1 -- 172.21.15.143:0/3284025230 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24000cd80 con 0x7ff24c102790 2024-09-10T10:47:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff2497fa640 1 -- 172.21.15.143:0/3284025230 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2400108e0 con 0x7ff24c102790 2024-09-10T10:47:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff2497fa640 1 -- 172.21.15.143:0/3284025230 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240018920 con 0x7ff24c102790 2024-09-10T10:47:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.921+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19d580 con 0x7ff24c102790 2024-09-10T10:47:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.922+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c19d9b0 con 0x7ff24c102790 2024-09-10T10:47:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.925+0000 7ff2497fa640 1 -- 172.21.15.143:0/3284025230 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff240010a90 con 0x7ff24c102790 2024-09-10T10:47:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.925+0000 7ff254649740 1 Processor -- start 2024-09-10T10:47:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.925+0000 7ff254649740 1 -- start start 2024-09-10T10:47:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff254649740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 0x562f60ad5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff25080d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 0x562f60ad5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff25080d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 0x562f60ad5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42510/0 (socket says 172.21.15.143:42510) 2024-09-10T10:47:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff254649740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad55d0 0x562f60ad5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff254649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60ae2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff254649740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562f60a3b2f0 con 0x562f60ad55d0 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff254649740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f609e2cf0 con 0x562f60ad6b40 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff254649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f60a372b0 con 0x562f60ae0370 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.926+0000 7ff25080d640 1 -- 172.21.15.143:0/1332412360 learned_addr learned my addr 172.21.15.143:0/1332412360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff24bfff640 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad55d0 0x562f60ad5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff25080d640 1 -- 172.21.15.143:0/1332412360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 msgr2=0x562f60ae2760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:47:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff25080d640 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60ae2760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff25080d640 1 -- 172.21.15.143:0/1332412360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad55d0 msgr2=0x562f60ad5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff2497fa640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2340a2cd0 0x7ff2340a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff25080d640 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad55d0 0x562f60ad5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff25080d640 1 -- 172.21.15.143:0/1332412360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f609c73a0 con 0x562f60ad6b40 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff2497fa640 1 -- 172.21.15.143:0/3284025230 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff2400cf5c0 con 0x7ff24c102790 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff24bfff640 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad55d0 0x562f60ad5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.927+0000 7ff25080d640 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 0x562f60ad5090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009900 tx=0x7ff23c032b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4f3d2d4737ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.928+0000 7ff2327fc640 1 -- 172.21.15.143:0/1332412360 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c035050 con 0x562f60ad6b40 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.928+0000 7ff2327fc640 1 -- 172.21.15.143:0/1332412360 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c032cf0 con 0x562f60ad6b40 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.928+0000 7ff2327fc640 1 -- 172.21.15.143:0/1332412360 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0043f0 con 0x562f60ad6b40 2024-09-10T10:47:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.928+0000 7ff24b7fe640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2340a2cd0 0x7ff2340a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.928+0000 7ff254649740 1 -- 172.21.15.143:0/1332412360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 msgr2=0x562f60ad5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 0x562f60ad5090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009900 tx=0x7ff23c032b00 comp rx=0 tx=0).stop 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 -- 172.21.15.143:0/1332412360 shutdown_connections 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60ae2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad55d0 0x562f60ad5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 --2- 172.21.15.143:0/1332412360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad6b40 0x562f60ad5090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 -- 172.21.15.143:0/1332412360 >> 172.21.15.143:0/1332412360 conn(0x562f60a9e310 msgr2=0x562f60ad1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff24b7fe640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2340a2cd0 0x7ff2340a5190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff24000c6a0 tx=0x7ff240097640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:33.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 -- 172.21.15.143:0/1332412360 shutdown_connections 2024-09-10T10:47:33.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.929+0000 7ff254649740 1 -- 172.21.15.143:0/1332412360 wait complete. 2024-09-10T10:47:33.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 -- start start 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 Processor -- start 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 0x562f60adfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60adcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ae0370 0x562f60add4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562f60ae2fb0 con 0x562f60ad6b40 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f609f6060 con 0x562f60ae0370 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.930+0000 7ff254649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f60ae3130 con 0x562f60ad55d0 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff254649740 10 client.0 _ll_register_callbacks cb 0x562f607498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 0x562f60adfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60adcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 0x562f60adfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53230/0 (socket says 172.21.15.143:53230) 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 -- 172.21.15.143:0/2288131396 learned_addr learned my addr 172.21.15.143:0/2288131396 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:33.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ae0370 0x562f60add4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 -- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ae0370 msgr2=0x562f60add4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ae0370 0x562f60add4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 -- 172.21.15.143:0/2288131396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 msgr2=0x562f60adcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60adcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 -- 172.21.15.143:0/2288131396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f607ac3c0 con 0x562f60ad55d0 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff24bfff640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60adcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff25080d640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 0x562f60adfc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff23c031440 tx=0x7ff23c031a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddbc009eccacfee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:33.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.931+0000 7ff22b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff22b7fe640 1 -- 172.21.15.143:0/2288131396 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c05a060 con 0x562f60ad55d0 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23c032cf0 con 0x562f60ad55d0 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff22b7fe640 1 -- 172.21.15.143:0/2288131396 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff23c005250 con 0x562f60ad55d0 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff22b7fe640 1 -- 172.21.15.143:0/2288131396 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff23c035030 con 0x562f60ad55d0 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff254649740 10 client.25546 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f60ae2fb0 con 0x562f60ad55d0 2024-09-10T10:47:33.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff22b7fe640 1 -- 172.21.15.143:0/2288131396 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c032cf0 con 0x562f60ad55d0 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.932+0000 7ff22b7fe640 1 -- 172.21.15.143:0/2288131396 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7ff23c0035f0 con 0x562f60ad55d0 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff254649740 10 client.25546 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff254649740 10 client.25546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff254649740 20 client.25546 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff254649740 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60aeff00 0x562f60af2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff254649740 4 client.25546 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f60a48860 con 0x562f60aeff00 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff24bfff640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60aeff00 0x562f60af2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff24bfff640 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60aeff00 0x562f60af2320 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b050ef1261be54e5 server_cookie=c1f0aeb538fc76be in_seq=0 out_seq=0 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.933+0000 7ff22b7fe640 10 client.25546 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff22b7fe640 1 -- 172.21.15.143:0/2288131396 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f60a48860 con 0x562f60aeff00 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff22b7fe640 10 client.25546 handle_command_reply: tid=0 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 client.25546 shutdown 2024-09-10T10:47:33.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 msgr2=0x562f60adfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 0x562f60adfc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff23c031440 tx=0x7ff23c031a30 comp rx=0 tx=0).stop 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 shutdown_connections 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60aeff00 0x562f60af2320 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ae0370 0x562f60add4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60adcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 --2- 172.21.15.143:0/2288131396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad55d0 0x562f60adfc60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 >> 172.21.15.143:0/2288131396 conn(0x562f60a9e310 msgr2=0x562f60ad7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff22b7fe640 0 client.25546 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.934+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 shutdown_connections 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.935+0000 7ff254649740 1 -- 172.21.15.143:0/2288131396 wait complete. 2024-09-10T10:47:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.935+0000 7ff254649740 20 client.25546 trim_cache size 0 max 16384 2024-09-10T10:47:33.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.960+0000 7ff254649740 1 Processor -- start 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.960+0000 7ff254649740 1 -- start start 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.960+0000 7ff254649740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60aec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.960+0000 7ff254649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad6b40 0x562f60ad1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.960+0000 7ff254649740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ae0370 0x562f60ad1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff254649740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1ac900 con 0x562f60ae0370 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff254649740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1ae470 con 0x562f60ad55d0 2024-09-10T10:47:33.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff254649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1acfd0 con 0x562f60ad6b40 2024-09-10T10:47:33.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60aec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60aec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42532/0 (socket says 172.21.15.143:42532) 2024-09-10T10:47:33.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 -- 172.21.15.143:0/355449844 learned_addr learned my addr 172.21.15.143:0/355449844 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:33.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ae0370 0x562f60ad1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff24bfff640 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad6b40 0x562f60ad1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 -- 172.21.15.143:0/355449844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad6b40 msgr2=0x562f60ad1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad6b40 0x562f60ad1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 -- 172.21.15.143:0/355449844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ae0370 msgr2=0x562f60ad1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ae0370 0x562f60ad1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 -- 172.21.15.143:0/355449844 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f60ad14f0 con 0x562f60ad55d0 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.961+0000 7ff25080d640 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60aec270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff23c031860 tx=0x7ff23c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f22de43b7d221c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.962+0000 7ff231ffb640 1 -- 172.21.15.143:0/355449844 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c05a020 con 0x562f60ad55d0 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.962+0000 7ff231ffb640 1 -- 172.21.15.143:0/355449844 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c040b80 con 0x562f60ad55d0 2024-09-10T10:47:33.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.962+0000 7ff231ffb640 1 -- 172.21.15.143:0/355449844 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c005250 con 0x562f60ad55d0 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 -- 172.21.15.143:0/355449844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 msgr2=0x562f60aec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60aec270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff23c031860 tx=0x7ff23c001e50 comp rx=0 tx=0).stop 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 -- 172.21.15.143:0/355449844 shutdown_connections 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ae0370 0x562f60ad1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ad6b40 0x562f60ad1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 --2- 172.21.15.143:0/355449844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60aec270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 -- 172.21.15.143:0/355449844 >> 172.21.15.143:0/355449844 conn(0x562f60a9e310 msgr2=0x562f60adcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 -- 172.21.15.143:0/355449844 shutdown_connections 2024-09-10T10:47:33.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 -- 172.21.15.143:0/355449844 wait complete. 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.963+0000 7ff254649740 1 -- start start 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 Processor -- start 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60ae3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60ae4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60b58bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1add70 con 0x562f60ad6b40 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1ad6a0 con 0x562f60ad55d0 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1aa8e0 con 0x562f60ae0370 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff254649740 10 client.0 _ll_register_callbacks cb 0x7ff2440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60ae4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60ae3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60ae4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54512/0 (socket says 172.21.15.143:54512) 2024-09-10T10:47:33.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60ae3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42544/0 (socket says 172.21.15.143:42544) 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff24bfff640 1 -- 172.21.15.143:0/786327698 learned_addr learned my addr 172.21.15.143:0/786327698 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60b58bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 -- 172.21.15.143:0/786327698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 msgr2=0x562f60b58bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60b58bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 -- 172.21.15.143:0/786327698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 msgr2=0x562f60ae4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60ae4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.964+0000 7ff25080d640 1 -- 172.21.15.143:0/786327698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f60b591b0 con 0x562f60ad55d0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff25080d640 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60ae3b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff23c003a40 tx=0x7ff23c057040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a9356c8d42b008 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff22b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff22b7fe640 1 -- 172.21.15.143:0/786327698 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c06d060 con 0x562f60ad55d0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23c040b80 con 0x562f60ad55d0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff22b7fe640 1 -- 172.21.15.143:0/786327698 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff23c0319f0 con 0x562f60ad55d0 2024-09-10T10:47:33.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff22b7fe640 1 -- 172.21.15.143:0/786327698 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff23c05a070 con 0x562f60ad55d0 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff254649740 10 client.25514 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f60b5a4b0 con 0x562f60ad55d0 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.965+0000 7ff22b7fe640 1 -- 172.21.15.143:0/786327698 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff23c005460 con 0x562f60ad55d0 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff22b7fe640 1 -- 172.21.15.143:0/786327698 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7ff23c004d00 con 0x562f60ad55d0 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff254649740 10 client.25514 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff254649740 10 client.25514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff254649740 20 client.25514 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff254649740 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60b5a8e0 0x562f60b5ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff254649740 4 client.25514 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff23c040b80 con 0x562f60b5a8e0 2024-09-10T10:47:33.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff24bfff640 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60b5a8e0 0x562f60b5ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff22b7fe640 10 client.25514 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.966+0000 7ff24bfff640 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60b5a8e0 0x562f60b5ccd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae8762013a4b9780 server_cookie=93e9e2aff09dfa24 in_seq=0 out_seq=0 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff22b7fe640 1 -- 172.21.15.143:0/786327698 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff23c040b80 con 0x562f60b5a8e0 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff22b7fe640 10 client.25514 handle_command_reply: tid=0 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 client.25514 shutdown 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 msgr2=0x562f60ae3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60ae3b40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff23c003a40 tx=0x7ff23c057040 comp rx=0 tx=0).stop 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 shutdown_connections 2024-09-10T10:47:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f60ae0370 0x562f60b58bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f60ad6b40 0x562f60ae4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x562f60b5a8e0 0x562f60b5ccd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff22b7fe640 0 client.25514 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 --2- 172.21.15.143:0/786327698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f60ad55d0 0x562f60ae3b40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 >> 172.21.15.143:0/786327698 conn(0x562f60a9e310 msgr2=0x562f60adb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.967+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 shutdown_connections 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.968+0000 7ff254649740 1 -- 172.21.15.143:0/786327698 wait complete. 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.968+0000 7ff254649740 20 client.25514 trim_cache size 0 max 16384 2024-09-10T10:47:33.973 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2340a2cd0 msgr2=0x7ff2340a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2340a2cd0 0x7ff2340a5190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff24000c6a0 tx=0x7ff240097640 comp rx=0 tx=0).stop 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 msgr2=0x7ff24c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 0x7ff24c198820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff24000e9f0 tx=0x7ff24000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 shutdown_connections 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2340a2cd0 0x7ff2340a5190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff24c106f10 0x7ff24c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff24c102790 0x7ff24c198820 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 --2- 172.21.15.143:0/3284025230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff24c101de0 0x7ff24c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 >> 172.21.15.143:0/3284025230 conn(0x7ff24c0fd880 msgr2=0x7ff24c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 shutdown_connections 2024-09-10T10:47:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:33.969+0000 7ff252270640 1 -- 172.21.15.143:0/3284025230 wait complete. 2024-09-10T10:47:34.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:34 smithi143 ceph-mon[31057]: pgmap v426: 129 pgs: 129 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T10:47:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:34 smithi178 ceph-mon[34813]: pgmap v426: 129 pgs: 129 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T10:47:34.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:34 smithi144 ceph-mon[34230]: pgmap v426: 129 pgs: 129 active+clean; 17 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T10:47:34.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:47:35.290 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 -- 172.21.15.143:0/288118668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 msgr2=0x7f6d601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 --2- 172.21.15.143:0/288118668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601067e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009900 tx=0x7f6d5402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 -- 172.21.15.143:0/288118668 shutdown_connections 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 --2- 172.21.15.143:0/288118668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d60106f10 0x7f6d6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 --2- 172.21.15.143:0/288118668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601067e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 --2- 172.21.15.143:0/288118668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 -- 172.21.15.143:0/288118668 >> 172.21.15.143:0/288118668 conn(0x7f6d600fd8c0 msgr2=0x7f6d600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 -- 172.21.15.143:0/288118668 shutdown_connections 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.850+0000 7f6d655c7640 1 -- 172.21.15.143:0/288118668 wait complete. 2024-09-10T10:47:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.851+0000 7f6d655c7640 1 Processor -- start 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.851+0000 7f6d655c7640 1 -- start start 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d655c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d655c7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d655c7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d60106f10 0x7f6d6019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53262/0 (socket says 172.21.15.143:53262) 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d5e7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d5e7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54526/0 (socket says 172.21.15.143:54526) 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d655c7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60111e20 con 0x7f6d60102790 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60111ca0 con 0x7f6d60106f10 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d5effd640 1 -- 172.21.15.143:0/2548026292 learned_addr learned my addr 172.21.15.143:0/2548026292 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60111fa0 con 0x7f6d60101de0 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.852+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d60106f10 0x7f6d6019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d5effd640 1 -- 172.21.15.143:0/2548026292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d60106f10 msgr2=0x7f6d6019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d5effd640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d60106f10 0x7f6d6019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d5effd640 1 -- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 msgr2=0x7f6d601988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d5effd640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d5effd640 1 -- 172.21.15.143:0/2548026292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6019d350 con 0x7f6d60101de0 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d5effd640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601983a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000e9f0 tx=0x7f6d5000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d47fff640 1 -- 172.21.15.143:0/2548026292 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000cd80 con 0x7f6d60101de0 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d47fff640 1 -- 172.21.15.143:0/2548026292 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d500054d0 con 0x7f6d60101de0 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d47fff640 1 -- 172.21.15.143:0/2548026292 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50010490 con 0x7f6d60101de0 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6019d640 con 0x7f6d60101de0 2024-09-10T10:47:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.853+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d60074420 con 0x7f6d60101de0 2024-09-10T10:47:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.856+0000 7f6d47fff640 1 -- 172.21.15.143:0/2548026292 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6d50005020 con 0x7f6d60101de0 2024-09-10T10:47:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.856+0000 7f6d679a0740 1 Processor -- start 2024-09-10T10:47:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 -- start start 2024-09-10T10:47:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d47fff640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d480a2c80 0x7f6d480a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338fe3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe35d0 0x55a338fe3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 0x55a338ff0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a338f492f0 con 0x55a338fe35d0 2024-09-10T10:47:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a338ef0cf0 con 0x55a338fe4b40 2024-09-10T10:47:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d679a0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a338f452b0 con 0x55a338fee370 2024-09-10T10:47:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d480a2c80 0x7f6d480a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338fe3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 0x55a338ff0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 0x55a338ff0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53272/0 (socket says 172.21.15.143:53272) 2024-09-10T10:47:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/2200660952 learned_addr learned my addr 172.21.15.143:0/2200660952 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.857+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe35d0 0x55a338fe3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d47fff640 1 -- 172.21.15.143:0/2548026292 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d500cf790 con 0x7f6d60101de0 2024-09-10T10:47:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/2200660952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 msgr2=0x55a338fe3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338fe3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/2200660952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe35d0 msgr2=0x55a338fe3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe35d0 0x55a338fe3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/2200660952 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a338ed53a0 con 0x55a338fee370 2024-09-10T10:47:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d480a2c80 0x7f6d480a5140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009fd0 tx=0x7f6d58009d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 0x55a338ff0760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009280 tx=0x7f6d5801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23bc96e455baf1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d457fa640 1 -- 172.21.15.143:0/2200660952 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58004b20 con 0x55a338fee370 2024-09-10T10:47:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d457fa640 1 -- 172.21.15.143:0/2200660952 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d58004cc0 con 0x55a338fee370 2024-09-10T10:47:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.858+0000 7f6d457fa640 1 -- 172.21.15.143:0/2200660952 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58005470 con 0x55a338fee370 2024-09-10T10:47:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 -- 172.21.15.143:0/2200660952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 msgr2=0x55a338ff0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 0x55a338ff0760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009280 tx=0x7f6d5801c000 comp rx=0 tx=0).stop 2024-09-10T10:47:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 -- 172.21.15.143:0/2200660952 shutdown_connections 2024-09-10T10:47:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fee370 0x55a338ff0760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe35d0 0x55a338fe3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 --2- 172.21.15.143:0/2200660952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338fe3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 -- 172.21.15.143:0/2200660952 >> 172.21.15.143:0/2200660952 conn(0x55a338fac310 msgr2=0x55a338fdfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 -- 172.21.15.143:0/2200660952 shutdown_connections 2024-09-10T10:47:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.859+0000 7f6d679a0740 1 -- 172.21.15.143:0/2200660952 wait complete. 2024-09-10T10:47:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 -- start start 2024-09-10T10:47:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 Processor -- start 2024-09-10T10:47:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338feaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fee370 0x55a338feb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fedd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53284/0 (socket says 172.21.15.143:53284) 2024-09-10T10:47:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d5e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338feaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a338ff1010 con 0x55a338fee370 2024-09-10T10:47:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d5effd640 1 -- 172.21.15.143:0/834837053 learned_addr learned my addr 172.21.15.143:0/834837053 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a338f04060 con 0x55a338fe4b40 2024-09-10T10:47:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fee370 0x55a338feb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a338ff1190 con 0x55a338fe35d0 2024-09-10T10:47:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.860+0000 7f6d679a0740 10 client.0 _ll_register_callbacks cb 0x55a338c578c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 msgr2=0x55a338fedd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fedd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/834837053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 msgr2=0x55a338feaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338feaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a338cba3c0 con 0x55a338fee370 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5effd640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338feaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fee370 0x55a338feb5e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6d58024000 tx=0x7f6d5800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57876bc5994c97f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d42ff5640 1 -- 172.21.15.143:0/834837053 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802e070 con 0x55a338fee370 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d58004cc0 con 0x55a338fee370 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.861+0000 7f6d42ff5640 1 -- 172.21.15.143:0/834837053 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d5800cac0 con 0x55a338fee370 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d42ff5640 1 -- 172.21.15.143:0/834837053 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d5803a050 con 0x55a338fee370 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:35.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a3390052a0 con 0x55a338fee370 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d42ff5640 1 -- 172.21.15.143:0/834837053 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d58004cc0 con 0x55a338fee370 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d42ff5640 1 -- 172.21.15.143:0/834837053 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f6d5801f070 con 0x55a338fee370 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 20 client.16137 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339006ac0 0x55a339008ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 4 client.16137 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a338f56860 con 0x55a339006ac0 2024-09-10T10:47:35.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.862+0000 7f6d5effd640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339006ac0 0x55a339008ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.863+0000 7f6d42ff5640 10 client.16137 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.863+0000 7f6d5effd640 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339006ac0 0x55a339008ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7972451f14689cc7 server_cookie=8c973fac094fc779 in_seq=0 out_seq=0 2024-09-10T10:47:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.863+0000 7f6d42ff5640 1 -- 172.21.15.143:0/834837053 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a338f56860 con 0x55a339006ac0 2024-09-10T10:47:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.863+0000 7f6d42ff5640 10 client.16137 handle_command_reply: tid=0 2024-09-10T10:47:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.863+0000 7f6d679a0740 1 client.16137 shutdown 2024-09-10T10:47:35.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fee370 msgr2=0x55a338feb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fee370 0x55a338feb5e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6d58024000 tx=0x7f6d5800cda0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 shutdown_connections 2024-09-10T10:47:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fee370 0x55a338feb5e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fe4b40 0x55a338feaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339006ac0 0x55a339008ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d42ff5640 0 client.16137 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 --2- 172.21.15.143:0/834837053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 >> 172.21.15.143:0/834837053 conn(0x55a338fac310 msgr2=0x55a338fe5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 shutdown_connections 2024-09-10T10:47:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 1 -- 172.21.15.143:0/834837053 wait complete. 2024-09-10T10:47:35.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.864+0000 7f6d679a0740 20 client.16137 trim_cache size 0 max 16384 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.885+0000 7f6d679a0740 1 Processor -- start 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.885+0000 7f6d679a0740 1 -- start start 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.885+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fe9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d679a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ac8e0 con 0x55a338fe4b40 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d679a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ae450 con 0x55a338fee370 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d679a0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601acfb0 con 0x55a338fe35d0 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fe9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fe9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53298/0 (socket says 172.21.15.143:53298) 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5effd640 1 -- 172.21.15.143:0/4082809635 learned_addr learned my addr 172.21.15.143:0/4082809635 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/4082809635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 msgr2=0x55a338fe9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fe9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/4082809635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 msgr2=0x55a338fecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 -- 172.21.15.143:0/4082809635 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a338fdf510 con 0x55a338fee370 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5effd640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fe9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:35.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.886+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe6120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800ce10 tx=0x7f6d58005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6b66610a70e529 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.887+0000 7f6d44ff9640 1 -- 172.21.15.143:0/4082809635 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802e070 con 0x55a338fee370 2024-09-10T10:47:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.887+0000 7f6d44ff9640 1 -- 172.21.15.143:0/4082809635 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d5802a7f0 con 0x55a338fee370 2024-09-10T10:47:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.887+0000 7f6d44ff9640 1 -- 172.21.15.143:0/4082809635 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d580124a0 con 0x55a338fee370 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.887+0000 7f6d679a0740 1 -- 172.21.15.143:0/4082809635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 msgr2=0x55a338fe6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.887+0000 7f6d679a0740 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe6120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800ce10 tx=0x7f6d58005bd0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 -- 172.21.15.143:0/4082809635 shutdown_connections 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe6120 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 --2- 172.21.15.143:0/4082809635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338fe9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 -- 172.21.15.143:0/4082809635 >> 172.21.15.143:0/4082809635 conn(0x55a338fac310 msgr2=0x55a338fec850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 -- 172.21.15.143:0/4082809635 shutdown_connections 2024-09-10T10:47:35.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 -- 172.21.15.143:0/4082809635 wait complete. 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 -- start start 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 Processor -- start 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.888+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338ffed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d679a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d679a0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601add50 con 0x55a338fe4b40 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d679a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ad680 con 0x55a338fee370 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338ffed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338ffed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53314/0 (socket says 172.21.15.143:53314) 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 -- 172.21.15.143:0/866514959 learned_addr learned my addr 172.21.15.143:0/866514959 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601aa8c0 con 0x55a338fe35d0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d679a0740 10 client.0 _ll_register_callbacks cb 0x7f6d54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5f7fe640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 -- 172.21.15.143:0/866514959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 msgr2=0x55a338fe7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 -- 172.21.15.143:0/866514959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 msgr2=0x55a338fff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 -- 172.21.15.143:0/866514959 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a338fff980 con 0x55a338fe35d0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d5effd640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338ffed00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c6a0 tx=0x7f6d50097f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdd91ec17b82b85c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.889+0000 7f6d42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:35.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d42ff5640 1 -- 172.21.15.143:0/866514959 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d500b9060 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d42ff5640 1 -- 172.21.15.143:0/866514959 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d50099f30 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d5802a7f0 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d42ff5640 1 -- 172.21.15.143:0/866514959 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d50005680 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 10 client.25570 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a339070e40 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d42ff5640 1 -- 172.21.15.143:0/866514959 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d5802a7f0 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d42ff5640 1 -- 172.21.15.143:0/866514959 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f6d5009be90 con 0x55a338fe35d0 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 10 client.25570 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 20 client.25570 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339071520 0x55a339073940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 4 client.25570 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.890+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a338f50d70 con 0x55a339071520 2024-09-10T10:47:35.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.891+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339071520 0x55a339073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.891+0000 7f6d42ff5640 10 client.25570 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.891+0000 7f6d5e7fc640 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339071520 0x55a339073940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b16d9ca660ee7a8 server_cookie=8bae939a3600d356 in_seq=0 out_seq=0 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.891+0000 7f6d42ff5640 1 -- 172.21.15.143:0/866514959 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a338f50d70 con 0x55a339071520 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.891+0000 7f6d42ff5640 10 client.25570 handle_command_reply: tid=0 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.891+0000 7f6d679a0740 1 client.25570 shutdown 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 msgr2=0x55a338ffed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338ffed00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c6a0 tx=0x7f6d50097f90 comp rx=0 tx=0).stop 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 shutdown_connections 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a338fee370 0x55a338fe7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a338fe4b40 0x55a338fff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a339071520 0x55a339073940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 --2- 172.21.15.143:0/866514959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a338fe35d0 0x55a338ffed00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 >> 172.21.15.143:0/866514959 conn(0x55a338fac310 msgr2=0x55a338fed070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d42ff5640 0 client.25570 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:35.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 shutdown_connections 2024-09-10T10:47:35.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.892+0000 7f6d679a0740 1 -- 172.21.15.143:0/866514959 wait complete. 2024-09-10T10:47:35.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d679a0740 20 client.25570 trim_cache size 0 max 16384 2024-09-10T10:47:35.900 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:47:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d480a2c80 msgr2=0x7f6d480a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d480a2c80 0x7f6d480a5140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009fd0 tx=0x7f6d58009d00 comp rx=0 tx=0).stop 2024-09-10T10:47:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 msgr2=0x7f6d601983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601983a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000e9f0 tx=0x7f6d5000edf0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 shutdown_connections 2024-09-10T10:47:35.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d480a2c80 0x7f6d480a5140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d60106f10 0x7f6d6019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d60102790 0x7f6d601988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 --2- 172.21.15.143:0/2548026292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d60101de0 0x7f6d601983a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.893+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 >> 172.21.15.143:0/2548026292 conn(0x7f6d600fd8c0 msgr2=0x7f6d600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.894+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 shutdown_connections 2024-09-10T10:47:35.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:35.894+0000 7f6d655c7640 1 -- 172.21.15.143:0/2548026292 wait complete. 2024-09-10T10:47:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:36 smithi178 ceph-mon[34813]: pgmap v427: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-09-10T10:47:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:36 smithi144 ceph-mon[34230]: pgmap v427: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-09-10T10:47:37.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:36 smithi143 ceph-mon[31057]: pgmap v427: 129 pgs: 129 active+clean; 20 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-09-10T10:47:37.759 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:47:37.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:47:38.107 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 -- 172.21.15.143:0/2642735874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc100e20 msgr2=0x7fc2fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 --2- 172.21.15.143:0/2642735874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc100e20 0x7fc2fc101200 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02a480 comp rx=0 tx=0).stop 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 -- 172.21.15.143:0/2642735874 shutdown_connections 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 --2- 172.21.15.143:0/2642735874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2fc102ef0 0x7fc2fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 --2- 172.21.15.143:0/2642735874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc102570 0x7fc2fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 --2- 172.21.15.143:0/2642735874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc100e20 0x7fc2fc101200 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 -- 172.21.15.143:0/2642735874 >> 172.21.15.143:0/2642735874 conn(0x7fc2fc0fc7d0 msgr2=0x7fc2fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 -- 172.21.15.143:0/2642735874 shutdown_connections 2024-09-10T10:47:38.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.623+0000 7fc300f71640 1 -- 172.21.15.143:0/2642735874 wait complete. 2024-09-10T10:47:38.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.624+0000 7fc300f71640 1 Processor -- start 2024-09-10T10:47:38.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.624+0000 7fc300f71640 1 -- start start 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc300f71640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 0x7fc2fc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc300f71640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2fc102570 0x7fc2fc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc300f71640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc102ef0 0x7fc2fc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc300f71640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc110e30 con 0x7fc2fc102ef0 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 0x7fc2fc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 0x7fc2fc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:53454/0 (socket says 172.21.15.143:53454) 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2182940336 learned_addr learned my addr 172.21.15.143:0/2182940336 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc102ef0 0x7fc2fc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc110cb0 con 0x7fc2fc100e20 2024-09-10T10:47:38.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc110fb0 con 0x7fc2fc102570 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2fc102570 0x7fc2fc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2182940336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2fc102570 msgr2=0x7fc2fc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2fc102570 0x7fc2fc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc102ef0 msgr2=0x7fc2fc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc102ef0 0x7fc2fc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.625+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2182940336 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc19f5d0 con 0x7fc2fc100e20 2024-09-10T10:47:38.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.626+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 0x7fc2fc19a620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec006010 tx=0x7fc2ec0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.626+0000 7fc2eb7fe640 1 -- 172.21.15.143:0/2182940336 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec03f030 con 0x7fc2fc100e20 2024-09-10T10:47:38.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.626+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc19f860 con 0x7fc2fc100e20 2024-09-10T10:47:38.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.626+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1a71a0 con 0x7fc2fc100e20 2024-09-10T10:47:38.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.626+0000 7fc2eb7fe640 1 -- 172.21.15.143:0/2182940336 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2ec0044c0 con 0x7fc2fc100e20 2024-09-10T10:47:38.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.627+0000 7fc2eb7fe640 1 -- 172.21.15.143:0/2182940336 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec035070 con 0x7fc2fc100e20 2024-09-10T10:47:38.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.629+0000 7fc2eb7fe640 1 -- 172.21.15.143:0/2182940336 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc2ec042030 con 0x7fc2fc100e20 2024-09-10T10:47:38.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.630+0000 7fc30334a740 1 Processor -- start 2024-09-10T10:47:38.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.630+0000 7fc2eb7fe640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2e40a2c80 0x7fc2e40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.630+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2e40a2c80 0x7fc2e40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 -- start start 2024-09-10T10:47:38.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2eb7fe640 1 -- 172.21.15.143:0/2182940336 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2ec0ef5b0 con 0x7fc2fc100e20 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c66800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3bc1280 con 0x55e6c3c66d40 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3b68b10 con 0x55e6c3c5c480 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc30334a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3bbd240 con 0x55e6c3c5bb60 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2e40a2c80 0x7fc2e40a5140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005fd0 tx=0x7fc2f4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2f9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c66800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c69130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43108/0 (socket says 172.21.15.143:43108) 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2fad9d640 1 -- 172.21.15.143:0/3162086935 learned_addr learned my addr 172.21.15.143:0/3162086935 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2f9d9b640 1 -- 172.21.15.143:0/3162086935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 msgr2=0x55e6c3c5bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2f9d9b640 1 -- 172.21.15.143:0/3162086935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 msgr2=0x55e6c3c69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2f9d9b640 1 -- 172.21.15.143:0/3162086935 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3b751f0 con 0x55e6c3c5c480 2024-09-10T10:47:38.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.631+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:38.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.632+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c66800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009b40 tx=0x7fc2f00113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a872a5435bb84bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.632+0000 7fc2e8ff9640 1 -- 172.21.15.143:0/3162086935 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0011e10 con 0x55e6c3c5c480 2024-09-10T10:47:38.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.632+0000 7fc2e8ff9640 1 -- 172.21.15.143:0/3162086935 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f0012900 con 0x55e6c3c5c480 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.632+0000 7fc2e8ff9640 1 -- 172.21.15.143:0/3162086935 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000f970 con 0x55e6c3c5c480 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 -- 172.21.15.143:0/3162086935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 msgr2=0x55e6c3c66800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c66800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009b40 tx=0x7fc2f00113e0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 -- 172.21.15.143:0/3162086935 shutdown_connections 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c66800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 --2- 172.21.15.143:0/3162086935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c5bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 -- 172.21.15.143:0/3162086935 >> 172.21.15.143:0/3162086935 conn(0x55e6c3c580f0 msgr2=0x55e6c3c584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 -- 172.21.15.143:0/3162086935 shutdown_connections 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 -- 172.21.15.143:0/3162086935 wait complete. 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 -- start start 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.633+0000 7fc30334a740 1 Processor -- start 2024-09-10T10:47:38.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c63bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c64100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c66d40 0x55e6c3c64640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3c69960 con 0x55e6c3c5bb60 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3b7bf10 con 0x55e6c3c5c480 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3c69ae0 con 0x55e6c3c66d40 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc30334a740 10 client.0 _ll_register_callbacks cb 0x55e6c3b44a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c63bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c63bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43114/0 (socket says 172.21.15.143:43114) 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 -- 172.21.15.143:0/86206089 learned_addr learned my addr 172.21.15.143:0/86206089 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c66d40 0x55e6c3c64640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 -- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c66d40 msgr2=0x55e6c3c64640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c66d40 0x55e6c3c64640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 -- 172.21.15.143:0/86206089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 msgr2=0x55e6c3c64100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c64100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 -- 172.21.15.143:0/86206089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c39326a0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c63bc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0c4190 tx=0x7fc2ec0bb650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc76cb5a84698a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.634+0000 7fc2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/86206089 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec049070 con 0x55e6c3c5bb60 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/86206089 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2ec0b80c0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f0012900 con 0x55e6c3c5bb60 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/86206089 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec0bfe00 con 0x55e6c3c5bb60 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc30334a740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6c3c755a0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/86206089 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f0012900 con 0x55e6c3c5bb60 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/86206089 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fc2ec0c42c0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc30334a740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc30334a740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.635+0000 7fc30334a740 20 client.16155 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.636+0000 7fc30334a740 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3c78e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.636+0000 7fc30334a740 4 client.16155 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.636+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c3bce7f0 con 0x55e6c3c76a10 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.636+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3c78e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.636+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3c78e30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa551f646baf5377 server_cookie=b14ad16988cc480b in_seq=0 out_seq=0 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.636+0000 7fc2e1ffb640 10 client.16155 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/86206089 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6c3bce7f0 con 0x55e6c3c76a10 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc2e1ffb640 10 client.16155 handle_command_reply: tid=0 2024-09-10T10:47:38.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 client.16155 shutdown 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 msgr2=0x55e6c3c63bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c63bc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0c4190 tx=0x7fc2ec0bb650 comp rx=0 tx=0).stop 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 shutdown_connections 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3c78e30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc2e1ffb640 0 client.16155 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c66d40 0x55e6c3c64640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c64100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 --2- 172.21.15.143:0/86206089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c63bc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 >> 172.21.15.143:0/86206089 conn(0x55e6c3c580f0 msgr2=0x55e6c3c5ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 shutdown_connections 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 1 -- 172.21.15.143:0/86206089 wait complete. 2024-09-10T10:47:38.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.637+0000 7fc30334a740 20 client.16155 trim_cache size 0 max 16384 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 Processor -- start 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 -- start start 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c7e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5c480 0x55e6c3c7ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc00fa00 con 0x55e6c3c66d40 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc011570 con 0x55e6c3c5bb60 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc30334a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0100d0 con 0x55e6c3c5c480 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.661+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c60e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5c480 0x55e6c3c7ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c60e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43116/0 (socket says 172.21.15.143:43116) 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2fad9d640 1 -- 172.21.15.143:0/3980513006 learned_addr learned my addr 172.21.15.143:0/3980513006 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c7e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 -- 172.21.15.143:0/3980513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5bb60 msgr2=0x55e6c3c7e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c7e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 -- 172.21.15.143:0/3980513006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 msgr2=0x55e6c3c60e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c60e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 -- 172.21.15.143:0/3980513006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3c53830 con 0x55e6c3c5c480 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c60e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5c480 0x55e6c3c7ec60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2f001c040 tx=0x7fc2f0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eff22d90495c0d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2e3fff640 1 -- 172.21.15.143:0/3980513006 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0005300 con 0x55e6c3c5c480 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2e3fff640 1 -- 172.21.15.143:0/3980513006 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f00054a0 con 0x55e6c3c5c480 2024-09-10T10:47:38.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.662+0000 7fc2e3fff640 1 -- 172.21.15.143:0/3980513006 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00127c0 con 0x55e6c3c5c480 2024-09-10T10:47:38.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 -- 172.21.15.143:0/3980513006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5c480 msgr2=0x55e6c3c7ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5c480 0x55e6c3c7ec60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2f001c040 tx=0x7fc2f0005f90 comp rx=0 tx=0).stop 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 -- 172.21.15.143:0/3980513006 shutdown_connections 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c60e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5c480 0x55e6c3c7ec60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 --2- 172.21.15.143:0/3980513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c7e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 -- 172.21.15.143:0/3980513006 >> 172.21.15.143:0/3980513006 conn(0x55e6c3c580f0 msgr2=0x55e6c3c65b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 -- 172.21.15.143:0/3980513006 shutdown_connections 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.663+0000 7fc30334a740 1 -- 172.21.15.143:0/3980513006 wait complete. 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 -- start start 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 Processor -- start 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c6edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c6f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c6bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc010e70 con 0x55e6c3c66d40 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0107a0 con 0x55e6c3c5c480 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc00d9e0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc30334a740 10 client.0 _ll_register_callbacks cb 0x7fc2ec005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c6bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c6edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c6edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42762/0 (socket says 172.21.15.143:42762) 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c6bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43132/0 (socket says 172.21.15.143:43132) 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2433816806 learned_addr learned my addr 172.21.15.143:0/2433816806 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c6f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2433816806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 msgr2=0x55e6c3c6f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c6f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2433816806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 msgr2=0x55e6c3c6bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c6bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.664+0000 7fc2fa59c640 1 -- 172.21.15.143:0/2433816806 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c3c6c2d0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2fad9d640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c6bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c6f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2fa59c640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c6edc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0c4630 tx=0x7fc2ec02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38d6ee0f69f37c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/2433816806 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec049040 con 0x55e6c3c5bb60 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/2433816806 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2ec0baea0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f00054a0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/2433816806 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec0bb820 con 0x55e6c3c5bb60 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 10 client.25585 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6c3ce1ec0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/2433816806 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f00054a0 con 0x55e6c3c5bb60 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/2433816806 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fc2ec0c0f90 con 0x55e6c3c5bb60 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 10 client.25585 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 20 client.25585 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3ce44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 4 client.25585 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.665+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6c3bc8d00 con 0x55e6c3c76a10 2024-09-10T10:47:38.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.666+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3ce44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.666+0000 7fc2f9d9b640 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3ce44d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae656df4509e8c3 server_cookie=985e1d40342eddfb in_seq=0 out_seq=0 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.666+0000 7fc2e1ffb640 10 client.25585 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc2e1ffb640 1 -- 172.21.15.143:0/2433816806 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6c3bc8d00 con 0x55e6c3c76a10 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc2e1ffb640 10 client.25585 handle_command_reply: tid=0 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 client.25585 shutdown 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 msgr2=0x55e6c3c6edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c6edc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0c4630 tx=0x7fc2ec02a480 comp rx=0 tx=0).stop 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 shutdown_connections 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c3c66d40 0x55e6c3c6bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6c3c5c480 0x55e6c3c6f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e6c3c76a10 0x55e6c3ce44d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 --2- 172.21.15.143:0/2433816806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e6c3c5bb60 0x55e6c3c6edc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 >> 172.21.15.143:0/2433816806 conn(0x55e6c3c580f0 msgr2=0x55e6c3c65200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.667+0000 7fc2e1ffb640 0 client.25585 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:38.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.668+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 shutdown_connections 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.668+0000 7fc30334a740 1 -- 172.21.15.143:0/2433816806 wait complete. 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.668+0000 7fc30334a740 20 client.25585 trim_cache size 0 max 16384 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "6e4f9feb-c385-4155-9032-9e489544d44a", 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:47:38.675 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2e40a2c80 msgr2=0x7fc2e40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2e40a2c80 0x7fc2e40a5140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005fd0 tx=0x7fc2f4009500 comp rx=0 tx=0).stop 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 msgr2=0x7fc2fc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 0x7fc2fc19a620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec006010 tx=0x7fc2ec0040c0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 shutdown_connections 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2e40a2c80 0x7fc2e40a5140 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2fc102ef0 0x7fc2fc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2fc102570 0x7fc2fc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 --2- 172.21.15.143:0/2182940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2fc100e20 0x7fc2fc19a620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 >> 172.21.15.143:0/2182940336 conn(0x7fc2fc0fc7d0 msgr2=0x7fc2fc06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 shutdown_connections 2024-09-10T10:47:38.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:38.669+0000 7fc300f71640 1 -- 172.21.15.143:0/2182940336 wait complete. 2024-09-10T10:47:38.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:38 smithi143 ceph-mon[31057]: pgmap v428: 129 pgs: 129 active+clean; 34 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-10T10:47:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:38 smithi178 ceph-mon[34813]: pgmap v428: 129 pgs: 129 active+clean; 34 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-10T10:47:38.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:38 smithi144 ceph-mon[34230]: pgmap v428: 129 pgs: 129 active+clean; 34 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-10T10:47:39.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:47:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:39 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:47:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:39 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:39 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:47:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:39 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:47:39.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:39 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:39 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:47:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:39 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:39 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:47:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:39 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:47:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:39 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:39.980 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:47:40.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:39 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:47:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:39 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:39 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:47:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:39 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:47:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:39 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.520+0000 7fa8420ad640 1 -- 172.21.15.143:0/3579136061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c100a80 msgr2=0x7fa83c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.520+0000 7fa8420ad640 1 --2- 172.21.15.143:0/3579136061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c100a80 0x7fa83c100ee0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa824009930 tx=0x7fa82402a420 comp rx=0 tx=0).stop 2024-09-10T10:47:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 -- 172.21.15.143:0/3579136061 shutdown_connections 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 --2- 172.21.15.143:0/3579136061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83c101420 0x7fa83c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 --2- 172.21.15.143:0/3579136061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c100a80 0x7fa83c100ee0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 --2- 172.21.15.143:0/3579136061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c106a80 0x7fa83c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 -- 172.21.15.143:0/3579136061 >> 172.21.15.143:0/3579136061 conn(0x7fa83c0fc7a0 msgr2=0x7fa83c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 -- 172.21.15.143:0/3579136061 shutdown_connections 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 -- 172.21.15.143:0/3579136061 wait complete. 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.521+0000 7fa8420ad640 1 Processor -- start 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 -- start start 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 0x7fa83c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83c101420 0x7fa83c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c106a80 0x7fa83c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c111e50 con 0x7fa83c106a80 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c111cd0 con 0x7fa83c101420 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa8420ad640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c111fd0 con 0x7fa83c100a80 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 0x7fa83c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 0x7fa83c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42784/0 (socket says 172.21.15.143:42784) 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa83b7fe640 1 -- 172.21.15.143:0/2759411554 learned_addr learned my addr 172.21.15.143:0/2759411554 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa83bfff640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c106a80 0x7fa83c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.522+0000 7fa83affd640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83c101420 0x7fa83c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa83b7fe640 1 -- 172.21.15.143:0/2759411554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83c101420 msgr2=0x7fa83c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83c101420 0x7fa83c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa83b7fe640 1 -- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c106a80 msgr2=0x7fa83c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c106a80 0x7fa83c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa83b7fe640 1 -- 172.21.15.143:0/2759411554 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c19d360 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 0x7fa83c1983b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa83400ebc0 tx=0x7fa83400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa838ff9640 1 -- 172.21.15.143:0/2759411554 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834010630 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa838ff9640 1 -- 172.21.15.143:0/2759411554 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa834010c80 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa838ff9640 1 -- 172.21.15.143:0/2759411554 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834018b00 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83c19d650 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.523+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83c1a4f30 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.526+0000 7fa838ff9640 1 -- 172.21.15.143:0/2759411554 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa834018ca0 con 0x7fa83c100a80 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.526+0000 7fa844486740 1 Processor -- start 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 -- start start 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d927ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d922030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55844d87e2e0 con 0x55844d928000 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55844d825ce0 con 0x55844d9276e0 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa844486740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55844d87a2a0 con 0x55844d9226a0 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d927ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa83affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa83affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42796/0 (socket says 172.21.15.143:42796) 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa83affd640 1 -- 172.21.15.143:0/1113341761 learned_addr learned my addr 172.21.15.143:0/1113341761 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d927ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:53506/0 (socket says 172.21.15.143:53506) 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.527+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d922030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa838ff9640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa8280a2c00 0x7fa8280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa83b7fe640 1 -- 172.21.15.143:0/1113341761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 msgr2=0x55844d922b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa83b7fe640 1 -- 172.21.15.143:0/1113341761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 msgr2=0x55844d927ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d927ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa83b7fe640 1 -- 172.21.15.143:0/1113341761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844d80a440 con 0x55844d928000 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.528+0000 7fa83bfff640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa8280a2c00 0x7fa8280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.529+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d922030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa8340cd0b0 tx=0x7fa834066b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a60b7307ddeab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.529+0000 7fa838ff9640 1 -- 172.21.15.143:0/2759411554 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa8340cf940 con 0x7fa83c100a80 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.529+0000 7fa831ffb640 1 -- 172.21.15.143:0/1113341761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83409b850 con 0x55844d928000 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.529+0000 7fa831ffb640 1 -- 172.21.15.143:0/1113341761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8340d1e50 con 0x55844d928000 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.529+0000 7fa831ffb640 1 -- 172.21.15.143:0/1113341761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83407c130 con 0x55844d928000 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 -- 172.21.15.143:0/1113341761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 msgr2=0x55844d922030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d922030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa8340cd0b0 tx=0x7fa834066b70 comp rx=0 tx=0).stop 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa83bfff640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa8280a2c00 0x7fa8280a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009d80 tx=0x7fa82c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 -- 172.21.15.143:0/1113341761 shutdown_connections 2024-09-10T10:47:40.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d922030 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 --2- 172.21.15.143:0/1113341761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d927ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 -- 172.21.15.143:0/1113341761 >> 172.21.15.143:0/1113341761 conn(0x55844d9144e0 msgr2=0x55844d9148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 -- 172.21.15.143:0/1113341761 shutdown_connections 2024-09-10T10:47:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 -- 172.21.15.143:0/1113341761 wait complete. 2024-09-10T10:47:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.530+0000 7fa844486740 1 -- start start 2024-09-10T10:47:40.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 Processor -- start 2024-09-10T10:47:40.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d92df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d92b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d92b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55844d92e9c0 con 0x55844d928000 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55844d839050 con 0x55844d9276e0 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55844d92eb40 con 0x55844d9226a0 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa844486740 10 client.0 _ll_register_callbacks cb 0x55844d879160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d92df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d92df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42802/0 (socket says 172.21.15.143:42802) 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa83b7fe640 1 -- 172.21.15.143:0/452520965 learned_addr learned my addr 172.21.15.143:0/452520965 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa83bfff640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d92b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.531+0000 7fa83affd640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d92b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83b7fe640 1 -- 172.21.15.143:0/452520965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 msgr2=0x55844d92b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d92b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83b7fe640 1 -- 172.21.15.143:0/452520965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 msgr2=0x55844d92b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d92b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83b7fe640 1 -- 172.21.15.143:0/452520965 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844d5ef3c0 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83affd640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d92b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d92df10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8340cd450 tx=0x7fa8340099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce853599dcf3343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa81affd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa81affd640 1 -- 172.21.15.143:0/452520965 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83409b850 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa81affd640 1 -- 172.21.15.143:0/452520965 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa834097d80 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8340d1e50 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa81affd640 1 -- 172.21.15.143:0/452520965 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa83406f4b0 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa844486740 10 client.25597 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.532+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55844d92e9c0 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa81affd640 1 -- 172.21.15.143:0/452520965 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83409bef0 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa81affd640 1 -- 172.21.15.143:0/452520965 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa83407c320 con 0x55844d9226a0 2024-09-10T10:47:40.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa844486740 10 client.25597 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa844486740 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa844486740 20 client.25597 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa844486740 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d93b570 0x55844d93d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa844486740 4 client.25597 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa8340d1e50 con 0x55844d93b570 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.533+0000 7fa83affd640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d93b570 0x55844d93d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.534+0000 7fa83affd640 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d93b570 0x55844d93d960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba7880a390b108e server_cookie=422db8e406d56863 in_seq=0 out_seq=0 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.534+0000 7fa81affd640 10 client.25597 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.534+0000 7fa81affd640 1 -- 172.21.15.143:0/452520965 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa8340d1e50 con 0x55844d93b570 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.534+0000 7fa81affd640 10 client.25597 handle_command_reply: tid=0 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.534+0000 7fa844486740 1 client.25597 shutdown 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 msgr2=0x55844d92df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d92df10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8340cd450 tx=0x7fa8340099b0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 shutdown_connections 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d92b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d92b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d93b570 0x55844d93d960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 --2- 172.21.15.143:0/452520965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d92df10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 >> 172.21.15.143:0/452520965 conn(0x55844d9144e0 msgr2=0x55844d913bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa81affd640 0 client.25597 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 shutdown_connections 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 1 -- 172.21.15.143:0/452520965 wait complete. 2024-09-10T10:47:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.535+0000 7fa844486740 20 client.25597 trim_cache size 0 max 16384 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 Processor -- start 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 -- start start 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d937c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d926b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1ac930 con 0x55844d928000 2024-09-10T10:47:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1ae4a0 con 0x55844d9276e0 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa844486740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1ad000 con 0x55844d9226a0 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d926b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d926b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43184/0 (socket says 172.21.15.143:43184) 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 -- 172.21.15.143:0/530801729 learned_addr learned my addr 172.21.15.143:0/530801729 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83affd640 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d937c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 -- 172.21.15.143:0/530801729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 msgr2=0x55844d922030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 -- 172.21.15.143:0/530801729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 msgr2=0x55844d937c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d937c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.559+0000 7fa83bfff640 1 -- 172.21.15.143:0/530801729 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844d93de30 con 0x55844d928000 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.560+0000 7fa83bfff640 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d926b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0043b0 tx=0x7fa82c013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe15c1f014b3b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.560+0000 7fa8317fa640 1 -- 172.21.15.143:0/530801729 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c02f070 con 0x55844d928000 2024-09-10T10:47:40.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.560+0000 7fa8317fa640 1 -- 172.21.15.143:0/530801729 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa82c02a910 con 0x55844d928000 2024-09-10T10:47:40.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.560+0000 7fa8317fa640 1 -- 172.21.15.143:0/530801729 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c01d900 con 0x55844d928000 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 -- 172.21.15.143:0/530801729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 msgr2=0x55844d926b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d926b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0043b0 tx=0x7fa82c013070 comp rx=0 tx=0).stop 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 -- 172.21.15.143:0/530801729 shutdown_connections 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d928000 0x55844d926b80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d937c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 --2- 172.21.15.143:0/530801729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d9226a0 0x55844d922030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 -- 172.21.15.143:0/530801729 >> 172.21.15.143:0/530801729 conn(0x55844d9144e0 msgr2=0x55844d913350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 -- 172.21.15.143:0/530801729 shutdown_connections 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.562+0000 7fa844486740 1 -- 172.21.15.143:0/530801729 wait complete. 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 -- start start 2024-09-10T10:47:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 Processor -- start 2024-09-10T10:47:40.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 0x55844d934410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d9428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d928000 0x55844d942df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1adda0 con 0x55844d9226a0 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1ad6d0 con 0x55844d9276e0 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1aa910 con 0x55844d928000 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa844486740 10 client.0 _ll_register_callbacks cb 0x7fa82c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 0x55844d934410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d928000 0x55844d942df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 0x55844d934410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43194/0 (socket says 172.21.15.143:43194) 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa83b7fe640 1 -- 172.21.15.143:0/4046398650 learned_addr learned my addr 172.21.15.143:0/4046398650 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.563+0000 7fa83affd640 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d9428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa83b7fe640 1 -- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d928000 msgr2=0x55844d942df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d928000 0x55844d942df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa83b7fe640 1 -- 172.21.15.143:0/4046398650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 msgr2=0x55844d9428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d9428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa83b7fe640 1 -- 172.21.15.143:0/4046398650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844d9434e0 con 0x55844d9226a0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa83b7fe640 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 0x55844d934410 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa8340988c0 tx=0x7fa834009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4df45d689631d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa81affd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa81affd640 1 -- 172.21.15.143:0/4046398650 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83409e060 con 0x55844d9226a0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa81affd640 1 -- 172.21.15.143:0/4046398650 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa83406d0b0 con 0x55844d9226a0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c02a910 con 0x55844d9226a0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa81affd640 1 -- 172.21.15.143:0/4046398650 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa83409b850 con 0x55844d9226a0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa844486740 10 client.16179 fetch_fsmap learned FSMap version 30 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55844d9a5f80 con 0x55844d9226a0 2024-09-10T10:47:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.564+0000 7fa81affd640 1 -- 172.21.15.143:0/4046398650 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa82c02a910 con 0x55844d9226a0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa81affd640 1 -- 172.21.15.143:0/4046398650 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa83409c150 con 0x55844d9226a0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa844486740 10 client.16179 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa844486740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa844486740 20 client.16179 populate_metadata read hostname 'smithi143' 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa844486740 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d9a63c0 0x55844d9a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa844486740 4 client.16179 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55844d88b850 con 0x55844d9a63c0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa83affd640 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d9a63c0 0x55844d9a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa81affd640 10 client.16179 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.565+0000 7fa83affd640 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d9a63c0 0x55844d9a87e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260cb989829fa893 server_cookie=af678459aca8cf7b in_seq=0 out_seq=0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa81affd640 1 -- 172.21.15.143:0/4046398650 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55844d88b850 con 0x55844d9a63c0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa81affd640 10 client.16179 handle_command_reply: tid=0 2024-09-10T10:47:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 client.16179 shutdown 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 msgr2=0x55844d934410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 0x55844d934410 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa8340988c0 tx=0x7fa834009a40 comp rx=0 tx=0).stop 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 shutdown_connections 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55844d9a63c0 0x55844d9a87e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55844d928000 0x55844d942df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa81affd640 0 client.16179 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55844d9276e0 0x55844d9428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 --2- 172.21.15.143:0/4046398650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55844d9226a0 0x55844d934410 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 >> 172.21.15.143:0/4046398650 conn(0x55844d9144e0 msgr2=0x55844d913b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 shutdown_connections 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 1 -- 172.21.15.143:0/4046398650 wait complete. 2024-09-10T10:47:40.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.566+0000 7fa844486740 20 client.16179 trim_cache size 0 max 16384 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: "6e4f9feb-c385-4155-9032-9e489544d44a": { 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "6e4f9feb-c385-4155-9032-9e489544d44a", 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:47:40.572 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:47:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.567+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa8280a2c00 msgr2=0x7fa8280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.567+0000 7fa8420ad640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa8280a2c00 0x7fa8280a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009d80 tx=0x7fa82c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.567+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 msgr2=0x7fa83c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.567+0000 7fa8420ad640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 0x7fa83c1983b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa83400ebc0 tx=0x7fa83400c6a0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 shutdown_connections 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa8280a2c00 0x7fa8280a50c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83c106a80 0x7fa83c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83c101420 0x7fa83c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 --2- 172.21.15.143:0/2759411554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa83c100a80 0x7fa83c1983b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 >> 172.21.15.143:0/2759411554 conn(0x7fa83c0fc7a0 msgr2=0x7fa83c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 shutdown_connections 2024-09-10T10:47:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:47:40.568+0000 7fa8420ad640 1 -- 172.21.15.143:0/2759411554 wait complete. 2024-09-10T10:47:40.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:40 smithi143 ceph-mon[31057]: pgmap v429: 129 pgs: 129 active+clean; 41 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-10T10:47:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:40 smithi178 ceph-mon[34813]: pgmap v429: 129 pgs: 129 active+clean; 41 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-10T10:47:40.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:40 smithi144 ceph-mon[34230]: pgmap v429: 129 pgs: 129 active+clean; 41 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-10T10:47:41.424 INFO:tasks.cephfs.filesystem:scrub status for tag:6e4f9feb-c385-4155-9032-9e489544d44a - {'path': '/', 'tag': '6e4f9feb-c385-4155-9032-9e489544d44a', 'options': 'recursive,force'} 2024-09-10T10:47:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:42 smithi178 ceph-mon[34813]: pgmap v430: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2024-09-10T10:47:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:42 smithi144 ceph-mon[34230]: pgmap v430: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2024-09-10T10:47:43.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:42 smithi143 ceph-mon[31057]: pgmap v430: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 184 op/s 2024-09-10T10:47:43.590 INFO:tasks.workunit.client.0.smithi143.stdout: 14 6405 716 4359 596 56 616 2024-09-10T10:47:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:44 smithi178 ceph-mon[34813]: pgmap v431: 129 pgs: 129 active+clean; 63 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2024-09-10T10:47:44.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:44 smithi144 ceph-mon[34230]: pgmap v431: 129 pgs: 129 active+clean; 63 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2024-09-10T10:47:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:44 smithi143 ceph-mon[31057]: pgmap v431: 129 pgs: 129 active+clean; 63 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2024-09-10T10:47:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:46 smithi178 ceph-mon[34813]: pgmap v432: 129 pgs: 129 active+clean; 73 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 234 op/s 2024-09-10T10:47:46.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:46 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:46 smithi144 ceph-mon[34230]: pgmap v432: 129 pgs: 129 active+clean; 73 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 234 op/s 2024-09-10T10:47:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:46 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:46 smithi143 ceph-mon[31057]: pgmap v432: 129 pgs: 129 active+clean; 73 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 234 op/s 2024-09-10T10:47:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:46 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:47:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:48 smithi178 ceph-mon[34813]: pgmap v433: 129 pgs: 129 active+clean; 81 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 275 op/s 2024-09-10T10:47:48.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:48 smithi144 ceph-mon[34230]: pgmap v433: 129 pgs: 129 active+clean; 81 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 275 op/s 2024-09-10T10:47:49.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:48 smithi143 ceph-mon[31057]: pgmap v433: 129 pgs: 129 active+clean; 81 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 275 op/s 2024-09-10T10:47:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:50 smithi178 ceph-mon[34813]: pgmap v434: 129 pgs: 129 active+clean; 91 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 283 op/s 2024-09-10T10:47:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:50 smithi144 ceph-mon[34230]: pgmap v434: 129 pgs: 129 active+clean; 91 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 283 op/s 2024-09-10T10:47:51.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:50 smithi143 ceph-mon[31057]: pgmap v434: 129 pgs: 129 active+clean; 91 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 283 op/s 2024-09-10T10:47:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:52 smithi178 ceph-mon[34813]: pgmap v435: 129 pgs: 129 active+clean; 101 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 294 op/s 2024-09-10T10:47:52.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:52 smithi144 ceph-mon[34230]: pgmap v435: 129 pgs: 129 active+clean; 101 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 294 op/s 2024-09-10T10:47:53.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:52 smithi143 ceph-mon[31057]: pgmap v435: 129 pgs: 129 active+clean; 101 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 294 op/s 2024-09-10T10:47:53.591 INFO:tasks.workunit.client.0.smithi143.stdout: 28 1278 602 2 595 0 1162 2024-09-10T10:47:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:54 smithi178 ceph-mon[34813]: pgmap v436: 129 pgs: 129 active+clean; 114 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 293 op/s 2024-09-10T10:47:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:54 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:47:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:54 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:47:54.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:54 smithi144 ceph-mon[34230]: pgmap v436: 129 pgs: 129 active+clean; 114 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 293 op/s 2024-09-10T10:47:54.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:54 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:47:54.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:54 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:47:55.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:54 smithi143 ceph-mon[31057]: pgmap v436: 129 pgs: 129 active+clean; 114 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 293 op/s 2024-09-10T10:47:55.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:54 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:47:55.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:54 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:47:55.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:55 smithi178 ceph-mon[34813]: pgmap v437: 129 pgs: 129 active+clean; 121 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-10T10:47:55.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:55 smithi144 ceph-mon[34230]: pgmap v437: 129 pgs: 129 active+clean; 121 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-10T10:47:56.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:55 smithi143 ceph-mon[31057]: pgmap v437: 129 pgs: 129 active+clean; 121 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-10T10:47:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:57 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:47:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:57 smithi178 ceph-mon[34813]: pgmap v438: 129 pgs: 129 active+clean; 129 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-09-10T10:47:57.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:57 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:47:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:57 smithi144 ceph-mon[34230]: pgmap v438: 129 pgs: 129 active+clean; 129 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-09-10T10:47:58.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:57 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:47:58.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:57 smithi143 ceph-mon[31057]: pgmap v438: 129 pgs: 129 active+clean; 129 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 322 op/s 2024-09-10T10:47:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:47:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:47:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:47:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:58 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:47:58.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:47:58.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:47:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:47:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:58 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:47:59.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:47:59.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:47:59.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:47:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:58 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:47:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:47:59 smithi178 ceph-mon[34813]: pgmap v439: 129 pgs: 129 active+clean; 138 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 324 op/s 2024-09-10T10:47:59.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:47:59 smithi144 ceph-mon[34230]: pgmap v439: 129 pgs: 129 active+clean; 138 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 324 op/s 2024-09-10T10:48:00.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:47:59 smithi143 ceph-mon[31057]: pgmap v439: 129 pgs: 129 active+clean; 138 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 324 op/s 2024-09-10T10:48:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:02 smithi178 ceph-mon[34813]: pgmap v440: 129 pgs: 129 active+clean; 149 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-09-10T10:48:02.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:02 smithi144 ceph-mon[34230]: pgmap v440: 129 pgs: 129 active+clean; 149 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-09-10T10:48:03.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:02 smithi143 ceph-mon[31057]: pgmap v440: 129 pgs: 129 active+clean; 149 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-09-10T10:48:03.591 INFO:tasks.workunit.client.0.smithi143.stdout: 40 599 625 92 448 0 1158 2024-09-10T10:48:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:04 smithi178 ceph-mon[34813]: pgmap v441: 129 pgs: 129 active+clean; 159 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 320 op/s 2024-09-10T10:48:04.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:04 smithi144 ceph-mon[34230]: pgmap v441: 129 pgs: 129 active+clean; 159 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 320 op/s 2024-09-10T10:48:05.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:04 smithi143 ceph-mon[31057]: pgmap v441: 129 pgs: 129 active+clean; 159 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.5 MiB/s wr, 320 op/s 2024-09-10T10:48:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:06 smithi178 ceph-mon[34813]: pgmap v442: 129 pgs: 129 active+clean; 166 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 318 op/s 2024-09-10T10:48:06.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:06 smithi144 ceph-mon[34230]: pgmap v442: 129 pgs: 129 active+clean; 166 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 318 op/s 2024-09-10T10:48:07.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:06 smithi143 ceph-mon[31057]: pgmap v442: 129 pgs: 129 active+clean; 166 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.0 MiB/s wr, 318 op/s 2024-09-10T10:48:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:08 smithi178 ceph-mon[34813]: pgmap v443: 129 pgs: 129 active+clean; 175 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 344 op/s 2024-09-10T10:48:08.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:08 smithi144 ceph-mon[34230]: pgmap v443: 129 pgs: 129 active+clean; 175 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 344 op/s 2024-09-10T10:48:09.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:08 smithi143 ceph-mon[31057]: pgmap v443: 129 pgs: 129 active+clean; 175 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 344 op/s 2024-09-10T10:48:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:10 smithi178 ceph-mon[34813]: pgmap v444: 129 pgs: 129 active+clean; 192 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 338 op/s 2024-09-10T10:48:10.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:10 smithi144 ceph-mon[34230]: pgmap v444: 129 pgs: 129 active+clean; 192 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 338 op/s 2024-09-10T10:48:11.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:10 smithi143 ceph-mon[31057]: pgmap v444: 129 pgs: 129 active+clean; 192 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.8 MiB/s wr, 338 op/s 2024-09-10T10:48:11.428 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:48:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.264+0000 7fbbb1578640 1 -- 172.21.15.143:0/3695357866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 msgr2=0x7fbbac107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.264+0000 7fbbb1578640 1 --2- 172.21.15.143:0/3695357866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac107b90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009930 tx=0x7fbba002a420 comp rx=0 tx=0).stop 2024-09-10T10:48:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 -- 172.21.15.143:0/3695357866 shutdown_connections 2024-09-10T10:48:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 --2- 172.21.15.143:0/3695357866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1082a0 0x7fbbac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 --2- 172.21.15.143:0/3695357866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac107b90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 --2- 172.21.15.143:0/3695357866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 0x7fbbac103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 -- 172.21.15.143:0/3695357866 >> 172.21.15.143:0/3695357866 conn(0x7fbbac0fc7d0 msgr2=0x7fbbac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 -- 172.21.15.143:0/3695357866 shutdown_connections 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 -- 172.21.15.143:0/3695357866 wait complete. 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.265+0000 7fbbb1578640 1 Processor -- start 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 -- start start 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 0x7fbbac1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1082a0 0x7fbbac19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111f50 con 0x7fbbac1036e0 2024-09-10T10:48:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111dd0 con 0x7fbbac1082a0 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbb1578640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1120d0 con 0x7fbbac102d30 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbaaffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 0x7fbbac1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34932/0 (socket says 172.21.15.143:34932) 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/1450932654 learned_addr learned my addr 172.21.15.143:0/1450932654 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1082a0 0x7fbbac19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.266+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/1450932654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 msgr2=0x7fbbac1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 0x7fbbac1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/1450932654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1082a0 msgr2=0x7fbbac19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1082a0 0x7fbbac19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/1450932654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19d4a0 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 0x7fbbac1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac198a30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbba0004770 tx=0x7fbba0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbb93fff640 1 -- 172.21.15.143:0/1450932654 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba003f030 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbb93fff640 1 -- 172.21.15.143:0/1450932654 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba00318a0 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac19d730 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbb93fff640 1 -- 172.21.15.143:0/1450932654 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0035070 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.267+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a5070 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbb93fff640 1 -- 172.21.15.143:0/1450932654 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbba0042030 con 0x7fbbac1036e0 2024-09-10T10:48:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 Processor -- start 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 -- start start 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f3dc0 0x5616e82fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e8300a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616e82592e0 con 0x5616e82f34a0 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616e8200ce0 con 0x5616e82f3dc0 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbb3951740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5616e82552a0 con 0x5616e82fe680 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34948/0 (socket says 172.21.15.143:34948) 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbab7fe640 1 -- 172.21.15.143:0/2107256534 learned_addr learned my addr 172.21.15.143:0/2107256534 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbb93fff640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbb940a2b80 0x7fbb940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.270+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e8300a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbb940a2b80 0x7fbb940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 -- 172.21.15.143:0/2107256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 msgr2=0x5616e8300a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e8300a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 -- 172.21.15.143:0/2107256534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f3dc0 msgr2=0x5616e82fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f3dc0 0x5616e82fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f3dc0 0x5616e82fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 -- 172.21.15.143:0/2107256534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e81e5440 con 0x5616e82f34a0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbb93fff640 1 -- 172.21.15.143:0/1450932654 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbba00ef840 con 0x7fbbac1036e0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f3dc0 0x5616e82fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f3880 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005c40 tx=0x7fbba4012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87176d4a0a8027 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbb917fa640 1 -- 172.21.15.143:0/2107256534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4022020 con 0x5616e82f34a0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbb940a2b80 0x7fbb940a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbba4023000 tx=0x7fbba4021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbb917fa640 1 -- 172.21.15.143:0/2107256534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba401ec80 con 0x5616e82f34a0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.271+0000 7fbb917fa640 1 -- 172.21.15.143:0/2107256534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba400fa70 con 0x5616e82f34a0 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 -- 172.21.15.143:0/2107256534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 msgr2=0x5616e82f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f3880 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005c40 tx=0x7fbba4012ee0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 -- 172.21.15.143:0/2107256534 shutdown_connections 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e8300a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f3dc0 0x5616e82fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2107256534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f3880 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 -- 172.21.15.143:0/2107256534 >> 172.21.15.143:0/2107256534 conn(0x5616e82efa30 msgr2=0x5616e82efe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 -- 172.21.15.143:0/2107256534 shutdown_connections 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 -- 172.21.15.143:0/2107256534 wait complete. 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 -- start start 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.272+0000 7fbbb3951740 1 Processor -- start 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e82fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e82fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5616e8301250 con 0x5616e82f3dc0 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616e8214050 con 0x5616e82f34a0 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5616e83013d0 con 0x5616e82fe680 2024-09-10T10:48:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e82fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbb3951740 10 client.0 _ll_register_callbacks cb 0x5616e8254160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e82fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34952/0 (socket says 172.21.15.143:34952) 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/67477675 learned_addr learned my addr 172.21.15.143:0/67477675 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbab7fe640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e82fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 -- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 msgr2=0x5616e82fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e82fbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 -- 172.21.15.143:0/67477675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 msgr2=0x5616e82fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e82fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 -- 172.21.15.143:0/67477675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e7fca3c0 con 0x5616e82f34a0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fe0b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0139a0 tx=0x7fbb9c013e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118c3157cfca77e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.273+0000 7fbb8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:48:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbb8eff5640 1 -- 172.21.15.143:0/67477675 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c020060 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbb8eff5640 1 -- 172.21.15.143:0/67477675 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbb9c004050 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba401ec80 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbb8eff5640 1 -- 172.21.15.143:0/67477675 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c027050 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 10 client.25577 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5616e830ccc0 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbb8eff5640 1 -- 172.21.15.143:0/67477675 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba401ec80 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbb8eff5640 1 -- 172.21.15.143:0/67477675 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fbb9c00f3c0 con 0x5616e82f34a0 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 10 client.25577 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 20 client.25577 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e830e180 0x5616e83105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 4 client.25577 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616e8266850 con 0x5616e830e180 2024-09-10T10:48:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.274+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e830e180 0x5616e83105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.275+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e830e180 0x5616e83105d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa663cb2b2c1892 server_cookie=ef0ddb5d3145c73a in_seq=0 out_seq=0 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.275+0000 7fbb8eff5640 10 client.25577 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.275+0000 7fbb8eff5640 1 -- 172.21.15.143:0/67477675 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616e8266850 con 0x5616e830e180 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.275+0000 7fbb8eff5640 10 client.25577 handle_command_reply: tid=0 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.275+0000 7fbbb3951740 1 client.25577 shutdown 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 msgr2=0x5616e82fe0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fe0b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0139a0 tx=0x7fbb9c013e60 comp rx=0 tx=0).stop 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 shutdown_connections 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e830e180 0x5616e83105d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e82fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbb8eff5640 0 client.25577 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e82fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 --2- 172.21.15.143:0/67477675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fe0b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 >> 172.21.15.143:0/67477675 conn(0x5616e82efa30 msgr2=0x5616e82f5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 shutdown_connections 2024-09-10T10:48:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 1 -- 172.21.15.143:0/67477675 wait complete. 2024-09-10T10:48:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.276+0000 7fbbb3951740 20 client.25577 trim_cache size 0 max 16384 2024-09-10T10:48:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.295+0000 7fbbb3951740 1 Processor -- start 2024-09-10T10:48:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.295+0000 7fbbb3951740 1 -- start start 2024-09-10T10:48:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.295+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.295+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e8315ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.295+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e83163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbb3951740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1ac930 con 0x5616e82f3dc0 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbb3951740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1ae4a0 con 0x5616e82f34a0 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbb3951740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1ad000 con 0x5616e82fe680 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e83163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e8315ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e83163e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45494/0 (socket says 172.21.15.143:45494) 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e8315ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34956/0 (socket says 172.21.15.143:34956) 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbab7fe640 1 -- 172.21.15.143:0/3299560424 learned_addr learned my addr 172.21.15.143:0/3299560424 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/3299560424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 msgr2=0x5616e83163e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e83163e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/3299560424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 msgr2=0x5616e82fac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.296+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/3299560424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e82f98b0 con 0x5616e82f3dc0 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.297+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.297+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e8315ea0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbba00ecfe0 tx=0x7fbba003bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b746e97843faef85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.297+0000 7fbb90ff9640 1 -- 172.21.15.143:0/3299560424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0049040 con 0x5616e82f3dc0 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.297+0000 7fbb90ff9640 1 -- 172.21.15.143:0/3299560424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba00c49b0 con 0x5616e82f3dc0 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.297+0000 7fbb90ff9640 1 -- 172.21.15.143:0/3299560424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba00c4520 con 0x5616e82f3dc0 2024-09-10T10:48:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 -- 172.21.15.143:0/3299560424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 msgr2=0x5616e8315ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e8315ea0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbba00ecfe0 tx=0x7fbba003bd30 comp rx=0 tx=0).stop 2024-09-10T10:48:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 -- 172.21.15.143:0/3299560424 shutdown_connections 2024-09-10T10:48:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82fe680 0x5616e83163e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f3dc0 0x5616e8315ea0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 --2- 172.21.15.143:0/3299560424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82f34a0 0x5616e82fac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.298+0000 7fbbb3951740 1 -- 172.21.15.143:0/3299560424 >> 172.21.15.143:0/3299560424 conn(0x5616e82efa30 msgr2=0x5616e82fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.299+0000 7fbbb3951740 1 -- 172.21.15.143:0/3299560424 shutdown_connections 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.299+0000 7fbbb3951740 1 -- 172.21.15.143:0/3299560424 wait complete. 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.299+0000 7fbbb3951740 1 -- start start 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.299+0000 7fbbb3951740 1 Processor -- start 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82f3dc0 0x5616e82f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82fe680 0x5616e82f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1adda0 con 0x5616e82f34a0 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1ad6d0 con 0x5616e82fe680 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1aa910 con 0x5616e82f3dc0 2024-09-10T10:48:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82f3dc0 0x5616e82f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82f3dc0 0x5616e82f87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45508/0 (socket says 172.21.15.143:45508) 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaa7fc640 1 -- 172.21.15.143:0/2076277176 learned_addr learned my addr 172.21.15.143:0/2076277176 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbb3951740 10 client.0 _ll_register_callbacks cb 0x7fbb9c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaaffd640 1 -- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82f3dc0 msgr2=0x5616e82f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82f3dc0 0x5616e82f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaaffd640 1 -- 172.21.15.143:0/2076277176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82fe680 msgr2=0x5616e82f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82fe680 0x5616e82f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.300+0000 7fbbaaffd640 1 -- 172.21.15.143:0/2076277176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e8377e70 con 0x5616e82f34a0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba00c49b0 con 0x5616e82f34a0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbb8eff5640 10 client.16197 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbbaaffd640 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f82a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000c00 tx=0x7fbb9c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ecd139b987a08f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbb8eff5640 1 -- 172.21.15.143:0/2076277176 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c030060 con 0x5616e82f34a0 2024-09-10T10:48:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbb8eff5640 1 -- 172.21.15.143:0/2076277176 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbb9c011c90 con 0x5616e82f34a0 2024-09-10T10:48:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbb8eff5640 1 -- 172.21.15.143:0/2076277176 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c00fc00 con 0x5616e82f34a0 2024-09-10T10:48:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbbb3951740 10 client.16197 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.301+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5616e8378fe0 con 0x5616e82f34a0 2024-09-10T10:48:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbb8eff5640 1 -- 172.21.15.143:0/2076277176 <== mon.0 v2:172.21.15.143:3300/0 4 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fbb9c0050c0 con 0x5616e82f34a0 2024-09-10T10:48:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 10 client.16197 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 20 client.16197 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e83795f0 0x5616e837ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 4 client.16197 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616e8260d60 con 0x5616e83795f0 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e83795f0 0x5616e837ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbb8eff5640 10 client.16197 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbaa7fc640 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e83795f0 0x5616e837ba30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2773e7f7b7709ca server_cookie=aa3011a44aeec12a in_seq=0 out_seq=0 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbb8eff5640 1 -- 172.21.15.143:0/2076277176 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5616e8260d60 con 0x5616e83795f0 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbb8eff5640 10 client.16197 handle_command_reply: tid=0 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.302+0000 7fbbb3951740 1 client.16197 shutdown 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 msgr2=0x5616e82f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f82a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000c00 tx=0x7fbb9c0117b0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 shutdown_connections 2024-09-10T10:48:12.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616e82fe680 0x5616e82f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5616e83795f0 0x5616e837ba30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5616e82f3dc0 0x5616e82f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 --2- 172.21.15.143:0/2076277176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5616e82f34a0 0x5616e82f82a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 >> 172.21.15.143:0/2076277176 conn(0x5616e82efa30 msgr2=0x5616e82f4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbb8eff5640 0 client.16197 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 shutdown_connections 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 1 -- 172.21.15.143:0/2076277176 wait complete. 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb3951740 20 client.16197 trim_cache size 0 max 16384 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:48:12.308 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbb940a2b80 msgr2=0x7fbb940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb1578640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbb940a2b80 0x7fbb940a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbba4023000 tx=0x7fbba4021000 comp rx=0 tx=0).stop 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 msgr2=0x7fbbac198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.303+0000 7fbbb1578640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac198a30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbba0004770 tx=0x7fbba0004270 comp rx=0 tx=0).stop 2024-09-10T10:48:12.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 shutdown_connections 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbb940a2b80 0x7fbb940a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1082a0 0x7fbbac19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbac1036e0 0x7fbbac198a30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 --2- 172.21.15.143:0/1450932654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbac102d30 0x7fbbac1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 >> 172.21.15.143:0/1450932654 conn(0x7fbbac0fc7d0 msgr2=0x7fbbac105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 shutdown_connections 2024-09-10T10:48:12.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:12.304+0000 7fbbb1578640 1 -- 172.21.15.143:0/1450932654 wait complete. 2024-09-10T10:48:12.894 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:12 smithi143 ceph-mon[31057]: pgmap v445: 129 pgs: 129 active+clean; 202 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 321 op/s 2024-09-10T10:48:12.894 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:12 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:48:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:12 smithi178 ceph-mon[34813]: pgmap v445: 129 pgs: 129 active+clean; 202 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 321 op/s 2024-09-10T10:48:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:12 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:48:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:12 smithi144 ceph-mon[34230]: pgmap v445: 129 pgs: 129 active+clean; 202 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 321 op/s 2024-09-10T10:48:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:12 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:48:13.177 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:48:13.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:48:13.480 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:13.591 INFO:tasks.workunit.client.0.smithi143.stdout: 52 1108 628 183 753 0 874 2024-09-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 -- 172.21.15.143:0/3698795431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c074f70 msgr2=0x7f726c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 --2- 172.21.15.143:0/3698795431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c074f70 0x7f726c10df10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7254009900 tx=0x7f725402a410 comp rx=0 tx=0).stop 2024-09-10T10:48:14.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 -- 172.21.15.143:0/3698795431 shutdown_connections 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 --2- 172.21.15.143:0/3698795431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f726c10e450 0x7f726c1108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 --2- 172.21.15.143:0/3698795431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c074f70 0x7f726c10df10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 --2- 172.21.15.143:0/3698795431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f726c0745c0 0x7f726c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 -- 172.21.15.143:0/3698795431 >> 172.21.15.143:0/3698795431 conn(0x7f726c0fd8d0 msgr2=0x7f726c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 -- 172.21.15.143:0/3698795431 shutdown_connections 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.120+0000 7f727097c640 1 -- 172.21.15.143:0/3698795431 wait complete. 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 Processor -- start 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 -- start start 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 0x7f726c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f726c074f70 0x7f726c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f726c10e450 0x7f726c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1140a0 con 0x7f726c0745c0 2024-09-10T10:48:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f726c113f20 con 0x7f726c074f70 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.121+0000 7f727097c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f726c114220 con 0x7f726c10e450 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 0x7f726c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 0x7f726c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34978/0 (socket says 172.21.15.143:34978) 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 -- 172.21.15.143:0/2758628809 learned_addr learned my addr 172.21.15.143:0/2758628809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726ad9d640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f726c10e450 0x7f726c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f7263fff640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f726c074f70 0x7f726c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 -- 172.21.15.143:0/2758628809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f726c10e450 msgr2=0x7f726c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f726c10e450 0x7f726c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 -- 172.21.15.143:0/2758628809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f726c074f70 msgr2=0x7f726c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f726c074f70 0x7f726c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 -- 172.21.15.143:0/2758628809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c19f4f0 con 0x7f726c0745c0 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f726a59c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 0x7f726c19a540 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c920 tx=0x7f725c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c007c70 con 0x7f726c0745c0 2024-09-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c19f7e0 con 0x7f726c0745c0 2024-09-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f725c007e10 con 0x7f726c0745c0 2024-09-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c018820 con 0x7f726c0745c0 2024-09-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.122+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c1a70c0 con 0x7f726c0745c0 2024-09-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.124+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726c0749a0 con 0x7f726c0745c0 2024-09-10T10:48:14.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.125+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f725c0189c0 con 0x7f726c0745c0 2024-09-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.126+0000 7f72637fe640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72580a2c00 0x7f72580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.127+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f725c0cf480 con 0x7f726c0745c0 2024-09-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.127+0000 7f7263fff640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72580a2c00 0x7f72580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:14.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.127+0000 7f7263fff640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72580a2c00 0x7f72580a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7254031440 tx=0x7f7254005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.132+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f725c014030 con 0x7f726c0745c0 2024-09-10T10:48:14.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.394+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f726c1a72f0 con 0x7f726c0745c0 2024-09-10T10:48:14.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:48:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.395+0000 7f72637fe640 1 -- 172.21.15.143:0/2758628809 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7f725c09c020 con 0x7f726c0745c0 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72580a2c00 msgr2=0x7f72580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72580a2c00 0x7f72580a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7254031440 tx=0x7f7254005b50 comp rx=0 tx=0).stop 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 msgr2=0x7f726c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 0x7f726c19a540 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c920 tx=0x7f725c00cde0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 shutdown_connections 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72580a2c00 0x7f72580a50c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f726c10e450 0x7f726c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f726c074f70 0x7f726c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 --2- 172.21.15.143:0/2758628809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f726c0745c0 0x7f726c19a540 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:14.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 >> 172.21.15.143:0/2758628809 conn(0x7f726c0fd8d0 msgr2=0x7f726c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 shutdown_connections 2024-09-10T10:48:14.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:14.399+0000 7f727097c640 1 -- 172.21.15.143:0/2758628809 wait complete. 2024-09-10T10:48:14.781 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:14 smithi143 ceph-mon[31057]: pgmap v446: 129 pgs: 129 active+clean; 212 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 319 op/s 2024-09-10T10:48:14.782 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:14 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2758628809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:48:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:14 smithi178 ceph-mon[34813]: pgmap v446: 129 pgs: 129 active+clean; 212 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 319 op/s 2024-09-10T10:48:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:14 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2758628809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:48:14.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:14 smithi144 ceph-mon[34230]: pgmap v446: 129 pgs: 129 active+clean; 212 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 319 op/s 2024-09-10T10:48:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:14 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2758628809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:48:15.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:48:15.543 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.046+0000 7efe08b28640 1 -- 172.21.15.143:0/3997510155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 msgr2=0x7efe0410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.046+0000 7efe08b28640 1 --2- 172.21.15.143:0/3997510155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0410d960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009cd0 tx=0x7efdfc02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 -- 172.21.15.143:0/3997510155 shutdown_connections 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 --2- 172.21.15.143:0/3997510155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0410d960 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 --2- 172.21.15.143:0/3997510155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04100a90 0x7efe04100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 --2- 172.21.15.143:0/3997510155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04106a90 0x7efe04106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 -- 172.21.15.143:0/3997510155 >> 172.21.15.143:0/3997510155 conn(0x7efe040fc7d0 msgr2=0x7efe040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:16.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 -- 172.21.15.143:0/3997510155 shutdown_connections 2024-09-10T10:48:16.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.047+0000 7efe08b28640 1 -- 172.21.15.143:0/3997510155 wait complete. 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 Processor -- start 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 -- start start 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04100a90 0x7efe0419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 0x7efe0419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe04110d40 con 0x7efe04101430 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe04110bc0 con 0x7efe04106a90 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe08b28640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe04110ec0 con 0x7efe04100a90 2024-09-10T10:48:16.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe01d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 0x7efe0419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04100a90 0x7efe0419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe02d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 0x7efe0419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56122/0 (socket says 172.21.15.143:56122) 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe01d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35004/0 (socket says 172.21.15.143:35004) 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe02d9d640 1 -- 172.21.15.143:0/1196163556 learned_addr learned my addr 172.21.15.143:0/1196163556 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.048+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04100a90 0x7efe0419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45540/0 (socket says 172.21.15.143:45540) 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe02d9d640 1 -- 172.21.15.143:0/1196163556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04100a90 msgr2=0x7efe0419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe02d9d640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04100a90 0x7efe0419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe02d9d640 1 -- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 msgr2=0x7efe0419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe02d9d640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe02d9d640 1 -- 172.21.15.143:0/1196163556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0419f4f0 con 0x7efe04106a90 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe01d9b640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:16.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe02d9d640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 0x7efe0419ee10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdfc033040 tx=0x7efdfc009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efdf37fe640 1 -- 172.21.15.143:0/1196163556 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc004040 con 0x7efe04106a90 2024-09-10T10:48:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efdf37fe640 1 -- 172.21.15.143:0/1196163556 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdfc004690 con 0x7efe04106a90 2024-09-10T10:48:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efdf37fe640 1 -- 172.21.15.143:0/1196163556 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc030da0 con 0x7efe04106a90 2024-09-10T10:48:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.049+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0419f780 con 0x7efe04106a90 2024-09-10T10:48:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.050+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041a70c0 con 0x7efe04106a90 2024-09-10T10:48:16.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.052+0000 7efdf37fe640 1 -- 172.21.15.143:0/1196163556 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efdfc03b400 con 0x7efe04106a90 2024-09-10T10:48:16.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.052+0000 7efe0af01740 1 Processor -- start 2024-09-10T10:48:16.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.052+0000 7efe0af01740 1 -- start start 2024-09-10T10:48:16.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0af01740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0af01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 0x561cf19cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0af01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0af01740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561cf19332f0 con 0x561cf19ceb40 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0af01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cf18dacf0 con 0x561cf19d8370 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0af01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561cf192f2b0 con 0x561cf19cd5d0 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe01d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 0x561cf19cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 0x561cf19cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45542/0 (socket says 172.21.15.143:45542) 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 -- 172.21.15.143:0/477766845 learned_addr learned my addr 172.21.15.143:0/477766845 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efdf37fe640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efdec0a2c00 0x7efdec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe02d9d640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efdec0a2c00 0x7efdec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efdf37fe640 1 -- 172.21.15.143:0/1196163556 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efdfc02e030 con 0x7efe04106a90 2024-09-10T10:48:16.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 -- 172.21.15.143:0/477766845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 msgr2=0x561cf19da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 -- 172.21.15.143:0/477766845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 msgr2=0x561cf19cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe0259c640 1 -- 172.21.15.143:0/477766845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf18bf3a0 con 0x561cf19cd5d0 2024-09-10T10:48:16.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.053+0000 7efe01d9b640 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:16.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.054+0000 7efe0259c640 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 0x561cf19cda10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009ca0 tx=0x7efdf4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0cdbda1c3862de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.054+0000 7efdf0ff9640 1 -- 172.21.15.143:0/477766845 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf401e070 con 0x561cf19cd5d0 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.054+0000 7efe02d9d640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efdec0a2c00 0x7efdec0a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0bc550 tx=0x7efdfc040000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.054+0000 7efdf0ff9640 1 -- 172.21.15.143:0/477766845 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf4019ae0 con 0x561cf19cd5d0 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.054+0000 7efdf0ff9640 1 -- 172.21.15.143:0/477766845 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf400e940 con 0x561cf19cd5d0 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 -- 172.21.15.143:0/477766845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 msgr2=0x561cf19cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 0x561cf19cda10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009ca0 tx=0x7efdf4011810 comp rx=0 tx=0).stop 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 -- 172.21.15.143:0/477766845 shutdown_connections 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19cd5d0 0x561cf19cda10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 --2- 172.21.15.143:0/477766845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 -- 172.21.15.143:0/477766845 >> 172.21.15.143:0/477766845 conn(0x561cf1996310 msgr2=0x561cf19c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 -- 172.21.15.143:0/477766845 shutdown_connections 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.055+0000 7efe0af01740 1 -- 172.21.15.143:0/477766845 wait complete. 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 -- start start 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 Processor -- start 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561cf19db070 con 0x561cf19ceb40 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:56138/0 (socket says 172.21.15.143:56138) 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0259c640 1 -- 172.21.15.143:0/3446734488 learned_addr learned my addr 172.21.15.143:0/3446734488 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe01d9b640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.056+0000 7efe0af01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cf18ee060 con 0x561cf19d8370 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561cf19db1f0 con 0x561cf19d5d30 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0af01740 10 client.0 _ll_register_callbacks cb 0x561cf16418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0259c640 1 -- 172.21.15.143:0/3446734488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 msgr2=0x561cf19e4520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0259c640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0259c640 1 -- 172.21.15.143:0/3446734488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 msgr2=0x561cf19d52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0259c640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19d52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0259c640 1 -- 172.21.15.143:0/3446734488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf16a43c0 con 0x561cf19d8370 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe01d9b640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19d52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efe0259c640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d57f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009c70 tx=0x7efdf40099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5f6bd1fdc46e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.057+0000 7efde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efde9ffb640 1 -- 172.21.15.143:0/3446734488 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4027020 con 0x561cf19d8370 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efde9ffb640 1 -- 172.21.15.143:0/3446734488 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efdf4004220 con 0x561cf19d8370 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf4019ae0 con 0x561cf19d8370 2024-09-10T10:48:16.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efde9ffb640 1 -- 172.21.15.143:0/3446734488 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efdf401e070 con 0x561cf19d8370 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 10 client.25592 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cf19e7180 con 0x561cf19d8370 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efde9ffb640 1 -- 172.21.15.143:0/3446734488 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf4019ae0 con 0x561cf19d8370 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efde9ffb640 1 -- 172.21.15.143:0/3446734488 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7efdf40046c0 con 0x561cf19d8370 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 10 client.25592 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 10 client.25592 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 20 client.25592 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf19e84d0 0x561cf19ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 4 client.25592 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.058+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cf1940860 con 0x561cf19e84d0 2024-09-10T10:48:16.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.059+0000 7efe01d9b640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf19e84d0 0x561cf19ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.059+0000 7efde9ffb640 10 client.25592 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.059+0000 7efe01d9b640 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf19e84d0 0x561cf19ea8c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75f0ea44df86c82b server_cookie=925ddd96d788b454 in_seq=0 out_seq=0 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efde9ffb640 1 -- 172.21.15.143:0/3446734488 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cf1940860 con 0x561cf19e84d0 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efde9ffb640 10 client.25592 handle_command_reply: tid=0 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 client.25592 shutdown 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 msgr2=0x561cf19d57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d57f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009c70 tx=0x7efdf40099f0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 shutdown_connections 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf19e84d0 0x561cf19ea8c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d57f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 --2- 172.21.15.143:0/3446734488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19d52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 >> 172.21.15.143:0/3446734488 conn(0x561cf1996310 msgr2=0x561cf19cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:16.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efde9ffb640 0 client.25592 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:48:16.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 shutdown_connections 2024-09-10T10:48:16.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.060+0000 7efe0af01740 1 -- 172.21.15.143:0/3446734488 wait complete. 2024-09-10T10:48:16.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.061+0000 7efe0af01740 20 client.25592 trim_cache size 0 max 16384 2024-09-10T10:48:16.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 Processor -- start 2024-09-10T10:48:16.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 -- start start 2024-09-10T10:48:16.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19ceb40 0x561cf19e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 0x561cf19d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b0c60 con 0x561cf19d8370 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b27d0 con 0x561cf19ceb40 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.077+0000 7efe0af01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b1330 con 0x561cf19d5d30 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 0x561cf19d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 0x561cf19d9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35012/0 (socket says 172.21.15.143:35012) 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 -- 172.21.15.143:0/2751206400 learned_addr learned my addr 172.21.15.143:0/2751206400 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19ceb40 0x561cf19e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 -- 172.21.15.143:0/2751206400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 msgr2=0x561cf19d9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19d9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 -- 172.21.15.143:0/2751206400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19ceb40 msgr2=0x561cf19e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19ceb40 0x561cf19e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 -- 172.21.15.143:0/2751206400 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf19ca960 con 0x561cf19d8370 2024-09-10T10:48:16.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe0259c640 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19d9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:16.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 0x561cf19d9ca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efdf8020630 tx=0x7efdf8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933ea3eb9dffca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:16.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efdebfff640 1 -- 172.21.15.143:0/2751206400 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8012110 con 0x561cf19d8370 2024-09-10T10:48:16.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efdebfff640 1 -- 172.21.15.143:0/2751206400 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf8005020 con 0x561cf19d8370 2024-09-10T10:48:16.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.078+0000 7efdebfff640 1 -- 172.21.15.143:0/2751206400 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf80044e0 con 0x561cf19d8370 2024-09-10T10:48:16.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 -- 172.21.15.143:0/2751206400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 msgr2=0x561cf19d9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 0x561cf19d9ca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efdf8020630 tx=0x7efdf8005f90 comp rx=0 tx=0).stop 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 -- 172.21.15.143:0/2751206400 shutdown_connections 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19d8370 0x561cf19d9ca0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19d9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 --2- 172.21.15.143:0/2751206400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19ceb40 0x561cf19e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 -- 172.21.15.143:0/2751206400 >> 172.21.15.143:0/2751206400 conn(0x561cf1996310 msgr2=0x561cf19d7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 -- 172.21.15.143:0/2751206400 shutdown_connections 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 -- 172.21.15.143:0/2751206400 wait complete. 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 -- start start 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.079+0000 7efe0af01740 1 Processor -- start 2024-09-10T10:48:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19e0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b20d0 con 0x561cf19ceb40 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b1a00 con 0x561cf19d8370 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe041aec40 con 0x561cf19d5d30 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0af01740 10 client.0 _ll_register_callbacks cb 0x7efdf401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45554/0 (socket says 172.21.15.143:45554) 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0259c640 1 -- 172.21.15.143:0/2640918695 learned_addr learned my addr 172.21.15.143:0/2640918695 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19e0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35028/0 (socket says 172.21.15.143:35028) 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 -- 172.21.15.143:0/2640918695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 msgr2=0x561cf19e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 -- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 msgr2=0x561cf19d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe01d9b640 1 -- 172.21.15.143:0/2640918695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf19d2a70 con 0x561cf19ceb40 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.080+0000 7efe0259c640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe01d9b640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19e0480 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efdf8001eb0 tx=0x7efdf80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986809b14b2d66a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efde9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efde9ffb640 1 -- 172.21.15.143:0/2640918695 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8000e60 con 0x561cf19ceb40 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efde9ffb640 1 -- 172.21.15.143:0/2640918695 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efdf80051d0 con 0x561cf19ceb40 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf8005020 con 0x561cf19ceb40 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efde9ffb640 1 -- 172.21.15.143:0/2640918695 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efdf8045050 con 0x561cf19ceb40 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe0af01740 10 client.16221 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:16.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cf19cf8d0 con 0x561cf19ceb40 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efde9ffb640 1 -- 172.21.15.143:0/2640918695 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf8005020 con 0x561cf19ceb40 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efde9ffb640 1 -- 172.21.15.143:0/2640918695 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7efdf800f790 con 0x561cf19ceb40 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe0af01740 10 client.16221 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe0af01740 10 client.16221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.081+0000 7efe0af01740 20 client.16221 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf1a53050 0x561cf1a55440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 4 client.16221 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cf193ad70 con 0x561cf1a53050 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0259c640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf1a53050 0x561cf1a55440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efde9ffb640 10 client.16221 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0259c640 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf1a53050 0x561cf1a55440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6aeeb21b8e254461 server_cookie=2a7e94030e5500c1 in_seq=0 out_seq=0 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efde9ffb640 1 -- 172.21.15.143:0/2640918695 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cf193ad70 con 0x561cf1a53050 2024-09-10T10:48:16.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efde9ffb640 10 client.16221 handle_command_reply: tid=0 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 client.16221 shutdown 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 msgr2=0x561cf19e0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19e0480 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efdf8001eb0 tx=0x7efdf80096c0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 shutdown_connections 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.082+0000 7efe0af01740 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x561cf1a53050 0x561cf1a55440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cf19d8370 0x561cf19d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561cf19d5d30 0x561cf19e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 1 --2- 172.21.15.143:0/2640918695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561cf19ceb40 0x561cf19e0480 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efde9ffb640 0 client.16221 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 >> 172.21.15.143:0/2640918695 conn(0x561cf1996310 msgr2=0x561cf19d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 shutdown_connections 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 1 -- 172.21.15.143:0/2640918695 wait complete. 2024-09-10T10:48:16.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe0af01740 20 client.16221 trim_cache size 0 max 16384 2024-09-10T10:48:16.090 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efdec0a2c00 msgr2=0x7efdec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe08b28640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efdec0a2c00 0x7efdec0a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0bc550 tx=0x7efdfc040000 comp rx=0 tx=0).stop 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 msgr2=0x7efe0419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.083+0000 7efe08b28640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 0x7efe0419ee10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdfc033040 tx=0x7efdfc009360 comp rx=0 tx=0).stop 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 shutdown_connections 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efdec0a2c00 0x7efdec0a50c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe04106a90 0x7efe0419ee10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe04101430 0x7efe0419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 --2- 172.21.15.143:0/1196163556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe04100a90 0x7efe0419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:16.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 >> 172.21.15.143:0/1196163556 conn(0x7efe040fc7d0 msgr2=0x7efe0406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:16.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 shutdown_connections 2024-09-10T10:48:16.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:16.084+0000 7efe08b28640 1 -- 172.21.15.143:0/1196163556 wait complete. 2024-09-10T10:48:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:16 smithi178 ceph-mon[34813]: pgmap v447: 129 pgs: 129 active+clean; 218 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.4 MiB/s wr, 311 op/s 2024-09-10T10:48:16.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:16 smithi144 ceph-mon[34230]: pgmap v447: 129 pgs: 129 active+clean; 218 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.4 MiB/s wr, 311 op/s 2024-09-10T10:48:16.963 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:16 smithi143 ceph-mon[31057]: pgmap v447: 129 pgs: 129 active+clean; 218 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.4 MiB/s wr, 311 op/s 2024-09-10T10:48:17.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:48:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 -- 172.21.15.143:0/3962218545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c1027a0 msgr2=0x7f466c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 --2- 172.21.15.143:0/3962218545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c1027a0 0x7f466c1067f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f465c009900 tx=0x7f465c02a410 comp rx=0 tx=0).stop 2024-09-10T10:48:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 -- 172.21.15.143:0/3962218545 shutdown_connections 2024-09-10T10:48:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 --2- 172.21.15.143:0/3962218545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f466c106f20 0x7f466c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 --2- 172.21.15.143:0/3962218545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c1027a0 0x7f466c1067f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 --2- 172.21.15.143:0/3962218545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f466c101df0 0x7f466c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 -- 172.21.15.143:0/3962218545 >> 172.21.15.143:0/3962218545 conn(0x7f466c0fd8f0 msgr2=0x7f466c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 -- 172.21.15.143:0/3962218545 shutdown_connections 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 -- 172.21.15.143:0/3962218545 wait complete. 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 Processor -- start 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.914+0000 7f46733ad640 1 -- start start 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f46733ad640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 0x7f466c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f46733ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f466c1027a0 0x7f466c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f46733ad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f466c106f20 0x7f466c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f46733ad640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f466c111e30 con 0x7f466c101df0 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f46733ad640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f466c111cb0 con 0x7f466c1027a0 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f46733ad640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f466c111fb0 con 0x7f466c106f20 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 0x7f466c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 0x7f466c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42308/0 (socket says 172.21.15.143:42308) 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 -- 172.21.15.143:0/2747575343 learned_addr learned my addr 172.21.15.143:0/2747575343 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f467194a640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f466c106f20 0x7f466c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4670948640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f466c1027a0 0x7f466c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 -- 172.21.15.143:0/2747575343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f466c106f20 msgr2=0x7f466c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f466c106f20 0x7f466c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 -- 172.21.15.143:0/2747575343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f466c1027a0 msgr2=0x7f466c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f466c1027a0 0x7f466c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.915+0000 7f4671149640 1 -- 172.21.15.143:0/2747575343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c19d330 con 0x7f466c101df0 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.916+0000 7f4671149640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 0x7f466c198380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f466800e9f0 tx=0x7f466800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.916+0000 7f46627fc640 1 -- 172.21.15.143:0/2747575343 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466800cd80 con 0x7f466c101df0 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.916+0000 7f46627fc640 1 -- 172.21.15.143:0/2747575343 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46680054d0 con 0x7f466c101df0 2024-09-10T10:48:17.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.916+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c19d620 con 0x7f466c101df0 2024-09-10T10:48:17.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.916+0000 7f46627fc640 1 -- 172.21.15.143:0/2747575343 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668010490 con 0x7f466c101df0 2024-09-10T10:48:17.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.916+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c1a4f00 con 0x7f466c101df0 2024-09-10T10:48:17.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.919+0000 7f46627fc640 1 -- 172.21.15.143:0/2747575343 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4668005020 con 0x7f466c101df0 2024-09-10T10:48:17.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.919+0000 7f4675786740 1 Processor -- start 2024-09-10T10:48:17.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.919+0000 7f4675786740 1 -- start start 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.919+0000 7f4675786740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4675786740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4675786740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60ea780 0x55b4a60ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4675786740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b4a60412f0 con 0x55b4a60ea780 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4675786740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b4a5fe8cf0 con 0x55b4a60db5d0 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4675786740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b4a603d2b0 con 0x55b4a60dcb40 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4671149640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60db090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47614/0 (socket says 172.21.15.143:47614) 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 -- 172.21.15.143:0/3133233098 learned_addr learned my addr 172.21.15.143:0/3133233098 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f4670948640 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60ea780 0x55b4a60ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 -- 172.21.15.143:0/3133233098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 msgr2=0x55b4a60dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 -- 172.21.15.143:0/3133233098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60ea780 msgr2=0x55b4a60ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60ea780 0x55b4a60ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f467194a640 1 -- 172.21.15.143:0/3133233098 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a5fcd3a0 con 0x55b4a60dcb40 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f46627fc640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f46580a2c00 0x7f46580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.920+0000 7f46627fc640 1 -- 172.21.15.143:0/2747575343 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f46680cf680 con 0x7f466c101df0 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.921+0000 7f467194a640 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60db090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4664009f50 tx=0x7f46640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32458c351eda5d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:17.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.921+0000 7f4671149640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f46580a2c00 0x7f46580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.921+0000 7f46577fe640 1 -- 172.21.15.143:0/3133233098 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664004b20 con 0x55b4a60dcb40 2024-09-10T10:48:17.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.921+0000 7f46577fe640 1 -- 172.21.15.143:0/3133233098 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4664003e20 con 0x55b4a60dcb40 2024-09-10T10:48:17.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.921+0000 7f46577fe640 1 -- 172.21.15.143:0/3133233098 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664004840 con 0x55b4a60dcb40 2024-09-10T10:48:17.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 -- 172.21.15.143:0/3133233098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 msgr2=0x55b4a60db090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4671149640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f46580a2c00 0x7f46580a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46680cd190 tx=0x7f46680d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:17.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60db090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4664009f50 tx=0x7f46640118d0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 -- 172.21.15.143:0/3133233098 shutdown_connections 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60ea780 0x55b4a60ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 --2- 172.21.15.143:0/3133233098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60db090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 -- 172.21.15.143:0/3133233098 >> 172.21.15.143:0/3133233098 conn(0x55b4a60a4310 msgr2=0x55b4a60d7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 -- 172.21.15.143:0/3133233098 shutdown_connections 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 -- 172.21.15.143:0/3133233098 wait complete. 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 -- start start 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.922+0000 7f4675786740 1 Processor -- start 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60ea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 0x55b4a60e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a60e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b4a60ed480 con 0x55b4a60dcb40 2024-09-10T10:48:17.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b4a5ffc060 con 0x55b4a60db5d0 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b4a60ed600 con 0x55b4a60ea780 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4675786740 10 client.0 _ll_register_callbacks cb 0x55b4a5d4f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 0x55b4a60e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 0x55b4a60e7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42314/0 (socket says 172.21.15.143:42314) 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 -- 172.21.15.143:0/4183207123 learned_addr learned my addr 172.21.15.143:0/4183207123 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f467194a640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 -- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 msgr2=0x55b4a60e7a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a60e7a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 -- 172.21.15.143:0/4183207123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 msgr2=0x55b4a60ea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60ea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f4670948640 1 -- 172.21.15.143:0/4183207123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a5db23c0 con 0x55b4a60dcb40 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.923+0000 7f467194a640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.924+0000 7f4670948640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 0x55b4a60e7420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f465c009900 tx=0x7f465c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ded09a7f2772981 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.924+0000 7f4654ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.924+0000 7f4654ff9640 1 -- 172.21.15.143:0/4183207123 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c035040 con 0x55b4a60dcb40 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.925+0000 7f4654ff9640 1 -- 172.21.15.143:0/4183207123 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f465c0316a0 con 0x55b4a60dcb40 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.925+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4664003e20 con 0x55b4a60dcb40 2024-09-10T10:48:17.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.925+0000 7f4654ff9640 1 -- 172.21.15.143:0/4183207123 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f465c0318a0 con 0x55b4a60dcb40 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.925+0000 7f4675786740 10 client.16233 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.925+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b4a60f8f20 con 0x55b4a60dcb40 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.925+0000 7f4654ff9640 1 -- 172.21.15.143:0/4183207123 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4664003e20 con 0x55b4a60dcb40 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4654ff9640 1 -- 172.21.15.143:0/4183207123 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f465c004750 con 0x55b4a60dcb40 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4675786740 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4675786740 10 client.16233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4675786740 20 client.16233 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4675786740 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4675786740 4 client.16233 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a604e860 con 0x55b4a60fa480 2024-09-10T10:48:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.926+0000 7f467194a640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.927+0000 7f467194a640 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60fc8b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7be6e388f769617 server_cookie=79ba87dd237363a3 in_seq=0 out_seq=0 2024-09-10T10:48:17.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.927+0000 7f4654ff9640 10 client.16233 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:48:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.929+0000 7f4654ff9640 1 -- 172.21.15.143:0/4183207123 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4a604e860 con 0x55b4a60fa480 2024-09-10T10:48:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.929+0000 7f4654ff9640 10 client.16233 handle_command_reply: tid=0 2024-09-10T10:48:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.929+0000 7f4675786740 1 client.16233 shutdown 2024-09-10T10:48:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.929+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 msgr2=0x55b4a60e7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.929+0000 7f4675786740 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 0x55b4a60e7420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f465c009900 tx=0x7f465c032da0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.929+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 shutdown_connections 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60fc8b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a60e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4654ff9640 0 client.16233 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60dcb40 0x55b4a60e7420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 --2- 172.21.15.143:0/4183207123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60db5d0 0x55b4a60ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 >> 172.21.15.143:0/4183207123 conn(0x55b4a60a4310 msgr2=0x55b4a60dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 shutdown_connections 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.930+0000 7f4675786740 1 -- 172.21.15.143:0/4183207123 wait complete. 2024-09-10T10:48:17.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.931+0000 7f4675786740 20 client.16233 trim_cache size 0 max 16384 2024-09-10T10:48:17.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 Processor -- start 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 -- start start 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60ea780 0x55b4a60e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ac990 con 0x55b4a60db5d0 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ae500 con 0x55b4a60ea780 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4675786740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ad060 con 0x55b4a60dcb40 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f467194a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f467194a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60e79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42316/0 (socket says 172.21.15.143:42316) 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f467194a640 1 -- 172.21.15.143:0/970898435 learned_addr learned my addr 172.21.15.143:0/970898435 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.945+0000 7f4670948640 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f467194a640 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60ea780 0x55b4a60e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4670948640 1 -- 172.21.15.143:0/970898435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60ea780 msgr2=0x55b4a60e5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4670948640 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60ea780 0x55b4a60e5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4670948640 1 -- 172.21.15.143:0/970898435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 msgr2=0x55b4a60e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4670948640 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60e79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4670948640 1 -- 172.21.15.143:0/970898435 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a60d7510 con 0x55b4a60dcb40 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f467194a640 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4670948640 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60f6720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b4a60e89d0 tx=0x7f465c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f7329150567da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4656ffd640 1 -- 172.21.15.143:0/970898435 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c035070 con 0x55b4a60dcb40 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4656ffd640 1 -- 172.21.15.143:0/970898435 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f465c005850 con 0x55b4a60dcb40 2024-09-10T10:48:17.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.946+0000 7f4656ffd640 1 -- 172.21.15.143:0/970898435 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0051f0 con 0x55b4a60dcb40 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 -- 172.21.15.143:0/970898435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 msgr2=0x55b4a60f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60f6720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b4a60e89d0 tx=0x7f465c005fd0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 -- 172.21.15.143:0/970898435 shutdown_connections 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60ea780 0x55b4a60e5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60dcb40 0x55b4a60f6720 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 --2- 172.21.15.143:0/970898435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 -- 172.21.15.143:0/970898435 >> 172.21.15.143:0/970898435 conn(0x55b4a60a4310 msgr2=0x55b4a60d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 -- 172.21.15.143:0/970898435 shutdown_connections 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 -- 172.21.15.143:0/970898435 wait complete. 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.947+0000 7f4675786740 1 -- start start 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 Processor -- start 2024-09-10T10:48:17.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60dcb40 0x55b4a60ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a615d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ade00 con 0x55b4a60db5d0 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ad730 con 0x55b4a60dcb40 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1aa970 con 0x55b4a60ea780 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4675786740 10 client.0 _ll_register_callbacks cb 0x7f4668005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60edfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42326/0 (socket says 172.21.15.143:42326) 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 -- 172.21.15.143:0/2091561759 learned_addr learned my addr 172.21.15.143:0/2091561759 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a615d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f4670948640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60dcb40 0x55b4a60ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 -- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 msgr2=0x55b4a615d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a615d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 -- 172.21.15.143:0/2091561759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60dcb40 msgr2=0x55b4a60ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.948+0000 7f467194a640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60dcb40 0x55b4a60ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f467194a640 1 -- 172.21.15.143:0/2091561759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a615d7f0 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f467194a640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60edfd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4664009f50 tx=0x7f4664000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c3180bb5b5298c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4654ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4654ff9640 1 -- 172.21.15.143:0/2091561759 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400f7e0 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4654ff9640 1 -- 172.21.15.143:0/2091561759 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f466400f980 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c005850 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4654ff9640 1 -- 172.21.15.143:0/2091561759 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f466400fbc0 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4675786740 10 client.16239 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b4a60dec70 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.949+0000 7f4654ff9640 1 -- 172.21.15.143:0/2091561759 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f466400fd60 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4654ff9640 1 -- 172.21.15.143:0/2091561759 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f466403b070 con 0x55b4a60db5d0 2024-09-10T10:48:17.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4675786740 10 client.16239 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4675786740 10 client.16239 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4675786740 20 client.16239 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4675786740 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4675786740 4 client.16239 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f465c005850 con 0x55b4a60fa480 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.950+0000 7f4670948640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.951+0000 7f4654ff9640 10 client.16239 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:48:17.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.951+0000 7f4670948640 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60e12f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=31197053b5d5e84c server_cookie=3f823fd60251fbf3 in_seq=0 out_seq=0 2024-09-10T10:48:17.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.954+0000 7f4654ff9640 1 -- 172.21.15.143:0/2091561759 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f465c005850 con 0x55b4a60fa480 2024-09-10T10:48:17.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.954+0000 7f4654ff9640 10 client.16239 handle_command_reply: tid=0 2024-09-10T10:48:17.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.954+0000 7f4675786740 1 client.16239 shutdown 2024-09-10T10:48:17.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 msgr2=0x55b4a60edfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60edfd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4664009f50 tx=0x7f4664000f40 comp rx=0 tx=0).stop 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 shutdown_connections 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b4a60fa480 0x55b4a60e12f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b4a60ea780 0x55b4a615d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a60dcb40 0x55b4a60ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 --2- 172.21.15.143:0/2091561759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b4a60db5d0 0x55b4a60edfd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 >> 172.21.15.143:0/2091561759 conn(0x55b4a60a4310 msgr2=0x55b4a60e8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4654ff9640 0 client.16239 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:48:17.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.955+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 shutdown_connections 2024-09-10T10:48:17.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.956+0000 7f4675786740 1 -- 172.21.15.143:0/2091561759 wait complete. 2024-09-10T10:48:17.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.956+0000 7f4675786740 20 client.16239 trim_cache size 0 max 16384 2024-09-10T10:48:17.958 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.956+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f46580a2c00 msgr2=0x7f46580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.956+0000 7f46733ad640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f46580a2c00 0x7f46580a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f46680cd190 tx=0x7f46680d5550 comp rx=0 tx=0).stop 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.956+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 msgr2=0x7f466c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.956+0000 7f46733ad640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 0x7f466c198380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f466800e9f0 tx=0x7f466800edf0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 shutdown_connections 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f46580a2c00 0x7f46580a50c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f466c106f20 0x7f466c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f466c1027a0 0x7f466c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 --2- 172.21.15.143:0/2747575343 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f466c101df0 0x7f466c198380 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:17.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 >> 172.21.15.143:0/2747575343 conn(0x7f466c0fd8f0 msgr2=0x7f466c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:17.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 shutdown_connections 2024-09-10T10:48:17.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:17.957+0000 7f46733ad640 1 -- 172.21.15.143:0/2747575343 wait complete. 2024-09-10T10:48:18.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:18 smithi143 ceph-mon[31057]: pgmap v448: 129 pgs: 129 active+clean; 232 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 336 op/s 2024-09-10T10:48:18.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:48:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:18 smithi178 ceph-mon[34813]: pgmap v448: 129 pgs: 129 active+clean; 232 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 336 op/s 2024-09-10T10:48:18.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:18 smithi144 ceph-mon[34230]: pgmap v448: 129 pgs: 129 active+clean; 232 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 336 op/s 2024-09-10T10:48:19.197 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3341452585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 msgr2=0x7f8f34101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3341452585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34101200 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009900 tx=0x7f8f2802a480 comp rx=0 tx=0).stop 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3341452585 shutdown_connections 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3341452585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 0x7f8f3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3341452585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 0x7f8f341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3341452585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34101200 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3341452585 >> 172.21.15.143:0/3341452585 conn(0x7f8f340fc7d0 msgr2=0x7f8f340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3341452585 shutdown_connections 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.203+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3341452585 wait complete. 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.204+0000 7f8f3aff9640 1 Processor -- start 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.204+0000 7f8f3aff9640 1 -- start start 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.204+0000 7f8f3aff9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f3aff9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 0x7f8f34198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f3aff9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 0x7f8f3419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f3aff9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34111ef0 con 0x7f8f34100e20 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f3aff9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34111d70 con 0x7f8f34102570 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f3aff9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34112070 con 0x7f8f34102ef0 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42358/0 (socket says 172.21.15.143:42358) 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f38d95640 1 -- 172.21.15.143:0/3767270053 learned_addr learned my addr 172.21.15.143:0/3767270053 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:20.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 0x7f8f34198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f39596640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 0x7f8f3419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 -- 172.21.15.143:0/3767270053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 msgr2=0x7f8f3419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 0x7f8f3419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 -- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 msgr2=0x7f8f34198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 -- 172.21.15.143:0/3767270053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3419d200 con 0x7f8f34102570 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f39596640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 0x7f8f3419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.205+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 0x7f8f34198790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e9f0 tx=0x7f8f2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.206+0000 7f8f31ffb640 1 -- 172.21.15.143:0/3767270053 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000cd80 con 0x7f8f34102570 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.206+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3419d4f0 con 0x7f8f34102570 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.206+0000 7f8f31ffb640 1 -- 172.21.15.143:0/3767270053 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f200035f0 con 0x7f8f34102570 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.206+0000 7f8f31ffb640 1 -- 172.21.15.143:0/3767270053 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20010670 con 0x7f8f34102570 2024-09-10T10:48:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.206+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3410e280 con 0x7f8f34102570 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 Processor -- start 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 -- start start 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f31ffb640 1 -- 172.21.15.143:0/3767270053 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8f200037a0 con 0x7f8f34102570 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d340bb32f0 con 0x55d340c4d5d0 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d340b5acf0 con 0x55d340c4eb40 2024-09-10T10:48:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d340baf2b0 con 0x55d340c58370 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c5a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47666/0 (socket says 172.21.15.143:47666) 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f33fff640 1 -- 172.21.15.143:0/735538100 learned_addr learned my addr 172.21.15.143:0/735538100 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.210+0000 7f8f39596640 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f38d95640 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f33fff640 1 -- 172.21.15.143:0/735538100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 msgr2=0x55d340c4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f33fff640 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f33fff640 1 -- 172.21.15.143:0/735538100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 msgr2=0x55d340c4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f33fff640 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c4da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f33fff640 1 -- 172.21.15.143:0/735538100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d340b3f3a0 con 0x55d340c58370 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f39596640 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f33fff640 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c5a760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f200d5a90 tx=0x7f8f200d5f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93334e2fcdd98c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:20.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f1effd640 1 -- 172.21.15.143:0/735538100 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200de070 con 0x55d340c58370 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f1effd640 1 -- 172.21.15.143:0/735538100 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f200d10d0 con 0x55d340c58370 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.211+0000 7f8f1effd640 1 -- 172.21.15.143:0/735538100 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200da680 con 0x55d340c58370 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f31ffb640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f240a2b80 0x7f8f240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f39596640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f240a2b80 0x7f8f240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f31ffb640 1 -- 172.21.15.143:0/3767270053 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f200cf700 con 0x7f8f34102570 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/735538100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 msgr2=0x55d340c5a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c5a760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f200d5a90 tx=0x7f8f200d5f50 comp rx=0 tx=0).stop 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/735538100 shutdown_connections 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c5a760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/735538100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/735538100 >> 172.21.15.143:0/735538100 conn(0x55d340c16310 msgr2=0x55d340c49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.212+0000 7f8f39596640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f240a2b80 0x7f8f240a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c005fd0 tx=0x7f8f2c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/735538100 shutdown_connections 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/735538100 wait complete. 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 -- start start 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 Processor -- start 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c57d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 0x55d340c54fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c58370 0x55d340c55570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d340c5b070 con 0x55d340c4d5d0 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d340b6e060 con 0x55d340c58370 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.213+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d340c5b1f0 con 0x55d340c4eb40 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 0x55d340c54fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f3d3d2740 10 client.0 _ll_register_callbacks cb 0x55d3408c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c57d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 0x55d340c54fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47676/0 (socket says 172.21.15.143:47676) 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 -- 172.21.15.143:0/1758843237 learned_addr learned my addr 172.21.15.143:0/1758843237 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f39596640 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c58370 0x55d340c55570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 -- 172.21.15.143:0/1758843237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c58370 msgr2=0x55d340c55570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c58370 0x55d340c55570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 -- 172.21.15.143:0/1758843237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 msgr2=0x55d340c57d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c57d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 -- 172.21.15.143:0/1758843237 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3409243c0 con 0x55d340c4eb40 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f33fff640 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 0x55d340c54fd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f20098000 tx=0x7f8f200d7f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881d0e5436bd7771 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f13fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:48:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f13fff640 1 -- 172.21.15.143:0/1758843237 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200de030 con 0x55d340c4eb40 2024-09-10T10:48:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f200d10d0 con 0x55d340c4eb40 2024-09-10T10:48:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.214+0000 7f8f13fff640 1 -- 172.21.15.143:0/1758843237 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f200cf3f0 con 0x55d340c4eb40 2024-09-10T10:48:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.215+0000 7f8f13fff640 1 -- 172.21.15.143:0/1758843237 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f20097e40 con 0x55d340c4eb40 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.215+0000 7f8f3d3d2740 10 client.25666 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.215+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d340c5b070 con 0x55d340c4eb40 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.215+0000 7f8f13fff640 1 -- 172.21.15.143:0/1758843237 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f200d10d0 con 0x55d340c4eb40 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.215+0000 7f8f13fff640 1 -- 172.21.15.143:0/1758843237 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f8f200ccec0 con 0x55d340c4eb40 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f3d3d2740 10 client.25666 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f3d3d2740 10 client.25666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f3d3d2740 20 client.25666 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340c78c80 0x55d340c7b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f3d3d2740 4 client.25666 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d340bc0860 con 0x55d340c78c80 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f38d95640 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340c78c80 0x55d340c7b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f38d95640 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340c78c80 0x55d340c7b070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c8e9010b4eeb6dc server_cookie=153feb4259dd602c in_seq=0 out_seq=0 2024-09-10T10:48:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.216+0000 7f8f13fff640 10 client.25666 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.217+0000 7f8f13fff640 1 -- 172.21.15.143:0/1758843237 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d340bc0860 con 0x55d340c78c80 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.217+0000 7f8f13fff640 10 client.25666 handle_command_reply: tid=0 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.217+0000 7f8f3d3d2740 1 client.25666 shutdown 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 msgr2=0x55d340c54fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 0x55d340c54fd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f20098000 tx=0x7f8f200d7f40 comp rx=0 tx=0).stop 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 shutdown_connections 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340c78c80 0x55d340c7b070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c58370 0x55d340c55570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c4eb40 0x55d340c54fd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/1758843237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c57d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 >> 172.21.15.143:0/1758843237 conn(0x55d340c16310 msgr2=0x55d340c4f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f13fff640 0 client.25666 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 shutdown_connections 2024-09-10T10:48:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/1758843237 wait complete. 2024-09-10T10:48:20.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.218+0000 7f8f3d3d2740 20 client.25666 trim_cache size 0 max 16384 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 Processor -- start 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 -- start start 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c49df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c80730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c80c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f340737d0 con 0x55d340c4d5d0 2024-09-10T10:48:20.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341a5cf0 con 0x55d340c4eb40 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34073ea0 con 0x55d340c58370 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c49df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f39596640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c80c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c49df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42380/0 (socket says 172.21.15.143:42380) 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f39596640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c80c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47684/0 (socket says 172.21.15.143:47684) 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 -- 172.21.15.143:0/537532891 learned_addr learned my addr 172.21.15.143:0/537532891 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f33fff640 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c80730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 -- 172.21.15.143:0/537532891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 msgr2=0x55d340c80c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c80c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 -- 172.21.15.143:0/537532891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 msgr2=0x55d340c80730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c80730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.237+0000 7f8f38d95640 1 -- 172.21.15.143:0/537532891 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d340c53520 con 0x55d340c4d5d0 2024-09-10T10:48:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.238+0000 7f8f38d95640 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c49df0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005ab0 tx=0x7f8f2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9988bccd5a2624ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.238+0000 7f8f1e7fc640 1 -- 172.21.15.143:0/537532891 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28035040 con 0x55d340c4d5d0 2024-09-10T10:48:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.238+0000 7f8f1e7fc640 1 -- 172.21.15.143:0/537532891 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f28005030 con 0x55d340c4d5d0 2024-09-10T10:48:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.238+0000 7f8f1e7fc640 1 -- 172.21.15.143:0/537532891 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280049e0 con 0x55d340c4d5d0 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/537532891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 msgr2=0x55d340c49df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c49df0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005ab0 tx=0x7f8f2800b040 comp rx=0 tx=0).stop 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/537532891 shutdown_connections 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340c80c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c80730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/537532891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c49df0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/537532891 >> 172.21.15.143:0/537532891 conn(0x55d340c16310 msgr2=0x55d340c496e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/537532891 shutdown_connections 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/537532891 wait complete. 2024-09-10T10:48:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 -- start start 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.239+0000 7f8f3d3d2740 1 Processor -- start 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c5b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c5ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340ce16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341a55f0 con 0x55d340c4d5d0 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341a4f20 con 0x55d340c4eb40 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f340717b0 con 0x55d340c58370 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c5b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c5b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42382/0 (socket says 172.21.15.143:42382) 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 -- 172.21.15.143:0/3733248828 learned_addr learned my addr 172.21.15.143:0/3733248828 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f3d3d2740 10 client.0 _ll_register_callbacks cb 0x7f8f200d58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c5ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f39596640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340ce16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 -- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 msgr2=0x55d340ce16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340ce16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 -- 172.21.15.143:0/3733248828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 msgr2=0x55d340c5ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c5ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.240+0000 7f8f38d95640 1 -- 172.21.15.143:0/3733248828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d340c5c370 con 0x55d340c4d5d0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.241+0000 7f8f38d95640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c5b530 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8f28016450 tx=0x7f8f28005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd79eabd285fa2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.241+0000 7f8f13fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.241+0000 7f8f13fff640 1 -- 172.21.15.143:0/3733248828 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28035070 con 0x55d340c4d5d0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.241+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f28005030 con 0x55d340c4d5d0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.241+0000 7f8f13fff640 1 -- 172.21.15.143:0/3733248828 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8f2803dcb0 con 0x55d340c4d5d0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f13fff640 1 -- 172.21.15.143:0/3733248828 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f2805d050 con 0x55d340c4d5d0 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f3d3d2740 10 client.16257 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d340ce2d60 con 0x55d340c4d5d0 2024-09-10T10:48:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f13fff640 1 -- 172.21.15.143:0/3733248828 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f28005030 con 0x55d340c4d5d0 2024-09-10T10:48:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f13fff640 1 -- 172.21.15.143:0/3733248828 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f8f2804a030 con 0x55d340c4d5d0 2024-09-10T10:48:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f3d3d2740 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f3d3d2740 10 client.16257 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f3d3d2740 20 client.16257 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.242+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340ce3410 0x55d340ce5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f3d3d2740 4 client.16257 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d340bbad70 con 0x55d340ce3410 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340ce3410 0x55d340ce5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f13fff640 10 client.16257 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f33fff640 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340ce3410 0x55d340ce5830 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6801e28d077b56b0 server_cookie=5232de4eca780ac0 in_seq=0 out_seq=0 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f13fff640 1 -- 172.21.15.143:0/3733248828 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d340bbad70 con 0x55d340ce3410 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f13fff640 10 client.16257 handle_command_reply: tid=0 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.243+0000 7f8f3d3d2740 1 client.16257 shutdown 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 msgr2=0x55d340c5b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c5b530 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8f28016450 tx=0x7f8f28005410 comp rx=0 tx=0).stop 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 shutdown_connections 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55d340ce3410 0x55d340ce5830 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d340c58370 0x55d340ce16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d340c4eb40 0x55d340c5ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f13fff640 0 client.16257 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 --2- 172.21.15.143:0/3733248828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d340c4d5d0 0x55d340c5b530 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 >> 172.21.15.143:0/3733248828 conn(0x55d340c16310 msgr2=0x55d340c538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 shutdown_connections 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 1 -- 172.21.15.143:0/3733248828 wait complete. 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.244+0000 7f8f3d3d2740 20 client.16257 trim_cache size 0 max 16384 2024-09-10T10:48:20.249 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:48:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f240a2b80 msgr2=0x7f8f240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f240a2b80 0x7f8f240a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c005fd0 tx=0x7f8f2c009500 comp rx=0 tx=0).stop 2024-09-10T10:48:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 msgr2=0x7f8f34198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 0x7f8f34198790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e9f0 tx=0x7f8f2000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 shutdown_connections 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8f240a2b80 0x7f8f240a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f34102ef0 0x7f8f3419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34102570 0x7f8f34198790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 --2- 172.21.15.143:0/3767270053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f34100e20 0x7f8f34198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 >> 172.21.15.143:0/3767270053 conn(0x7f8f340fc7d0 msgr2=0x7f8f34105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.245+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 shutdown_connections 2024-09-10T10:48:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:20.246+0000 7f8f3aff9640 1 -- 172.21.15.143:0/3767270053 wait complete. 2024-09-10T10:48:20.849 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:20 smithi143 ceph-mon[31057]: pgmap v449: 129 pgs: 129 active+clean; 240 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-09-10T10:48:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:20 smithi178 ceph-mon[34813]: pgmap v449: 129 pgs: 129 active+clean; 240 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-09-10T10:48:20.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:20 smithi144 ceph-mon[34230]: pgmap v449: 129 pgs: 129 active+clean; 240 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-09-10T10:48:21.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:48:21.629 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 -- 172.21.15.143:0/1868965375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280106f20 msgr2=0x7f428010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 --2- 172.21.15.143:0/1868965375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280106f20 0x7f428010eaa0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4278009ca0 tx=0x7f427802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 -- 172.21.15.143:0/1868965375 shutdown_connections 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 --2- 172.21.15.143:0/1868965375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280106f20 0x7f428010eaa0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 --2- 172.21.15.143:0/1868965375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42801027a0 0x7f42801067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 --2- 172.21.15.143:0/1868965375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4280101df0 0x7f42801021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 -- 172.21.15.143:0/1868965375 >> 172.21.15.143:0/1868965375 conn(0x7f42800fd8f0 msgr2=0x7f42800ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 -- 172.21.15.143:0/1868965375 shutdown_connections 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.161+0000 7f4285eab640 1 -- 172.21.15.143:0/1868965375 wait complete. 2024-09-10T10:48:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.162+0000 7f4285eab640 1 Processor -- start 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.162+0000 7f4285eab640 1 -- start start 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.162+0000 7f4285eab640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 0x7f4280198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f4285eab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42801027a0 0x7f4280198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f4285eab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280106f20 0x7f428019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 0x7f4280198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 0x7f4280198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42392/0 (socket says 172.21.15.143:42392) 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427f7fe640 1 -- 172.21.15.143:0/3800534073 learned_addr learned my addr 172.21.15.143:0/3800534073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42801027a0 0x7f4280198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4280111e30 con 0x7f4280101df0 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4280111cb0 con 0x7f42801027a0 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4280111fb0 con 0x7f4280106f20 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427ffff640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280106f20 0x7f428019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427effd640 1 -- 172.21.15.143:0/3800534073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280106f20 msgr2=0x7f428019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427effd640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280106f20 0x7f428019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427effd640 1 -- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 msgr2=0x7f4280198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427effd640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 0x7f4280198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427effd640 1 -- 172.21.15.143:0/3800534073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428019d2f0 con 0x7f42801027a0 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.163+0000 7f427f7fe640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 0x7f4280198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.164+0000 7f427effd640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42801027a0 0x7f4280198880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f427400e9f0 tx=0x7f427400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.164+0000 7f427cff9640 1 -- 172.21.15.143:0/3800534073 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427400ccc0 con 0x7f42801027a0 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.164+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428019d5e0 con 0x7f42801027a0 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.164+0000 7f427cff9640 1 -- 172.21.15.143:0/3800534073 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42740108e0 con 0x7f42801027a0 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.164+0000 7f427cff9640 1 -- 172.21.15.143:0/3800534073 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274018880 con 0x7f42801027a0 2024-09-10T10:48:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.164+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42801a4ec0 con 0x7f42801027a0 2024-09-10T10:48:22.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.166+0000 7f427cff9640 1 -- 172.21.15.143:0/3800534073 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4274010a90 con 0x7f42801027a0 2024-09-10T10:48:22.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.167+0000 7f427cff9640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f42680a2c80 0x7f42680a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.167+0000 7f427cff9640 1 -- 172.21.15.143:0/3800534073 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f42740cf6e0 con 0x7f42801027a0 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.167+0000 7f427f7fe640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f42680a2c80 0x7f42680a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.167+0000 7f4288284740 1 Processor -- start 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 -- start start 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d002e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d002ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d003b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5644cff942f0 con 0x5644d002fb40 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5644cff3bcf0 con 0x5644d0039370 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f4288284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644cff902b0 con 0x5644d002e5d0 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f427f7fe640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f42680a2c80 0x7f42680a5140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4270011550 tx=0x7f4270005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f427ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f427ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d003b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:45430/0 (socket says 172.21.15.143:45430) 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f427ffff640 1 -- 172.21.15.143:0/2518666853 learned_addr learned my addr 172.21.15.143:0/2518666853 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f427effd640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d002ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.168+0000 7f427f7fe640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d002e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427effd640 1 -- 172.21.15.143:0/2518666853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 msgr2=0x5644d003b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427effd640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d003b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427effd640 1 -- 172.21.15.143:0/2518666853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 msgr2=0x5644d002e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427effd640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d002e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427effd640 1 -- 172.21.15.143:0/2518666853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644cff203a0 con 0x5644d002e5d0 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427ffff640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d003b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427f7fe640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d002e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:22.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f427effd640 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d002ea10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f427409bb10 tx=0x7f427409ba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a61fef845ce40a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:22.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f4265ffb640 1 -- 172.21.15.143:0/2518666853 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427409ff30 con 0x5644d002e5d0 2024-09-10T10:48:22.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f4265ffb640 1 -- 172.21.15.143:0/2518666853 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42740a47f0 con 0x5644d002e5d0 2024-09-10T10:48:22.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.169+0000 7f4265ffb640 1 -- 172.21.15.143:0/2518666853 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427409b230 con 0x5644d002e5d0 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 -- 172.21.15.143:0/2518666853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 msgr2=0x5644d002ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d002ea10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f427409bb10 tx=0x7f427409ba80 comp rx=0 tx=0).stop 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 -- 172.21.15.143:0/2518666853 shutdown_connections 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d003b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d002ea10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 --2- 172.21.15.143:0/2518666853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d002e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 -- 172.21.15.143:0/2518666853 >> 172.21.15.143:0/2518666853 conn(0x5644cfff7310 msgr2=0x5644d002ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 -- 172.21.15.143:0/2518666853 shutdown_connections 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.170+0000 7f4288284740 1 -- 172.21.15.143:0/2518666853 wait complete. 2024-09-10T10:48:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.171+0000 7f4288284740 1 -- start start 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.171+0000 7f4288284740 1 Processor -- start 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.171+0000 7f4288284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d0038d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f4288284740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d0036000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f4288284740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d00365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d0038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f4288284740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5644d003c0b0 con 0x5644d002fb40 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d0038d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47728/0 (socket says 172.21.15.143:47728) 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f4288284740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5644cff4f060 con 0x5644d0039370 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644d003c230 con 0x5644d002e5d0 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 -- 172.21.15.143:0/3594411189 learned_addr learned my addr 172.21.15.143:0/3594411189 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f4288284740 10 client.0 _ll_register_callbacks cb 0x5644cfca28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427effd640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d0036000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427ffff640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d00365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 -- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 msgr2=0x5644d00365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d00365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 -- 172.21.15.143:0/3594411189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 msgr2=0x5644d0036000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d0036000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 -- 172.21.15.143:0/3594411189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644cfd053c0 con 0x5644d002e5d0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427ffff640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d00365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427effd640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d0036000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f427f7fe640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d0038d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427000fab0 tx=0x7f4270004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7def6ccdec15a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f425effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.172+0000 7f425effd640 1 -- 172.21.15.143:0/3594411189 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427002a060 con 0x5644d002e5d0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.173+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42740a47f0 con 0x5644d002e5d0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.173+0000 7f425effd640 1 -- 172.21.15.143:0/3594411189 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4270009050 con 0x5644d002e5d0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.173+0000 7f425effd640 1 -- 172.21.15.143:0/3594411189 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f427001cd20 con 0x5644d002e5d0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.173+0000 7f4288284740 10 client.25684 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.173+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644d00503a0 con 0x5644d002e5d0 2024-09-10T10:48:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.173+0000 7f425effd640 1 -- 172.21.15.143:0/3594411189 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42740a47f0 con 0x5644d002e5d0 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f425effd640 1 -- 172.21.15.143:0/3594411189 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f4270004ca0 con 0x5644d002e5d0 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f4288284740 10 client.25684 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f4288284740 10 client.25684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f4288284740 20 client.25684 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f4288284740 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d0053c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f4288284740 4 client.25684 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644cffa1860 con 0x5644d0051860 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.174+0000 7f427effd640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d0053c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.175+0000 7f425effd640 10 client.25684 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.175+0000 7f427effd640 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d0053c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adbe21b504381cba server_cookie=13a5ee4d96e8e0b5 in_seq=0 out_seq=0 2024-09-10T10:48:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f425effd640 1 -- 172.21.15.143:0/3594411189 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644cffa1860 con 0x5644d0051860 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f425effd640 10 client.25684 handle_command_reply: tid=0 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 client.25684 shutdown 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 msgr2=0x5644d0038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d0038d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427000fab0 tx=0x7f4270004730 comp rx=0 tx=0).stop 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 shutdown_connections 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d0053c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f425effd640 0 client.25684 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d0039370 0x5644d00365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d002fb40 0x5644d0036000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 --2- 172.21.15.143:0/3594411189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002e5d0 0x5644d0038d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 >> 172.21.15.143:0/3594411189 conn(0x5644cfff7310 msgr2=0x5644d0030960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 shutdown_connections 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 1 -- 172.21.15.143:0/3594411189 wait complete. 2024-09-10T10:48:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.176+0000 7f4288284740 20 client.25684 trim_cache size 0 max 16384 2024-09-10T10:48:22.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 Processor -- start 2024-09-10T10:48:22.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 -- start start 2024-09-10T10:48:22.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d004dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0059520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d0059a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ac9b0 con 0x5644d0039370 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ae520 con 0x5644d002e5d0 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f4288284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ad080 con 0x5644d002fb40 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f427ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d0059a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f427ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d0059a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42416/0 (socket says 172.21.15.143:42416) 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f427ffff640 1 -- 172.21.15.143:0/497566905 learned_addr learned my addr 172.21.15.143:0/497566905 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.197+0000 7f427effd640 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0059520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427f7fe640 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d004dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427ffff640 1 -- 172.21.15.143:0/497566905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 msgr2=0x5644d0059520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427ffff640 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0059520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427ffff640 1 -- 172.21.15.143:0/497566905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 msgr2=0x5644d004dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427ffff640 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d004dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427ffff640 1 -- 172.21.15.143:0/497566905 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d0036940 con 0x5644d0039370 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f427ffff640 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d0059a60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4278035000 tx=0x7f4278031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89b26a2c1a26543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f42657fa640 1 -- 172.21.15.143:0/497566905 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427804a070 con 0x5644d0039370 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f42657fa640 1 -- 172.21.15.143:0/497566905 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4278036040 con 0x5644d0039370 2024-09-10T10:48:22.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.198+0000 7f42657fa640 1 -- 172.21.15.143:0/497566905 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278004040 con 0x5644d0039370 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 -- 172.21.15.143:0/497566905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 msgr2=0x5644d0059a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d0059a60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4278035000 tx=0x7f4278031a40 comp rx=0 tx=0).stop 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 -- 172.21.15.143:0/497566905 shutdown_connections 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d0059a60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0059520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 --2- 172.21.15.143:0/497566905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d004dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.199+0000 7f4288284740 1 -- 172.21.15.143:0/497566905 >> 172.21.15.143:0/497566905 conn(0x5644cfff7310 msgr2=0x5644d00348d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.200+0000 7f4288284740 1 -- 172.21.15.143:0/497566905 shutdown_connections 2024-09-10T10:48:22.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.200+0000 7f4288284740 1 -- 172.21.15.143:0/497566905 wait complete. 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.200+0000 7f4288284740 1 -- start start 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.200+0000 7f4288284740 1 Processor -- start 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d0045d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0046260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d00467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ade20 con 0x5644d0039370 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42801ad750 con 0x5644d002e5d0 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42801aa990 con 0x5644d002fb40 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f4288284740 10 client.0 _ll_register_callbacks cb 0x7f4270009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d0045d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d0045d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:45438/0 (socket says 172.21.15.143:45438) 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 -- 172.21.15.143:0/4268415763 learned_addr learned my addr 172.21.15.143:0/4268415763 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427effd640 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0046260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 -- 172.21.15.143:0/4268415763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 msgr2=0x5644d0046260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0046260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 -- 172.21.15.143:0/4268415763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 msgr2=0x5644d00467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d00467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 -- 172.21.15.143:0/4268415763 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d00bb740 con 0x5644d002e5d0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f427f7fe640 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d0045d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427000fab0 tx=0x7f427002f650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704d183be9b170c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.201+0000 7f425effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f425effd640 1 -- 172.21.15.143:0/4268415763 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427003c060 con 0x5644d002e5d0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f425effd640 1 -- 172.21.15.143:0/4268415763 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4270022040 con 0x5644d002e5d0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4278036040 con 0x5644d002e5d0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f425effd640 1 -- 172.21.15.143:0/4268415763 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f427002a020 con 0x5644d002e5d0 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 10 client.25649 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:22.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644d00bca00 con 0x5644d002e5d0 2024-09-10T10:48:22.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f425effd640 1 -- 172.21.15.143:0/4268415763 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4270025d00 con 0x5644d002e5d0 2024-09-10T10:48:22.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f425effd640 1 -- 172.21.15.143:0/4268415763 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f4270036070 con 0x5644d002e5d0 2024-09-10T10:48:22.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 10 client.25649 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 10 client.25649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 20 client.25649 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d00bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 4 client.25649 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.202+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4278036040 con 0x5644d0051860 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.203+0000 7f427effd640 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d00bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.203+0000 7f427effd640 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d00bf220 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e187a7fb1c2fb907 server_cookie=cc5e0c9533a34430 in_seq=0 out_seq=0 2024-09-10T10:48:22.207 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:48:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.203+0000 7f425effd640 10 client.25649 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:48:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.203+0000 7f425effd640 1 -- 172.21.15.143:0/4268415763 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4278036040 con 0x5644d0051860 2024-09-10T10:48:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.203+0000 7f425effd640 10 client.25649 handle_command_reply: tid=0 2024-09-10T10:48:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.203+0000 7f4288284740 1 client.25649 shutdown 2024-09-10T10:48:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 msgr2=0x5644d0045d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d0045d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427000fab0 tx=0x7f427002f650 comp rx=0 tx=0).stop 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 shutdown_connections 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5644d0039370 0x5644d00467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644d002fb40 0x5644d0046260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x5644d0051860 0x5644d00bf220 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 --2- 172.21.15.143:0/4268415763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644d002e5d0 0x5644d0045d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 >> 172.21.15.143:0/4268415763 conn(0x5644cfff7310 msgr2=0x5644d00294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f425effd640 0 client.25649 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 shutdown_connections 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 1 -- 172.21.15.143:0/4268415763 wait complete. 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.204+0000 7f4288284740 20 client.25649 trim_cache size 0 max 16384 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f42680a2c80 msgr2=0x7f42680a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f42680a2c80 0x7f42680a5140 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4270011550 tx=0x7f4270005f90 comp rx=0 tx=0).stop 2024-09-10T10:48:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42801027a0 msgr2=0x7f4280198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42801027a0 0x7f4280198880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f427400e9f0 tx=0x7f427400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 shutdown_connections 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f42680a2c80 0x7f42680a5140 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4280106f20 0x7f428019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42801027a0 0x7f4280198880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 --2- 172.21.15.143:0/3800534073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4280101df0 0x7f4280198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 >> 172.21.15.143:0/3800534073 conn(0x7f42800fd8f0 msgr2=0x7f42800fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 shutdown_connections 2024-09-10T10:48:22.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:22.205+0000 7f4285eab640 1 -- 172.21.15.143:0/3800534073 wait complete. 2024-09-10T10:48:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:22 smithi178 ceph-mon[34813]: pgmap v450: 129 pgs: 129 active+clean; 251 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 325 op/s 2024-09-10T10:48:22.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:22 smithi144 ceph-mon[34230]: pgmap v450: 129 pgs: 129 active+clean; 251 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 325 op/s 2024-09-10T10:48:23.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:22 smithi143 ceph-mon[31057]: pgmap v450: 129 pgs: 129 active+clean; 251 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.6 MiB/s wr, 325 op/s 2024-09-10T10:48:23.591 INFO:tasks.workunit.client.0.smithi143.stdout: 63 1690 654 612 558 532 977 2024-09-10T10:48:23.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:48:24.137 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 -- 172.21.15.143:0/4033418968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 msgr2=0x7fc96c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 --2- 172.21.15.143:0/4033418968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 0x7fc96c10a940 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc954009900 tx=0x7fc95402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 -- 172.21.15.143:0/4033418968 shutdown_connections 2024-09-10T10:48:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 --2- 172.21.15.143:0/4033418968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c1139e0 0x7fc96c115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 --2- 172.21.15.143:0/4033418968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c10ae80 0x7fc96c113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 --2- 172.21.15.143:0/4033418968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 0x7fc96c10a940 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 -- 172.21.15.143:0/4033418968 >> 172.21.15.143:0/4033418968 conn(0x7fc96c06e2d0 msgr2=0x7fc96c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 -- 172.21.15.143:0/4033418968 shutdown_connections 2024-09-10T10:48:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.726+0000 7fc971089640 1 -- 172.21.15.143:0/4033418968 wait complete. 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.727+0000 7fc971089640 1 Processor -- start 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.727+0000 7fc971089640 1 -- start start 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.727+0000 7fc971089640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 0x7fc96c1a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc971089640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 0x7fc96c1a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc971089640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 0x7fc96c1a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc971089640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c119580 con 0x7fc96c10a560 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc971089640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c119400 con 0x7fc96c1139e0 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc971089640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c119700 con 0x7fc96c10ae80 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 0x7fc96c1a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 0x7fc96c1a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 0x7fc96c1a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47776/0 (socket says 172.21.15.143:47776) 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 0x7fc96c1a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:45456/0 (socket says 172.21.15.143:45456) 2024-09-10T10:48:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 -- 172.21.15.143:0/1805555137 learned_addr learned my addr 172.21.15.143:0/1805555137 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 0x7fc96c1a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 -- 172.21.15.143:0/1805555137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 msgr2=0x7fc96c1a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 0x7fc96c1a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 -- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 msgr2=0x7fc96c1a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 0x7fc96c1a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96a59c640 1 -- 172.21.15.143:0/1805555137 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c1a7d60 con 0x7fc96c10ae80 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.728+0000 7fc96b59e640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 0x7fc96c1a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.729+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 0x7fc96c1a32f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00ea80 tx=0x7fc95c00ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.729+0000 7fc963fff640 1 -- 172.21.15.143:0/1805555137 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00ce40 con 0x7fc96c10ae80 2024-09-10T10:48:24.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.729+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c1a8050 con 0x7fc96c10ae80 2024-09-10T10:48:24.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.729+0000 7fc963fff640 1 -- 172.21.15.143:0/1805555137 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc95c011920 con 0x7fc96c10ae80 2024-09-10T10:48:24.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.729+0000 7fc963fff640 1 -- 172.21.15.143:0/1805555137 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01aaa0 con 0x7fc96c10ae80 2024-09-10T10:48:24.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.729+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c1af930 con 0x7fc96c10ae80 2024-09-10T10:48:24.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.733+0000 7fc973462740 1 Processor -- start 2024-09-10T10:48:24.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.733+0000 7fc973462740 1 -- start start 2024-09-10T10:48:24.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc973462740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c82b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc973462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c82ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc973462740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c840f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc973462740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56150c7912f0 con 0x56150c82cb40 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc973462740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56150c738cf0 con 0x56150c83eb80 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc973462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56150c78d2b0 con 0x56150c82b5d0 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc963fff640 1 -- 172.21.15.143:0/1805555137 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc95c011ad0 con 0x7fc96c10ae80 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc96ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c82ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc96b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c82b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc96b59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c82b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42448/0 (socket says 172.21.15.143:42448) 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.734+0000 7fc96b59e640 1 -- 172.21.15.143:0/1153940245 learned_addr learned my addr 172.21.15.143:0/1153940245 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c840f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96b59e640 1 -- 172.21.15.143:0/1153940245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 msgr2=0x56150c82ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96b59e640 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c82ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96b59e640 1 -- 172.21.15.143:0/1153940245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 msgr2=0x56150c840f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96b59e640 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c840f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96b59e640 1 -- 172.21.15.143:0/1153940245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150c71d3a0 con 0x56150c82cb40 2024-09-10T10:48:24.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc96b59e640 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c82b090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc964011540 tx=0x7fc964011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e075510e6a67506b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc9617fa640 1 -- 172.21.15.143:0/1153940245 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96400f5c0 con 0x56150c82cb40 2024-09-10T10:48:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.735+0000 7fc9617fa640 1 -- 172.21.15.143:0/1153940245 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc96400fc10 con 0x56150c82cb40 2024-09-10T10:48:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.736+0000 7fc963fff640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9580a2c00 0x7fc9580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.736+0000 7fc963fff640 1 -- 172.21.15.143:0/1805555137 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc95c0cf980 con 0x7fc96c10ae80 2024-09-10T10:48:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.736+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9580a2c00 0x7fc9580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.736+0000 7fc9617fa640 1 -- 172.21.15.143:0/1153940245 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964019c30 con 0x56150c82cb40 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 -- 172.21.15.143:0/1153940245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 msgr2=0x56150c82b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c82b090 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc964011540 tx=0x7fc964011a00 comp rx=0 tx=0).stop 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 -- 172.21.15.143:0/1153940245 shutdown_connections 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c840f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c82ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 --2- 172.21.15.143:0/1153940245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c82b090 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 -- 172.21.15.143:0/1153940245 >> 172.21.15.143:0/1153940245 conn(0x56150c7f4310 msgr2=0x56150c827c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9580a2c00 0x7fc9580a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc954005d90 tx=0x7fc954009640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 -- 172.21.15.143:0/1153940245 shutdown_connections 2024-09-10T10:48:24.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.737+0000 7fc973462740 1 -- 172.21.15.143:0/1153940245 wait complete. 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 -- start start 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 Processor -- start 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c83e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c83b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c83bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56150c8418d0 con 0x56150c82cb40 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56150c74c060 con 0x56150c83eb80 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.738+0000 7fc973462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56150c841a50 con 0x56150c82b5d0 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c83e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c83e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47786/0 (socket says 172.21.15.143:47786) 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1444110986 learned_addr learned my addr 172.21.15.143:0/1444110986 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc973462740 10 client.0 _ll_register_callbacks cb 0x56150c49f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c83b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96b59e640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c83bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1444110986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 msgr2=0x56150c83bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c83bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1444110986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 msgr2=0x56150c83b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c83b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1444110986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56150c5023c0 con 0x56150c82b5d0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c83e4c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc954033000 tx=0x7fc954009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef933bd398304d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc9527fc640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.739+0000 7fc9527fc640 1 -- 172.21.15.143:0/1444110986 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954035020 con 0x56150c82b5d0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96400fc10 con 0x56150c82b5d0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc9527fc640 1 -- 172.21.15.143:0/1444110986 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc954031d60 con 0x56150c82b5d0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc9527fc640 1 -- 172.21.15.143:0/1444110986 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9540046c0 con 0x56150c82b5d0 2024-09-10T10:48:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc973462740 10 client.25702 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56150c84d340 con 0x56150c82b5d0 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc9527fc640 1 -- 172.21.15.143:0/1444110986 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc96400fc10 con 0x56150c82b5d0 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.740+0000 7fc9527fc640 1 -- 172.21.15.143:0/1444110986 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fc954045030 con 0x56150c82b5d0 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.741+0000 7fc973462740 10 client.25702 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.741+0000 7fc973462740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.741+0000 7fc973462740 20 client.25702 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.741+0000 7fc973462740 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c84e750 0x56150c850bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.741+0000 7fc973462740 4 client.25702 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.741+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56150c79e860 con 0x56150c84e750 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.742+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c84e750 0x56150c850bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.742+0000 7fc9527fc640 10 client.25702 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.742+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c84e750 0x56150c850bc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8e2e49e0ccb531 server_cookie=24dbb31514262c2d in_seq=0 out_seq=0 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.743+0000 7fc9527fc640 1 -- 172.21.15.143:0/1444110986 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56150c79e860 con 0x56150c84e750 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.743+0000 7fc9527fc640 10 client.25702 handle_command_reply: tid=0 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.743+0000 7fc973462740 1 client.25702 shutdown 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.743+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 msgr2=0x56150c83e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.743+0000 7fc973462740 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c83e4c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc954033000 tx=0x7fc954009260 comp rx=0 tx=0).stop 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 shutdown_connections 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc973462740 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c83bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc973462740 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c83b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc973462740 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c84e750 0x56150c850bc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc973462740 1 --2- 172.21.15.143:0/1444110986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c83e4c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 >> 172.21.15.143:0/1444110986 conn(0x56150c7f4310 msgr2=0x56150c82d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.744+0000 7fc9527fc640 0 client.25702 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.745+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 shutdown_connections 2024-09-10T10:48:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.745+0000 7fc973462740 1 -- 172.21.15.143:0/1444110986 wait complete. 2024-09-10T10:48:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.745+0000 7fc973462740 20 client.25702 trim_cache size 0 max 16384 2024-09-10T10:48:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.761+0000 7fc973462740 1 Processor -- start 2024-09-10T10:48:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 -- start start 2024-09-10T10:48:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 0x56150c84abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c82cb40 0x56150c856490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c83eb80 0x56150c8569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1b51a0 con 0x56150c82b5d0 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1b6d10 con 0x56150c82cb40 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.762+0000 7fc973462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1b5870 con 0x56150c83eb80 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 0x56150c84abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 0x56150c84abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42470/0 (socket says 172.21.15.143:42470) 2024-09-10T10:48:24.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1277970001 learned_addr learned my addr 172.21.15.143:0/1277970001 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96a59c640 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c82cb40 0x56150c856490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1277970001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c83eb80 msgr2=0x56150c8569d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c83eb80 0x56150c8569d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1277970001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c82cb40 msgr2=0x56150c856490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c82cb40 0x56150c856490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 -- 172.21.15.143:0/1277970001 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150c83e0e0 con 0x56150c82b5d0 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.763+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 0x56150c84abc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc954003ac0 tx=0x7fc954005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23889502a9bc01f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.764+0000 7fc960ff9640 1 -- 172.21.15.143:0/1277970001 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954035040 con 0x56150c82b5d0 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.764+0000 7fc960ff9640 1 -- 172.21.15.143:0/1277970001 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc95404d070 con 0x56150c82b5d0 2024-09-10T10:48:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.764+0000 7fc960ff9640 1 -- 172.21.15.143:0/1277970001 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954004bc0 con 0x56150c82b5d0 2024-09-10T10:48:24.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 -- 172.21.15.143:0/1277970001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 msgr2=0x56150c84abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 0x56150c84abc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc954003ac0 tx=0x7fc954005df0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 -- 172.21.15.143:0/1277970001 shutdown_connections 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c83eb80 0x56150c8569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c82cb40 0x56150c856490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 --2- 172.21.15.143:0/1277970001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82b5d0 0x56150c84abc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.765+0000 7fc973462740 1 -- 172.21.15.143:0/1277970001 >> 172.21.15.143:0/1277970001 conn(0x56150c7f4310 msgr2=0x56150c83b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.766+0000 7fc973462740 1 -- 172.21.15.143:0/1277970001 shutdown_connections 2024-09-10T10:48:24.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.766+0000 7fc973462740 1 -- 172.21.15.143:0/1277970001 wait complete. 2024-09-10T10:48:24.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 -- start start 2024-09-10T10:48:24.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 Processor -- start 2024-09-10T10:48:24.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c8432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c843830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c8b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1b6610 con 0x56150c82cb40 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1b5f40 con 0x56150c83eb80 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1b3180 con 0x56150c82b5d0 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc973462740 10 client.0 _ll_register_callbacks cb 0x7fc954009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc96a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c843830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc96ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c8432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.767+0000 7fc96a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c843830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42476/0 (socket says 172.21.15.143:42476) 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 -- 172.21.15.143:0/3876505871 learned_addr learned my addr 172.21.15.143:0/3876505871 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:24.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96b59e640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c8b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 -- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 msgr2=0x56150c8432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c8432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 -- 172.21.15.143:0/3876505871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 msgr2=0x56150c8b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c8b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 -- 172.21.15.143:0/3876505871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56150c8b2a70 con 0x56150c82cb40 2024-09-10T10:48:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96b59e640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c8b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc96a59c640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c843830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc95c09bdb0 tx=0x7fc95c01a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7704b8e3d23302b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.768+0000 7fc9527fc640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc9527fc640 1 -- 172.21.15.143:0/3876505871 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c09c520 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc9527fc640 1 -- 172.21.15.143:0/3876505871 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc95c0cd1d0 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95404d070 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc9527fc640 1 -- 172.21.15.143:0/3876505871 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc95c09be20 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc973462740 10 client.16290 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56150c8b9e70 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.769+0000 7fc9527fc640 1 -- 172.21.15.143:0/3876505871 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc95404d070 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc9527fc640 1 -- 172.21.15.143:0/3876505871 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fc95c022030 con 0x56150c82cb40 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc973462740 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc973462740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc973462740 20 client.16290 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc973462740 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c8ba560 0x56150c8bc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:24.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc973462740 4 client.16290 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:48:24.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56150c798d70 con 0x56150c8ba560 2024-09-10T10:48:24.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.770+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c8ba560 0x56150c8bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.771+0000 7fc96ad9d640 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c8ba560 0x56150c8bc980 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5d62c5488aaed2 server_cookie=7b213fd29b47328a in_seq=0 out_seq=0 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.771+0000 7fc9527fc640 10 client.16290 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.771+0000 7fc9527fc640 1 -- 172.21.15.143:0/3876505871 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56150c798d70 con 0x56150c8ba560 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.771+0000 7fc9527fc640 10 client.16290 handle_command_reply: tid=0 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.771+0000 7fc973462740 1 client.16290 shutdown 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 msgr2=0x56150c843830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c843830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc95c09bdb0 tx=0x7fc95c01a400 comp rx=0 tx=0).stop 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 shutdown_connections 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56150c83eb80 0x56150c8b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56150c82cb40 0x56150c843830 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56150c8ba560 0x56150c8bc980 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 --2- 172.21.15.143:0/3876505871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56150c82b5d0 0x56150c8432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 >> 172.21.15.143:0/3876505871 conn(0x56150c7f4310 msgr2=0x56150c83cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc9527fc640 0 client.16290 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 shutdown_connections 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.772+0000 7fc973462740 1 -- 172.21.15.143:0/3876505871 wait complete. 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.773+0000 7fc973462740 20 client.16290 trim_cache size 0 max 16384 2024-09-10T10:48:24.775 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:48:24.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.773+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9580a2c00 msgr2=0x7fc9580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.773+0000 7fc971089640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9580a2c00 0x7fc9580a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc954005d90 tx=0x7fc954009640 comp rx=0 tx=0).stop 2024-09-10T10:48:24.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.773+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 msgr2=0x7fc96c1a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:24.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.773+0000 7fc971089640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 0x7fc96c1a32f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00ea80 tx=0x7fc95c00ef40 comp rx=0 tx=0).stop 2024-09-10T10:48:24.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 shutdown_connections 2024-09-10T10:48:24.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc9580a2c00 0x7fc9580a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96c1139e0 0x7fc96c1a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc96c10ae80 0x7fc96c1a32f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 --2- 172.21.15.143:0/1805555137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc96c10a560 0x7fc96c1a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:24.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 >> 172.21.15.143:0/1805555137 conn(0x7fc96c06e2d0 msgr2=0x7fc96c109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:24.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 shutdown_connections 2024-09-10T10:48:24.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:24.774+0000 7fc971089640 1 -- 172.21.15.143:0/1805555137 wait complete. 2024-09-10T10:48:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:24 smithi178 ceph-mon[34813]: pgmap v451: 129 pgs: 129 active+clean; 261 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 327 op/s 2024-09-10T10:48:24.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:24 smithi144 ceph-mon[34230]: pgmap v451: 129 pgs: 129 active+clean; 261 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 327 op/s 2024-09-10T10:48:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:24 smithi143 ceph-mon[31057]: pgmap v451: 129 pgs: 129 active+clean; 261 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 327 op/s 2024-09-10T10:48:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:26 smithi178 ceph-mon[34813]: pgmap v452: 129 pgs: 129 active+clean; 267 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 320 op/s 2024-09-10T10:48:26.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:26 smithi144 ceph-mon[34230]: pgmap v452: 129 pgs: 129 active+clean; 267 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 320 op/s 2024-09-10T10:48:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:26 smithi143 ceph-mon[31057]: pgmap v452: 129 pgs: 129 active+clean; 267 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 320 op/s 2024-09-10T10:48:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:28 smithi178 ceph-mon[34813]: pgmap v453: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 353 op/s 2024-09-10T10:48:28.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:28 smithi144 ceph-mon[34230]: pgmap v453: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 353 op/s 2024-09-10T10:48:29.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:28 smithi143 ceph-mon[31057]: pgmap v453: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 353 op/s 2024-09-10T10:48:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:30 smithi178 ceph-mon[34813]: pgmap v454: 129 pgs: 129 active+clean; 288 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 328 op/s 2024-09-10T10:48:30.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:30 smithi144 ceph-mon[34230]: pgmap v454: 129 pgs: 129 active+clean; 288 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 328 op/s 2024-09-10T10:48:31.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:30 smithi143 ceph-mon[31057]: pgmap v454: 129 pgs: 129 active+clean; 288 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 328 op/s 2024-09-10T10:48:31.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:48:31.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:48:32.113 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:32 smithi178 ceph-mon[34813]: pgmap v455: 129 pgs: 129 active+clean; 300 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 305 op/s 2024-09-10T10:48:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:32 smithi144 ceph-mon[34230]: pgmap v455: 129 pgs: 129 active+clean; 300 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 305 op/s 2024-09-10T10:48:33.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:32 smithi143 ceph-mon[31057]: pgmap v455: 129 pgs: 129 active+clean; 300 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 305 op/s 2024-09-10T10:48:33.591 INFO:tasks.workunit.client.0.smithi143.stdout: 69 5797 402 3779 416 3131 478 2024-09-10T10:48:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 -- 172.21.15.143:0/1925183449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 msgr2=0x7efe28101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/1925183449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28101200 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efe24009900 tx=0x7efe2402a480 comp rx=0 tx=0).stop 2024-09-10T10:48:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 -- 172.21.15.143:0/1925183449 shutdown_connections 2024-09-10T10:48:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/1925183449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102ef0 0x7efe2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/1925183449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28102570 0x7efe281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/1925183449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28101200 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 -- 172.21.15.143:0/1925183449 >> 172.21.15.143:0/1925183449 conn(0x7efe280fc7d0 msgr2=0x7efe280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 -- 172.21.15.143:0/1925183449 shutdown_connections 2024-09-10T10:48:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.709+0000 7efe2fcfa640 1 -- 172.21.15.143:0/1925183449 wait complete. 2024-09-10T10:48:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 Processor -- start 2024-09-10T10:48:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 -- start start 2024-09-10T10:48:33.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102570 0x7efe28198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28102ef0 0x7efe2819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe28110e30 con 0x7efe28100e20 2024-09-10T10:48:33.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe28110cb0 con 0x7efe28102ef0 2024-09-10T10:48:33.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2fcfa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe28110fb0 con 0x7efe28102570 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2da96640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2da96640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54484/0 (socket says 172.21.15.143:54484) 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2d295640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102570 0x7efe28198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.710+0000 7efe2da96640 1 -- 172.21.15.143:0/4149175680 learned_addr learned my addr 172.21.15.143:0/4149175680 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2e297640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28102ef0 0x7efe2819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2d295640 1 -- 172.21.15.143:0/4149175680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28102ef0 msgr2=0x7efe2819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2d295640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28102ef0 0x7efe2819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2d295640 1 -- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 msgr2=0x7efe28198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2d295640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2d295640 1 -- 172.21.15.143:0/4149175680 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2819d3d0 con 0x7efe28102570 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2da96640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2d295640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102570 0x7efe28198960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe18009bc0 tx=0x7efe1800b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe1effd640 1 -- 172.21.15.143:0/4149175680 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1801e070 con 0x7efe28102570 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2819d6c0 con 0x7efe28102570 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe1effd640 1 -- 172.21.15.143:0/4149175680 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe180198c0 con 0x7efe28102570 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe1effd640 1 -- 172.21.15.143:0/4149175680 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18018710 con 0x7efe28102570 2024-09-10T10:48:33.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.711+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe280fcbb0 con 0x7efe28102570 2024-09-10T10:48:33.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.714+0000 7efe1effd640 1 -- 172.21.15.143:0/4149175680 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efe18021070 con 0x7efe28102570 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 Processor -- start 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 -- start start 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe1effd640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efe140a2c00 0x7efe140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a35800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a115990280 con 0x55a115a2b480 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a115937b10 con 0x55a115a2ab60 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe320d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a11598c240 con 0x55a115a35d40 2024-09-10T10:48:33.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe1effd640 1 -- 172.21.15.143:0/4149175680 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efe180d2a50 con 0x7efe28102570 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe2e297640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efe140a2c00 0x7efe140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe2d295640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a35800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.715+0000 7efe2d295640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a35800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54486/0 (socket says 172.21.15.143:54486) 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 -- 172.21.15.143:0/2879719389 learned_addr learned my addr 172.21.15.143:0/2879719389 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2da96640 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2e297640 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 -- 172.21.15.143:0/2879719389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 msgr2=0x55a115a38130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a38130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 -- 172.21.15.143:0/2879719389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 msgr2=0x55a115a2af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a2af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 -- 172.21.15.143:0/2879719389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1159441f0 con 0x55a115a2b480 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2e297640 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:48:33.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2e297640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efe140a2c00 0x7efe140a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55a115a28470 tx=0x7efe20009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe2d295640 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a35800 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe180a7690 tx=0x7efe1809eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4f3c5ca6cb6686 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe13fff640 1 -- 172.21.15.143:0/2879719389 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe180a3300 con 0x55a115a2b480 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.716+0000 7efe13fff640 1 -- 172.21.15.143:0/2879719389 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe180a7bc0 con 0x55a115a2b480 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe13fff640 1 -- 172.21.15.143:0/2879719389 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe180bddd0 con 0x55a115a2b480 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 -- 172.21.15.143:0/2879719389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 msgr2=0x55a115a35800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a35800 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe180a7690 tx=0x7efe1809eee0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 -- 172.21.15.143:0/2879719389 shutdown_connections 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a35800 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 --2- 172.21.15.143:0/2879719389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a2af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.717+0000 7efe320d3740 1 -- 172.21.15.143:0/2879719389 >> 172.21.15.143:0/2879719389 conn(0x55a115a270f0 msgr2=0x55a115a274d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:33.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.718+0000 7efe320d3740 1 -- 172.21.15.143:0/2879719389 shutdown_connections 2024-09-10T10:48:33.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.718+0000 7efe320d3740 1 -- 172.21.15.143:0/2879719389 wait complete. 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.718+0000 7efe320d3740 1 -- start start 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.718+0000 7efe320d3740 1 Processor -- start 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a32bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a33110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a33650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a115a38960 con 0x55a115a2b480 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a11594af10 con 0x55a115a2ab60 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a115a38ae0 con 0x55a115a35d40 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe320d3740 10 client.0 _ll_register_callbacks cb 0x55a115913a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2d295640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a33110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2da96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a32bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a33650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2d295640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a33110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54498/0 (socket says 172.21.15.143:54498) 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2da96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a32bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:50408/0 (socket says 172.21.15.143:50408) 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2d295640 1 -- 172.21.15.143:0/3103050911 learned_addr learned my addr 172.21.15.143:0/3103050911 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 -- 172.21.15.143:0/3103050911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 msgr2=0x55a115a32bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a32bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 -- 172.21.15.143:0/3103050911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 msgr2=0x55a115a33110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a33110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 -- 172.21.15.143:0/3103050911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1157016a0 con 0x55a115a35d40 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2da96640 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a32bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe2e297640 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a33650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efe20005fd0 tx=0x7efe20004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259ee2cc2dfc9cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.719+0000 7efe117fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:48:33.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe117fa640 1 -- 172.21.15.143:0/3103050911 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20024030 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe117fa640 1 -- 172.21.15.143:0/3103050911 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efe20009040 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe180a7bc0 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe117fa640 1 -- 172.21.15.143:0/3103050911 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe20030050 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 10 client.25714 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a115a444e0 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe117fa640 1 -- 172.21.15.143:0/3103050911 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe180a7bc0 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe117fa640 1 -- 172.21.15.143:0/3103050911 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7efe200283c0 con 0x55a115a35d40 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 10 client.25714 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 20 client.25714 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115a45940 0x55a115a47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 4 client.25714 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:33.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.720+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a11599d7f0 con 0x55a115a45940 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.721+0000 7efe2da96640 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115a45940 0x55a115a47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.721+0000 7efe117fa640 10 client.25714 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.721+0000 7efe2da96640 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115a45940 0x55a115a47da0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1f48637719ae10 server_cookie=a2b621afbbe53f52 in_seq=0 out_seq=0 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe117fa640 1 -- 172.21.15.143:0/3103050911 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a11599d7f0 con 0x55a115a45940 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe117fa640 10 client.25714 handle_command_reply: tid=0 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 client.25714 shutdown 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 msgr2=0x55a115a33650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a33650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efe20005fd0 tx=0x7efe20004a40 comp rx=0 tx=0).stop 2024-09-10T10:48:33.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 shutdown_connections 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115a45940 0x55a115a47da0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a33650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115a33110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe117fa640 0 client.25714 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 --2- 172.21.15.143:0/3103050911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115a32bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.722+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 >> 172.21.15.143:0/3103050911 conn(0x55a115a270f0 msgr2=0x55a115a2ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.723+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 shutdown_connections 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.723+0000 7efe320d3740 1 -- 172.21.15.143:0/3103050911 wait complete. 2024-09-10T10:48:33.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.723+0000 7efe320d3740 20 client.25714 trim_cache size 0 max 16384 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 Processor -- start 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 -- start start 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 0x55a115a2d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2b480 0x55a115a4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a4dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe281ac8d0 con 0x55a115a2ab60 2024-09-10T10:48:33.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe281ae440 con 0x55a115a2b480 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe320d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe281acfa0 con 0x55a115a35d40 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe2da96640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 0x55a115a2d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe2da96640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 0x55a115a2d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54506/0 (socket says 172.21.15.143:54506) 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe2da96640 1 -- 172.21.15.143:0/1847081706 learned_addr learned my addr 172.21.15.143:0/1847081706 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.748+0000 7efe2d295640 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2b480 0x55a115a4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe2da96640 1 -- 172.21.15.143:0/1847081706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 msgr2=0x55a115a4dbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe2da96640 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a4dbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe2da96640 1 -- 172.21.15.143:0/1847081706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2b480 msgr2=0x55a115a4d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe2da96640 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2b480 0x55a115a4d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe2da96640 1 -- 172.21.15.143:0/1847081706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a115a22830 con 0x55a115a2ab60 2024-09-10T10:48:33.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe2da96640 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 0x55a115a2d5c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efe2402a4c0 tx=0x7efe24005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2346faa984feee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:33.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe137fe640 1 -- 172.21.15.143:0/1847081706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24035040 con 0x55a115a2ab60 2024-09-10T10:48:33.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe137fe640 1 -- 172.21.15.143:0/1847081706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe2404b070 con 0x55a115a2ab60 2024-09-10T10:48:33.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.749+0000 7efe137fe640 1 -- 172.21.15.143:0/1847081706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240049c0 con 0x55a115a2ab60 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 -- 172.21.15.143:0/1847081706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 msgr2=0x55a115a2d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 0x55a115a2d5c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efe2402a4c0 tx=0x7efe24005a20 comp rx=0 tx=0).stop 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 -- 172.21.15.143:0/1847081706 shutdown_connections 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a4dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2b480 0x55a115a4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 --2- 172.21.15.143:0/1847081706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2ab60 0x55a115a2d5c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 -- 172.21.15.143:0/1847081706 >> 172.21.15.143:0/1847081706 conn(0x55a115a270f0 msgr2=0x55a115a34b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:33.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 -- 172.21.15.143:0/1847081706 shutdown_connections 2024-09-10T10:48:33.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.750+0000 7efe320d3740 1 -- 172.21.15.143:0/1847081706 wait complete. 2024-09-10T10:48:33.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 -- start start 2024-09-10T10:48:33.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 Processor -- start 2024-09-10T10:48:33.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115aa9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115aaa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a3dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe281add40 con 0x55a115a2b480 2024-09-10T10:48:33.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe281ad670 con 0x55a115a2ab60 2024-09-10T10:48:33.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.751+0000 7efe320d3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe281aa8b0 con 0x55a115a35d40 2024-09-10T10:48:33.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe320d3740 10 client.0 _ll_register_callbacks cb 0x7efe2000e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:33.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2e297640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a3dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115aaa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115aaa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54508/0 (socket says 172.21.15.143:54508) 2024-09-10T10:48:33.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2e297640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a3dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35752/0 (socket says 172.21.15.143:35752) 2024-09-10T10:48:33.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 -- 172.21.15.143:0/2777650675 learned_addr learned my addr 172.21.15.143:0/2777650675 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:33.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2da96640 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115aa9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 -- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 msgr2=0x55a115a3dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a3dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 -- 172.21.15.143:0/2777650675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 msgr2=0x55a115aa9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:33.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115aa9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 -- 172.21.15.143:0/2777650675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a115a2b2b0 con 0x55a115a2b480 2024-09-10T10:48:33.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe2d295640 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115aaa190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efe1809b460 tx=0x7efe1800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346959538f2a45b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:33.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.752+0000 7efe117fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:48:33.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe117fa640 1 -- 172.21.15.143:0/2777650675 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18082060 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe117fa640 1 -- 172.21.15.143:0/2777650675 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efe18010400 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe2404b070 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe117fa640 1 -- 172.21.15.143:0/2777650675 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe18019a70 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 10 client.16308 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a115ab9610 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe117fa640 1 -- 172.21.15.143:0/2777650675 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe2404b070 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe117fa640 1 -- 172.21.15.143:0/2777650675 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7efe180bdcb0 con 0x55a115a2b480 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 10 client.16308 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 20 client.16308 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:33.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115ab9990 0x55a115abbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 4 client.16308 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.753+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a115997d00 con 0x55a115ab9990 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.754+0000 7efe2da96640 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115ab9990 0x55a115abbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.755+0000 7efe117fa640 10 client.16308 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.755+0000 7efe2da96640 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115ab9990 0x55a115abbdb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a4d8e408a3b3b5 server_cookie=ed49c08454d93956 in_seq=0 out_seq=0 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.756+0000 7efe117fa640 1 -- 172.21.15.143:0/2777650675 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a115997d00 con 0x55a115ab9990 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.756+0000 7efe117fa640 10 client.16308 handle_command_reply: tid=0 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.756+0000 7efe320d3740 1 client.16308 shutdown 2024-09-10T10:48:33.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.756+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 msgr2=0x55a115aaa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.756+0000 7efe320d3740 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115aaa190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efe1809b460 tx=0x7efe1800be60 comp rx=0 tx=0).stop 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.756+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 shutdown_connections 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55a115ab9990 0x55a115abbdb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a115a35d40 0x55a115a3dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a115a2b480 0x55a115aaa190 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 --2- 172.21.15.143:0/2777650675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a115a2ab60 0x55a115aa9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 >> 172.21.15.143:0/2777650675 conn(0x55a115a270f0 msgr2=0x55a115a32b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe117fa640 0 client.16308 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 shutdown_connections 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 1 -- 172.21.15.143:0/2777650675 wait complete. 2024-09-10T10:48:33.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.757+0000 7efe320d3740 20 client.16308 trim_cache size 0 max 16384 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "e0fc5415-c5c1-440e-b0d4-f065b710998b", 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efe140a2c00 msgr2=0x7efe140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efe140a2c00 0x7efe140a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55a115a28470 tx=0x7efe20009da0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102570 msgr2=0x7efe28198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102570 0x7efe28198960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe18009bc0 tx=0x7efe1800b690 comp rx=0 tx=0).stop 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 shutdown_connections 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efe140a2c00 0x7efe140a50c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe28102ef0 0x7efe2819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe28102570 0x7efe28198960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 --2- 172.21.15.143:0/4149175680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe28100e20 0x7efe28198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 >> 172.21.15.143:0/4149175680 conn(0x7efe280fc7d0 msgr2=0x7efe2806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:33.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 shutdown_connections 2024-09-10T10:48:33.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:33.758+0000 7efe2fcfa640 1 -- 172.21.15.143:0/4149175680 wait complete. 2024-09-10T10:48:34.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: pgmap v456: 129 pgs: 129 active+clean; 303 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:48:34.886 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:34 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: pgmap v456: 129 pgs: 129 active+clean; 303 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:48:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:34 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: pgmap v456: 129 pgs: 129 active+clean; 303 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2024-09-10T10:48:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:48:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:48:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:48:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:48:34.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:34 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:35.208 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:48:35.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 -- 172.21.15.143:0/178208788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc102ef0 msgr2=0x7fc2cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 --2- 172.21.15.143:0/178208788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10da40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009c10 tx=0x7fc2c402a770 comp rx=0 tx=0).stop 2024-09-10T10:48:35.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 -- 172.21.15.143:0/178208788 shutdown_connections 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 --2- 172.21.15.143:0/178208788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10da40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 --2- 172.21.15.143:0/178208788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 0x7fc2cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 --2- 172.21.15.143:0/178208788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc100e20 0x7fc2cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 -- 172.21.15.143:0/178208788 >> 172.21.15.143:0/178208788 conn(0x7fc2cc0fc7d0 msgr2=0x7fc2cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 -- 172.21.15.143:0/178208788 shutdown_connections 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.749+0000 7fc2d3469640 1 -- 172.21.15.143:0/178208788 wait complete. 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 Processor -- start 2024-09-10T10:48:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 -- start start 2024-09-10T10:48:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc100e20 0x7fc2cc10fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 0x7fc2cc10e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc111f40 con 0x7fc2cc102ef0 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc111dc0 con 0x7fc2cc100e20 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d3469640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1120c0 con 0x7fc2cc102570 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d1a06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d1205640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc100e20 0x7fc2cc10fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d1a06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54528/0 (socket says 172.21.15.143:54528) 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d1a06640 1 -- 172.21.15.143:0/2872599887 learned_addr learned my addr 172.21.15.143:0/2872599887 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.750+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 0x7fc2cc10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2d0a04640 1 -- 172.21.15.143:0/2872599887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc100e20 msgr2=0x7fc2cc10fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc100e20 0x7fc2cc10fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2d0a04640 1 -- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc102ef0 msgr2=0x7fc2cc10e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2d0a04640 1 -- 172.21.15.143:0/2872599887 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc10ef90 con 0x7fc2cc102570 2024-09-10T10:48:35.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 0x7fc2cc10e350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9f0 tx=0x7fc2bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:35.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2c27fc640 1 -- 172.21.15.143:0/2872599887 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00ccc0 con 0x7fc2cc102570 2024-09-10T10:48:35.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2c27fc640 1 -- 172.21.15.143:0/2872599887 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2bc005020 con 0x7fc2cc102570 2024-09-10T10:48:35.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.751+0000 7fc2c27fc640 1 -- 172.21.15.143:0/2872599887 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc004a00 con 0x7fc2cc102570 2024-09-10T10:48:35.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.752+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1af420 con 0x7fc2cc102570 2024-09-10T10:48:35.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.752+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1af820 con 0x7fc2cc102570 2024-09-10T10:48:35.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.755+0000 7fc2c27fc640 1 -- 172.21.15.143:0/2872599887 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc2bc004ba0 con 0x7fc2cc102570 2024-09-10T10:48:35.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.756+0000 7fc2c27fc640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2b80a2c00 0x7fc2b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.756+0000 7fc2d1205640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2b80a2c00 0x7fc2b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.756+0000 7fc2d5842740 1 Processor -- start 2024-09-10T10:48:35.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.756+0000 7fc2d5842740 1 -- start start 2024-09-10T10:48:35.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d1205640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2b80a2c00 0x7fc2b80a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009530 tx=0x7fc2c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:35.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2c27fc640 1 -- 172.21.15.143:0/2872599887 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2bc0cf310 con 0x7fc2cc102570 2024-09-10T10:48:35.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a22c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 0x55ef7a237140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a239a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d5842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ef7a1922e0 con 0x55ef7a22cdc0 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d5842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef7a139ce0 con 0x55ef7a237680 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d5842740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef7a18e2a0 con 0x55ef7a22c4a0 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 0x55ef7a237140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 0x55ef7a237140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54538/0 (socket says 172.21.15.143:54538) 2024-09-10T10:48:35.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 -- 172.21.15.143:0/1328665075 learned_addr learned my addr 172.21.15.143:0/1328665075 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d1205640 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a22c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d1a06640 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a239a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 -- 172.21.15.143:0/1328665075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 msgr2=0x55ef7a22c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a22c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 -- 172.21.15.143:0/1328665075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 msgr2=0x55ef7a239a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a239a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.757+0000 7fc2d0a04640 1 -- 172.21.15.143:0/1328665075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7a11e440 con 0x55ef7a22cdc0 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.758+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 0x55ef7a237140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc098180 tx=0x7fc2bc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded95756a288393a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.758+0000 7fc2b77fe640 1 -- 172.21.15.143:0/1328665075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc01c040 con 0x55ef7a22cdc0 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.758+0000 7fc2b77fe640 1 -- 172.21.15.143:0/1328665075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2bc0a4420 con 0x55ef7a22cdc0 2024-09-10T10:48:35.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.758+0000 7fc2b77fe640 1 -- 172.21.15.143:0/1328665075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc097e20 con 0x55ef7a22cdc0 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 -- 172.21.15.143:0/1328665075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 msgr2=0x55ef7a237140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 0x55ef7a237140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc098180 tx=0x7fc2bc009f60 comp rx=0 tx=0).stop 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 -- 172.21.15.143:0/1328665075 shutdown_connections 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a239a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22cdc0 0x55ef7a237140 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 --2- 172.21.15.143:0/1328665075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a22c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 -- 172.21.15.143:0/1328665075 >> 172.21.15.143:0/1328665075 conn(0x55ef7a228a30 msgr2=0x55ef7a228e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:35.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 -- 172.21.15.143:0/1328665075 shutdown_connections 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 -- 172.21.15.143:0/1328665075 wait complete. 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.759+0000 7fc2d5842740 1 -- start start 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 Processor -- start 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a234490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a2349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a234f10 0x55ef7a24bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ef7a23a250 con 0x55ef7a234f10 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef7a14d050 con 0x55ef7a237680 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef7a23a3d0 con 0x55ef7a22c4a0 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d5842740 10 client.0 _ll_register_callbacks cb 0x55ef7a18d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d1a06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a234f10 0x55ef7a24bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d1205640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a234490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d1a06640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a234f10 0x55ef7a24bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54552/0 (socket says 172.21.15.143:54552) 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d1a06640 1 -- 172.21.15.143:0/3903796757 learned_addr learned my addr 172.21.15.143:0/3903796757 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.760+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a2349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d1205640 1 -- 172.21.15.143:0/3903796757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 msgr2=0x55ef7a2349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d1205640 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a2349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d1205640 1 -- 172.21.15.143:0/3903796757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a234f10 msgr2=0x55ef7a24bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d1205640 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a234f10 0x55ef7a24bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d1205640 1 -- 172.21.15.143:0/3903796757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef79f033c0 con 0x55ef7a22c4a0 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d1205640 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a234490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009fd0 tx=0x7fc2c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b368fe2a966bfd01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:35.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/3903796757 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8005020 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/3903796757 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2c8009040 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.761+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc0a4420 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/3903796757 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2c8030050 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 10 client.25729 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef7a24ea60 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/3903796757 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2c800e7c0 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/3903796757 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fc2c8024070 con 0x55ef7a22c4a0 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 10 client.25729 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:35.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 20 client.25729 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a24ff20 0x55ef7a252350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.762+0000 7fc2d5842740 4 client.25729 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.763+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2bc0a4420 con 0x55ef7a24ff20 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.763+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a24ff20 0x55ef7a252350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.763+0000 7fc2b4ff9640 10 client.25729 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.763+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a24ff20 0x55ef7a252350 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb24120b8daced91 server_cookie=6d2bd414fc34070 in_seq=0 out_seq=0 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/3903796757 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc2bc0a4420 con 0x55ef7a24ff20 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2b4ff9640 10 client.25729 handle_command_reply: tid=0 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 client.25729 shutdown 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 msgr2=0x55ef7a234490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a234490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009fd0 tx=0x7fc2c8004730 comp rx=0 tx=0).stop 2024-09-10T10:48:35.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 shutdown_connections 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a234f10 0x55ef7a24bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a237680 0x55ef7a2349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a24ff20 0x55ef7a252350 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 --2- 172.21.15.143:0/3903796757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a22c4a0 0x55ef7a234490 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 >> 172.21.15.143:0/3903796757 conn(0x55ef7a228a30 msgr2=0x55ef7a22e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.764+0000 7fc2b4ff9640 0 client.25729 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.765+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 shutdown_connections 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.765+0000 7fc2d5842740 1 -- 172.21.15.143:0/3903796757 wait complete. 2024-09-10T10:48:35.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.765+0000 7fc2d5842740 20 client.25729 trim_cache size 0 max 16384 2024-09-10T10:48:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.794+0000 7fc2d5842740 1 Processor -- start 2024-09-10T10:48:35.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.794+0000 7fc2d5842740 1 -- start start 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.794+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a228a30 0x55ef7a24bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.794+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 0x55ef7a24c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a234f10 0x55ef7a257c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d5842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1b51c0 con 0x55ef7a228a30 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d5842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1b6d30 con 0x55ef7a22c4a0 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d5842740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1b5890 con 0x55ef7a234f10 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 0x55ef7a24c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 0x55ef7a24c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:50482/0 (socket says 172.21.15.143:50482) 2024-09-10T10:48:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 -- 172.21.15.143:0/4114803332 learned_addr learned my addr 172.21.15.143:0/4114803332 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:35.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d1a06640 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a234f10 0x55ef7a257c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d1205640 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a228a30 0x55ef7a24bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 -- 172.21.15.143:0/4114803332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a234f10 msgr2=0x55ef7a257c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a234f10 0x55ef7a257c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 -- 172.21.15.143:0/4114803332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a228a30 msgr2=0x55ef7a24bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a228a30 0x55ef7a24bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 -- 172.21.15.143:0/4114803332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7a235430 con 0x55ef7a22c4a0 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.795+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 0x55ef7a24c2b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc09b740 tx=0x7fc2bc09b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af3ee2bbd64d106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.796+0000 7fc2b6ffd640 1 -- 172.21.15.143:0/4114803332 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc01c040 con 0x55ef7a22c4a0 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.796+0000 7fc2b6ffd640 1 -- 172.21.15.143:0/4114803332 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2bc005560 con 0x55ef7a22c4a0 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.796+0000 7fc2b6ffd640 1 -- 172.21.15.143:0/4114803332 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0a3f90 con 0x55ef7a22c4a0 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 -- 172.21.15.143:0/4114803332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 msgr2=0x55ef7a24c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 0x55ef7a24c2b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc09b740 tx=0x7fc2bc09b8f0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 -- 172.21.15.143:0/4114803332 shutdown_connections 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a234f10 0x55ef7a257c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a22c4a0 0x55ef7a24c2b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 --2- 172.21.15.143:0/4114803332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a228a30 0x55ef7a24bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 -- 172.21.15.143:0/4114803332 >> 172.21.15.143:0/4114803332 conn(0x55ef7a2294b0 msgr2=0x55ef7a227b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 -- 172.21.15.143:0/4114803332 shutdown_connections 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.797+0000 7fc2d5842740 1 -- 172.21.15.143:0/4114803332 wait complete. 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.798+0000 7fc2d5842740 1 -- start start 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.798+0000 7fc2d5842740 1 Processor -- start 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 0x55ef7a231530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22c4a0 0x55ef7a231a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 0x55ef7a231fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1b6630 con 0x55ef7a22c4a0 2024-09-10T10:48:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1b5f60 con 0x55ef7a234f10 2024-09-10T10:48:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1b31a0 con 0x55ef7a228a30 2024-09-10T10:48:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d5842740 10 client.0 _ll_register_callbacks cb 0x7fc2c801c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1205640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 0x55ef7a231530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 0x55ef7a231fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 0x55ef7a231fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:50484/0 (socket says 172.21.15.143:50484) 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 -- 172.21.15.143:0/859452384 learned_addr learned my addr 172.21.15.143:0/859452384 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1205640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 0x55ef7a231530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35818/0 (socket says 172.21.15.143:35818) 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22c4a0 0x55ef7a231a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 -- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 msgr2=0x55ef7a231530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 0x55ef7a231530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 -- 172.21.15.143:0/859452384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22c4a0 msgr2=0x55ef7a231a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22c4a0 0x55ef7a231a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1a06640 1 -- 172.21.15.143:0/859452384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7a2b9c70 con 0x55ef7a234f10 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d1205640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 0x55ef7a231530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.799+0000 7fc2d0a04640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22c4a0 0x55ef7a231a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2d1a06640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 0x55ef7a231fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4004af0 tx=0x7fc2c4003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf88416b36e00f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/859452384 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4036040 con 0x55ef7a234f10 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/859452384 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2c4030920 con 0x55ef7a234f10 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc005560 con 0x55ef7a234f10 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/859452384 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2c4053050 con 0x55ef7a234f10 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2d5842740 10 client.25703 fetch_fsmap learned FSMap version 32 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.800+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef7a2baf60 con 0x55ef7a234f10 2024-09-10T10:48:35.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/859452384 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2bc005560 con 0x55ef7a234f10 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/859452384 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fc2c403c040 con 0x55ef7a234f10 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d5842740 10 client.25703 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d5842740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d5842740 20 client.25703 populate_metadata read hostname 'smithi143' 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d5842740 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a237680 0x55ef7a2bd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d5842740 4 client.25703 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef7a228300 con 0x55ef7a237680 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.801+0000 7fc2d1205640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a237680 0x55ef7a2bd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2d1205640 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a237680 0x55ef7a2bd7a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9946468b449def7 server_cookie=ff7a08bacbc20f3f in_seq=0 out_seq=0 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2b4ff9640 10 client.25703 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2b4ff9640 1 -- 172.21.15.143:0/859452384 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ef7a228300 con 0x55ef7a237680 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2b4ff9640 10 client.25703 handle_command_reply: tid=0 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2d5842740 1 client.25703 shutdown 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 msgr2=0x55ef7a231fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.802+0000 7fc2d5842740 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 0x55ef7a231fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4004af0 tx=0x7fc2c4003cc0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 shutdown_connections 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef7a234f10 0x55ef7a231fb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ef7a22c4a0 0x55ef7a231a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55ef7a237680 0x55ef7a2bd7a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 --2- 172.21.15.143:0/859452384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef7a228a30 0x55ef7a231530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 >> 172.21.15.143:0/859452384 conn(0x55ef7a2294b0 msgr2=0x55ef7a233ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2b4ff9640 0 client.25703 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 shutdown_connections 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 1 -- 172.21.15.143:0/859452384 wait complete. 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.803+0000 7fc2d5842740 20 client.25703 trim_cache size 0 max 16384 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:48:35.812 INFO:teuthology.orchestra.run.smithi143.stdout: "e0fc5415-c5c1-440e-b0d4-f065b710998b": { 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "e0fc5415-c5c1-440e-b0d4-f065b710998b", 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.804+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2b80a2c00 msgr2=0x7fc2b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.804+0000 7fc2d3469640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2b80a2c00 0x7fc2b80a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009530 tx=0x7fc2c8009260 comp rx=0 tx=0).stop 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 msgr2=0x7fc2cc10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 0x7fc2cc10e350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9f0 tx=0x7fc2bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 shutdown_connections 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2b80a2c00 0x7fc2b80a50c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2cc102ef0 0x7fc2cc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102570 0x7fc2cc10e350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 --2- 172.21.15.143:0/2872599887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2cc100e20 0x7fc2cc10fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 >> 172.21.15.143:0/2872599887 conn(0x7fc2cc0fc7d0 msgr2=0x7fc2cc1048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 shutdown_connections 2024-09-10T10:48:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:48:35.805+0000 7fc2d3469640 1 -- 172.21.15.143:0/2872599887 wait complete. 2024-09-10T10:48:36.627 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:36 smithi143 ceph-mon[31057]: pgmap v457: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 260 op/s 2024-09-10T10:48:36.629 INFO:tasks.cephfs.filesystem:scrub status for tag:e0fc5415-c5c1-440e-b0d4-f065b710998b - {'path': '/', 'tag': 'e0fc5415-c5c1-440e-b0d4-f065b710998b', 'options': 'recursive,force'} 2024-09-10T10:48:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:36 smithi178 ceph-mon[34813]: pgmap v457: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 260 op/s 2024-09-10T10:48:36.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:36 smithi144 ceph-mon[34230]: pgmap v457: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 260 op/s 2024-09-10T10:48:37.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:37 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:37.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:37 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:38.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:37 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:48:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:38 smithi178 ceph-mon[34813]: pgmap v458: 129 pgs: 129 active+clean; 316 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 253 op/s 2024-09-10T10:48:38.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:38 smithi144 ceph-mon[34230]: pgmap v458: 129 pgs: 129 active+clean; 316 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 253 op/s 2024-09-10T10:48:38.951 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:38 smithi143 ceph-mon[31057]: pgmap v458: 129 pgs: 129 active+clean; 316 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 253 op/s 2024-09-10T10:48:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:39 smithi178 ceph-mon[34813]: pgmap v459: 129 pgs: 129 active+clean; 317 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 187 op/s 2024-09-10T10:48:39.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:39 smithi144 ceph-mon[34230]: pgmap v459: 129 pgs: 129 active+clean; 317 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 187 op/s 2024-09-10T10:48:39.964 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:39 smithi143 ceph-mon[31057]: pgmap v459: 129 pgs: 129 active+clean; 317 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 187 op/s 2024-09-10T10:48:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:42 smithi178 ceph-mon[34813]: pgmap v460: 129 pgs: 129 active+clean; 322 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 154 op/s 2024-09-10T10:48:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:42 smithi144 ceph-mon[34230]: pgmap v460: 129 pgs: 129 active+clean; 322 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 154 op/s 2024-09-10T10:48:43.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:42 smithi143 ceph-mon[31057]: pgmap v460: 129 pgs: 129 active+clean; 322 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 154 op/s 2024-09-10T10:48:43.591 INFO:tasks.workunit.client.0.smithi143.stdout: 70 11413 77 8709 71 5556 82 2024-09-10T10:48:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:44 smithi178 ceph-mon[34813]: pgmap v461: 129 pgs: 129 active+clean; 326 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-10T10:48:44.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:44 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:48:44.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:44 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:48:44.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:44 smithi144 ceph-mon[34230]: pgmap v461: 129 pgs: 129 active+clean; 326 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-10T10:48:44.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:44 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:48:44.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:44 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:48:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:44 smithi143 ceph-mon[31057]: pgmap v461: 129 pgs: 129 active+clean; 326 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-10T10:48:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:44 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:48:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:44 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:48:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:46 smithi178 ceph-mon[34813]: pgmap v462: 129 pgs: 129 active+clean; 327 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-09-10T10:48:46.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:46 smithi144 ceph-mon[34230]: pgmap v462: 129 pgs: 129 active+clean; 327 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-09-10T10:48:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:46 smithi143 ceph-mon[31057]: pgmap v462: 129 pgs: 129 active+clean; 327 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 96 op/s 2024-09-10T10:48:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:48 smithi178 ceph-mon[34813]: pgmap v463: 129 pgs: 129 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 95 op/s 2024-09-10T10:48:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:48 smithi144 ceph-mon[34230]: pgmap v463: 129 pgs: 129 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 95 op/s 2024-09-10T10:48:49.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:48 smithi143 ceph-mon[31057]: pgmap v463: 129 pgs: 129 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 95 op/s 2024-09-10T10:48:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:50 smithi178 ceph-mon[34813]: pgmap v464: 129 pgs: 129 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 71 op/s 2024-09-10T10:48:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:50 smithi144 ceph-mon[34230]: pgmap v464: 129 pgs: 129 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 71 op/s 2024-09-10T10:48:50.999 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:50 smithi143 ceph-mon[31057]: pgmap v464: 129 pgs: 129 active+clean; 337 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 71 op/s 2024-09-10T10:48:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:52 smithi178 ceph-mon[34813]: pgmap v465: 129 pgs: 129 active+clean; 347 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 73 op/s 2024-09-10T10:48:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:52 smithi144 ceph-mon[34230]: pgmap v465: 129 pgs: 129 active+clean; 347 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 73 op/s 2024-09-10T10:48:53.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:52 smithi143 ceph-mon[31057]: pgmap v465: 129 pgs: 129 active+clean; 347 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 73 op/s 2024-09-10T10:48:53.591 INFO:tasks.workunit.client.0.smithi143.stdout: 72 12477 59 9122 76 6406 65 2024-09-10T10:48:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:54 smithi178 ceph-mon[34813]: pgmap v466: 129 pgs: 129 active+clean; 347 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 69 op/s 2024-09-10T10:48:54.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:54 smithi144 ceph-mon[34230]: pgmap v466: 129 pgs: 129 active+clean; 347 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 69 op/s 2024-09-10T10:48:55.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:54 smithi143 ceph-mon[31057]: pgmap v466: 129 pgs: 129 active+clean; 347 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 69 op/s 2024-09-10T10:48:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:56 smithi178 ceph-mon[34813]: pgmap v467: 129 pgs: 129 active+clean; 352 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 68 op/s 2024-09-10T10:48:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:56 smithi144 ceph-mon[34230]: pgmap v467: 129 pgs: 129 active+clean; 352 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 68 op/s 2024-09-10T10:48:57.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:56 smithi143 ceph-mon[31057]: pgmap v467: 129 pgs: 129 active+clean; 352 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 68 op/s 2024-09-10T10:48:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:58 smithi178 ceph-mon[34813]: pgmap v468: 129 pgs: 129 active+clean; 355 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 76 op/s 2024-09-10T10:48:58.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:58 smithi144 ceph-mon[34230]: pgmap v468: 129 pgs: 129 active+clean; 355 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 76 op/s 2024-09-10T10:48:59.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:58 smithi143 ceph-mon[31057]: pgmap v468: 129 pgs: 129 active+clean; 355 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 76 op/s 2024-09-10T10:48:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:48:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:48:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:48:59.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:48:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:48:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:48:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:48:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:48:59.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:48:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:49:00.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:49:00.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:49:00.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:49:00.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:48:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:49:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:00 smithi178 ceph-mon[34813]: pgmap v469: 129 pgs: 129 active+clean; 361 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 64 op/s 2024-09-10T10:49:00.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:00 smithi144 ceph-mon[34230]: pgmap v469: 129 pgs: 129 active+clean; 361 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 64 op/s 2024-09-10T10:49:01.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:00 smithi143 ceph-mon[31057]: pgmap v469: 129 pgs: 129 active+clean; 361 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 64 op/s 2024-09-10T10:49:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:02 smithi178 ceph-mon[34813]: pgmap v470: 129 pgs: 129 active+clean; 368 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 68 op/s 2024-09-10T10:49:02.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:02 smithi144 ceph-mon[34230]: pgmap v470: 129 pgs: 129 active+clean; 368 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 68 op/s 2024-09-10T10:49:03.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:02 smithi143 ceph-mon[31057]: pgmap v470: 129 pgs: 129 active+clean; 368 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 68 op/s 2024-09-10T10:49:03.591 INFO:tasks.workunit.client.0.smithi143.stdout: 73 12423 62 8731 105 5170 48 2024-09-10T10:49:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:04 smithi178 ceph-mon[34813]: pgmap v471: 129 pgs: 129 active+clean; 371 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 64 op/s 2024-09-10T10:49:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:04 smithi144 ceph-mon[34230]: pgmap v471: 129 pgs: 129 active+clean; 371 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 64 op/s 2024-09-10T10:49:05.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:04 smithi143 ceph-mon[31057]: pgmap v471: 129 pgs: 129 active+clean; 371 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 64 op/s 2024-09-10T10:49:05.192 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T10:49:05.192 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T10:49:05.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:49:05.543 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 -- 172.21.15.143:0/411158967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 msgr2=0x7fa628110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/411158967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa628110ca0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa620009ca0 tx=0x7fa62002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 -- 172.21.15.143:0/411158967 shutdown_connections 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/411158967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa628110ca0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/411158967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6281049a0 0x7fa6281089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/411158967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa628103ff0 0x7fa6281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 -- 172.21.15.143:0/411158967 >> 172.21.15.143:0/411158967 conn(0x7fa6280ffab0 msgr2=0x7fa628101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 -- 172.21.15.143:0/411158967 shutdown_connections 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.127+0000 7fa62d5c2640 1 -- 172.21.15.143:0/411158967 wait complete. 2024-09-10T10:49:06.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.128+0000 7fa62d5c2640 1 Processor -- start 2024-09-10T10:49:06.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.128+0000 7fa62d5c2640 1 -- start start 2024-09-10T10:49:06.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa62d5c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 0x7fa62819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:06.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa62d5c2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6281049a0 0x7fa62819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:06.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa62d5c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa62819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:06.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa6267fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6281049a0 0x7fa62819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:06.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 0x7fa62819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:06.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 0x7fa62819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:60276/0 (socket says 172.21.15.143:60276) 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa6277fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa62819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa6277fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa62819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36642/0 (socket says 172.21.15.143:36642) 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa6277fe640 1 -- 172.21.15.143:0/3560869082 learned_addr learned my addr 172.21.15.143:0/3560869082 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113fe0 con 0x7fa628109120 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113e60 con 0x7fa628103ff0 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.129+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa628114160 con 0x7fa6281049a0 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa6267fc640 1 -- 172.21.15.143:0/3560869082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 msgr2=0x7fa62819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa6267fc640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 0x7fa62819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa6267fc640 1 -- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 msgr2=0x7fa62819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa6267fc640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa62819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa6267fc640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819f480 con 0x7fa6281049a0 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa626ffd640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 0x7fa62819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:06.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa6267fc640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6281049a0 0x7fa62819aa10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00d6d0 tx=0x7fa61c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:06.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.130+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c017070 con 0x7fa6281049a0 2024-09-10T10:49:06.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.131+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61c009820 con 0x7fa6281049a0 2024-09-10T10:49:06.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.131+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819f770 con 0x7fa6281049a0 2024-09-10T10:49:06.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.131+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004c40 con 0x7fa6281049a0 2024-09-10T10:49:06.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.131+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a7050 con 0x7fa6281049a0 2024-09-10T10:49:06.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.132+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6281043d0 con 0x7fa6281049a0 2024-09-10T10:49:06.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.133+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa61c0099d0 con 0x7fa6281049a0 2024-09-10T10:49:06.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.134+0000 7fa60ffff640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6100a2c00 0x7fa6100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:06.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.134+0000 7fa626ffd640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6100a2c00 0x7fa6100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:06.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.134+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa61c0cedd0 con 0x7fa6281049a0 2024-09-10T10:49:06.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.135+0000 7fa626ffd640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6100a2c00 0x7fa6100a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa618009fc0 tx=0x7fa618005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:06.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.139+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa61c09b9d0 con 0x7fa6281049a0 2024-09-10T10:49:06.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.491+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa62810bdd0 con 0x7fa6281049a0 2024-09-10T10:49:06.493 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:49:06.494 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:49:06.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.492+0000 7fa60ffff640 1 -- 172.21.15.143:0/3560869082 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7fa61c09b2d0 con 0x7fa6281049a0 2024-09-10T10:49:06.494 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:49:06.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6100a2c00 msgr2=0x7fa6100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6100a2c00 0x7fa6100a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa618009fc0 tx=0x7fa618005f90 comp rx=0 tx=0).stop 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6281049a0 msgr2=0x7fa62819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6281049a0 0x7fa62819aa10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00d6d0 tx=0x7fa61c00db90 comp rx=0 tx=0).stop 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 shutdown_connections 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6100a2c00 0x7fa6100a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa628109120 0x7fa62819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6281049a0 0x7fa62819aa10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 --2- 172.21.15.143:0/3560869082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa628103ff0 0x7fa62819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 >> 172.21.15.143:0/3560869082 conn(0x7fa6280ffab0 msgr2=0x7fa628100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:06.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 shutdown_connections 2024-09-10T10:49:06.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:06.495+0000 7fa62d5c2640 1 -- 172.21.15.143:0/3560869082 wait complete. 2024-09-10T10:49:06.632 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:49:06.664 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:06 smithi143 ceph-mon[31057]: pgmap v472: 129 pgs: 129 active+clean; 372 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 60 op/s 2024-09-10T10:49:06.664 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:06 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3560869082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:06 smithi178 ceph-mon[34813]: pgmap v472: 129 pgs: 129 active+clean; 372 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 60 op/s 2024-09-10T10:49:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:06 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3560869082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:06 smithi144 ceph-mon[34230]: pgmap v472: 129 pgs: 129 active+clean; 372 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 60 op/s 2024-09-10T10:49:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:06 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3560869082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:07.375 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T10:49:07.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T10:49:07.448 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 -- 172.21.15.143:0/22412566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c41008d0 msgr2=0x7f99c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/22412566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c41008d0 0x7f99c4100d30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009930 tx=0x7f99b002a420 comp rx=0 tx=0).stop 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 -- 172.21.15.143:0/22412566 shutdown_connections 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/22412566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c4101270 0x7f99c410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/22412566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c41008d0 0x7f99c4100d30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/22412566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c41068d0 0x7f99c4106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 -- 172.21.15.143:0/22412566 >> 172.21.15.143:0/22412566 conn(0x7f99c40fc610 msgr2=0x7f99c40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 -- 172.21.15.143:0/22412566 shutdown_connections 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.024+0000 7f99c9f9f640 1 -- 172.21.15.143:0/22412566 wait complete. 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 Processor -- start 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 -- start start 2024-09-10T10:49:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 0x7f99c4196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101270 0x7f99c4197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c41068d0 0x7f99c41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4110b80 con 0x7f99c4101270 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4110a00 con 0x7f99c41068d0 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.025+0000 7f99c9f9f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4110d00 con 0x7f99c41008d0 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101270 0x7f99c4197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 0x7f99c4196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 0x7f99c4196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36844/0 (socket says 172.21.15.143:36844) 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101270 0x7f99c4197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58330/0 (socket says 172.21.15.143:58330) 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 -- 172.21.15.143:0/2029723861 learned_addr learned my addr 172.21.15.143:0/2029723861 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c979e640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c41068d0 0x7f99c41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 -- 172.21.15.143:0/2029723861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c41068d0 msgr2=0x7f99c41978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c41068d0 0x7f99c41978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 -- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101270 msgr2=0x7f99c4197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101270 0x7f99c4197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 -- 172.21.15.143:0/2029723861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c41a6690 con 0x7f99c41008d0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 0x7f99c4196e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99b800d950 tx=0x7f99b800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c1ffb640 1 -- 172.21.15.143:0/2029723861 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8017070 con 0x7f99c41008d0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c1ffb640 1 -- 172.21.15.143:0/2029723861 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b8009c30 con 0x7f99c41008d0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.026+0000 7f99c1ffb640 1 -- 172.21.15.143:0/2029723861 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800f620 con 0x7f99c41008d0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.027+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c41a6980 con 0x7f99c41008d0 2024-09-10T10:49:08.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.027+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c41a6e40 con 0x7f99c41008d0 2024-09-10T10:49:08.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.030+0000 7f99c1ffb640 1 -- 172.21.15.143:0/2029723861 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f99b80109a0 con 0x7f99c41008d0 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 Processor -- start 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 -- start start 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c1ffb640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99b40a2cd0 0x7f99b40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a92653a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92660c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555a925b92e0 con 0x555a9265e850 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555a92560ce0 con 0x555a92653f90 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99cd5da740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a925b52a0 con 0x555a92653670 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c1ffb640 1 -- 172.21.15.143:0/2029723861 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99b80cee80 con 0x7f99c41008d0 2024-09-10T10:49:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42514/0 (socket says 172.21.15.143:42514) 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c3fff640 1 -- 172.21.15.143:0/1097599471 learned_addr learned my addr 172.21.15.143:0/1097599471 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a92653a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.031+0000 7f99c979e640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99b40a2cd0 0x7f99b40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c979e640 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92660c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c3fff640 1 -- 172.21.15.143:0/1097599471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 msgr2=0x555a92653a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c3fff640 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a92653a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c3fff640 1 -- 172.21.15.143:0/1097599471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 msgr2=0x555a92660c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c3fff640 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92660c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c3fff640 1 -- 172.21.15.143:0/1097599471 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a92545440 con 0x555a92653f90 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a92653a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c979e640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99b40a2cd0 0x7f99b40a5190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009270 tx=0x7f99bc009ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99c3fff640 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265e310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99b002a940 tx=0x7f99b0031ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a71ced921bfd69f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99aeffd640 1 -- 172.21.15.143:0/1097599471 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0040070 con 0x555a92653f90 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.032+0000 7f99aeffd640 1 -- 172.21.15.143:0/1097599471 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b0031e50 con 0x555a92653f90 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.033+0000 7f99aeffd640 1 -- 172.21.15.143:0/1097599471 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0035040 con 0x555a92653f90 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.033+0000 7f99cd5da740 1 -- 172.21.15.143:0/1097599471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 msgr2=0x555a9265e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.033+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265e310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99b002a940 tx=0x7f99b0031ba0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 -- 172.21.15.143:0/1097599471 shutdown_connections 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92660c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265e310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1097599471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a92653a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 -- 172.21.15.143:0/1097599471 >> 172.21.15.143:0/1097599471 conn(0x555a9264fc00 msgr2=0x555a9264ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 -- 172.21.15.143:0/1097599471 shutdown_connections 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 -- 172.21.15.143:0/1097599471 wait complete. 2024-09-10T10:49:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 -- start start 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.034+0000 7f99cd5da740 1 Processor -- start 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a9265e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a9265b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555a92661420 con 0x555a9265e850 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555a92574050 con 0x555a92653f90 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a926615a0 con 0x555a92653670 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99c979e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a9265b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99cd5da740 10 client.0 _ll_register_callbacks cb 0x555a925b4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99c8f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a9265e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99c8f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a9265e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36862/0 (socket says 172.21.15.143:36862) 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42516/0 (socket says 172.21.15.143:42516) 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.035+0000 7f99c8f9d640 1 -- 172.21.15.143:0/3240597392 learned_addr learned my addr 172.21.15.143:0/3240597392 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c8f9d640 1 -- 172.21.15.143:0/3240597392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 msgr2=0x555a9265b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c8f9d640 1 -- 172.21.15.143:0/3240597392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 msgr2=0x555a9265b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a9265b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c8f9d640 1 -- 172.21.15.143:0/3240597392 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a9232a3c0 con 0x555a92653670 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c979e640 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a9265b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a9265e170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99b80cc670 tx=0x7f99b8097690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d33213d3acb8ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.036+0000 7f99a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99a3fff640 1 -- 172.21.15.143:0/3240597392 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8020040 con 0x555a92653670 2024-09-10T10:49:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b0031e50 con 0x555a92653670 2024-09-10T10:49:08.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99a3fff640 1 -- 172.21.15.143:0/3240597392 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99b80979f0 con 0x555a92653670 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99a3fff640 1 -- 172.21.15.143:0/3240597392 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99b809b380 con 0x555a92653670 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99cd5da740 10 client.25753 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a9266cda0 con 0x555a92653670 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99a3fff640 1 -- 172.21.15.143:0/3240597392 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b0031e50 con 0x555a92653670 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.037+0000 7f99a3fff640 1 -- 172.21.15.143:0/3240597392 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f99b80b4120 con 0x555a92653670 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99cd5da740 10 client.25753 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:08.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99cd5da740 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99cd5da740 20 client.25753 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99cd5da740 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a9266e1b0 0x555a926705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99cd5da740 4 client.25753 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a925c6850 con 0x555a9266e1b0 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99c3fff640 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a9266e1b0 0x555a926705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99a3fff640 10 client.25753 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.038+0000 7f99c3fff640 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a9266e1b0 0x555a926705a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1efab22dbda56cba server_cookie=bef8612e4d89b405 in_seq=0 out_seq=0 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99a3fff640 1 -- 172.21.15.143:0/3240597392 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a925c6850 con 0x555a9266e1b0 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99a3fff640 10 client.25753 handle_command_reply: tid=0 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 client.25753 shutdown 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 msgr2=0x555a9265e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a9265e170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99b80cc670 tx=0x7f99b8097690 comp rx=0 tx=0).stop 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 shutdown_connections 2024-09-10T10:49:08.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a9265b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653f90 0x555a9265b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a9266e1b0 0x555a926705a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 --2- 172.21.15.143:0/3240597392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653670 0x555a9265e170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 >> 172.21.15.143:0/3240597392 conn(0x555a9264fc00 msgr2=0x555a926558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99a3fff640 0 client.25753 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 shutdown_connections 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.039+0000 7f99cd5da740 1 -- 172.21.15.143:0/3240597392 wait complete. 2024-09-10T10:49:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.040+0000 7f99cd5da740 20 client.25753 trim_cache size 0 max 16384 2024-09-10T10:49:08.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.070+0000 7f99cd5da740 1 Processor -- start 2024-09-10T10:49:08.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 -- start start 2024-09-10T10:49:08.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a9264ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a9265d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92675e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b0ae0 con 0x555a9265e850 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b2650 con 0x555a92653670 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99cd5da740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b11b0 con 0x555a92653f90 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a9265d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a9265d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36870/0 (socket says 172.21.15.143:36870) 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 -- 172.21.15.143:0/900398081 learned_addr learned my addr 172.21.15.143:0/900398081 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a9264ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c979e640 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92675e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 -- 172.21.15.143:0/900398081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 msgr2=0x555a9264ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a9264ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 -- 172.21.15.143:0/900398081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 msgr2=0x555a92675e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.071+0000 7f99c3fff640 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92675e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.072+0000 7f99c3fff640 1 -- 172.21.15.143:0/900398081 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a9265a3d0 con 0x555a92653f90 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.072+0000 7f99c3fff640 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a9265d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99b002a940 tx=0x7f99b0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a0cb475e80c064 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:08.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.072+0000 7f99ae7fc640 1 -- 172.21.15.143:0/900398081 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0040040 con 0x555a92653f90 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.072+0000 7f99ae7fc640 1 -- 172.21.15.143:0/900398081 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b00054f0 con 0x555a92653f90 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.072+0000 7f99ae7fc640 1 -- 172.21.15.143:0/900398081 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0035040 con 0x555a92653f90 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 -- 172.21.15.143:0/900398081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 msgr2=0x555a9265d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a9265d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f99b002a940 tx=0x7f99b0031400 comp rx=0 tx=0).stop 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 -- 172.21.15.143:0/900398081 shutdown_connections 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a92675e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a9265d390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 --2- 172.21.15.143:0/900398081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a9264ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 -- 172.21.15.143:0/900398081 >> 172.21.15.143:0/900398081 conn(0x555a9264fc00 msgr2=0x555a9265cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 -- 172.21.15.143:0/900398081 shutdown_connections 2024-09-10T10:49:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 -- 172.21.15.143:0/900398081 wait complete. 2024-09-10T10:49:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.073+0000 7f99cd5da740 1 -- start start 2024-09-10T10:49:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.074+0000 7f99cd5da740 1 Processor -- start 2024-09-10T10:49:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.074+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a92666e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.074+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a92664160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.074+0000 7f99cd5da740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a926646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a92664160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99cd5da740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b1f50 con 0x555a9265e850 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99cd5da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b1880 con 0x555a92653670 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99cd5da740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41aeac0 con 0x555a92653f90 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a92664160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36882/0 (socket says 172.21.15.143:36882) 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 -- 172.21.15.143:0/1163380815 learned_addr learned my addr 172.21.15.143:0/1163380815 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99cd5da740 10 client.0 _ll_register_callbacks cb 0x7f99b8096f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c979e640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a926646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a92666e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 -- 172.21.15.143:0/1163380815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 msgr2=0x555a92666e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a92666e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 -- 172.21.15.143:0/1163380815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 msgr2=0x555a926646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a926646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 -- 172.21.15.143:0/1163380815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a92653dc0 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99c3fff640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a92664160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99b00317a0 tx=0x7f99b00317d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66f42d4ab601f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.075+0000 7f99a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99a3fff640 1 -- 172.21.15.143:0/1163380815 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b005c020 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99a3fff640 1 -- 172.21.15.143:0/1163380815 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f99b0004770 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b00054f0 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99a3fff640 1 -- 172.21.15.143:0/1163380815 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99b0040070 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99cd5da740 10 client.25759 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a926e1340 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.076+0000 7f99a3fff640 1 -- 172.21.15.143:0/1163380815 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99b00054f0 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99a3fff640 1 -- 172.21.15.143:0/1163380815 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f99b003c7b0 con 0x555a92653f90 2024-09-10T10:49:08.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99cd5da740 10 client.25759 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99cd5da740 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99cd5da740 20 client.25759 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a926e1570 0x555a926e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99cd5da740 4 client.25759 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:49:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a925c0d60 con 0x555a926e1570 2024-09-10T10:49:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.077+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a926e1570 0x555a926e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99c8f9d640 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a926e1570 0x555a926e3960 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eef4b61e4c15ee6 server_cookie=7d3e23ae803a64a7 in_seq=0 out_seq=0 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99a3fff640 10 client.25759 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99a3fff640 1 -- 172.21.15.143:0/1163380815 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a925c0d60 con 0x555a926e1570 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99a3fff640 10 client.25759 handle_command_reply: tid=0 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 client.25759 shutdown 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 msgr2=0x555a92664160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a92664160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99b00317a0 tx=0x7f99b00317d0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 shutdown_connections 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555a9265e850 0x555a926646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x555a926e1570 0x555a926e3960 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99a3fff640 0 client.25759 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a92653f90 0x555a92664160 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99b00317a0 tx=0x7f99b00317d0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 --2- 172.21.15.143:0/1163380815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a92653670 0x555a92666e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.078+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 >> 172.21.15.143:0/1163380815 conn(0x555a9264fc00 msgr2=0x555a9264ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.079+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 shutdown_connections 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.079+0000 7f99cd5da740 1 -- 172.21.15.143:0/1163380815 wait complete. 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.079+0000 7f99cd5da740 20 client.25759 trim_cache size 0 max 16384 2024-09-10T10:49:08.083 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:08.084 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:49:08.084 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:49:08.084 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:08.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99b40a2cd0 msgr2=0x7f99b40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99b40a2cd0 0x7f99b40a5190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009270 tx=0x7f99bc009ef0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 msgr2=0x7f99c4196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 0x7f99c4196e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99b800d950 tx=0x7f99b800de10 comp rx=0 tx=0).stop 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 shutdown_connections 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f99b40a2cd0 0x7f99b40a5190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c41068d0 0x7f99c41978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c4101270 0x7f99c4197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 --2- 172.21.15.143:0/2029723861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c41008d0 0x7f99c4196e20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 >> 172.21.15.143:0/2029723861 conn(0x7f99c40fc610 msgr2=0x7f99c40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 shutdown_connections 2024-09-10T10:49:08.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:08.080+0000 7f99c9f9f640 1 -- 172.21.15.143:0/2029723861 wait complete. 2024-09-10T10:49:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:08.912 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:08 smithi143 ceph-mon[31057]: pgmap v473: 129 pgs: 129 active+clean; 385 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-09-10T10:49:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:08 smithi178 ceph-mon[34813]: pgmap v473: 129 pgs: 129 active+clean; 385 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-09-10T10:49:08.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:08 smithi144 ceph-mon[34230]: pgmap v473: 129 pgs: 129 active+clean; 385 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 64 op/s 2024-09-10T10:49:08.949 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:49:08.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 -- 172.21.15.143:0/153079725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010c160 msgr2=0x7f8cf010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/153079725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010c160 0x7f8cf010e600 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009ca0 tx=0x7f8ce802a950 comp rx=0 tx=0).stop 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 -- 172.21.15.143:0/153079725 shutdown_connections 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/153079725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf010c160 0x7f8cf010e600 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/153079725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf0103650 0x7f8cf010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/153079725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0102d30 0x7f8cf0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 -- 172.21.15.143:0/153079725 >> 172.21.15.143:0/153079725 conn(0x7f8cf00fc790 msgr2=0x7f8cf00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 -- 172.21.15.143:0/153079725 shutdown_connections 2024-09-10T10:49:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.533+0000 7f8cf70c3640 1 -- 172.21.15.143:0/153079725 wait complete. 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 Processor -- start 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 -- start start 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 0x7f8cf0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0103650 0x7f8cf0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf010c160 0x7f8cf0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0111e10 con 0x7f8cf0102d30 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0111c90 con 0x7f8cf010c160 2024-09-10T10:49:09.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf70c3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0111f90 con 0x7f8cf0103650 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.534+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 0x7f8cf0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 0x7f8cf0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58368/0 (socket says 172.21.15.143:58368) 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/755221845 learned_addr learned my addr 172.21.15.143:0/755221845 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8ceffff640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0103650 0x7f8cf0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/755221845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0103650 msgr2=0x7f8cf0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf5660640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf010c160 0x7f8cf0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0103650 0x7f8cf0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/755221845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf010c160 msgr2=0x7f8cf0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf010c160 0x7f8cf0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/755221845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01aaae0 con 0x7f8cf0102d30 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf5660640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf010c160 0x7f8cf0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8ceffff640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0103650 0x7f8cf0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:09.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 0x7f8cf0100990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400e9f0 tx=0x7f8ce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:09.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cedffb640 1 -- 172.21.15.143:0/755221845 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400cd80 con 0x7f8cf0102d30 2024-09-10T10:49:09.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01aad10 con 0x7f8cf0102d30 2024-09-10T10:49:09.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cedffb640 1 -- 172.21.15.143:0/755221845 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce40108e0 con 0x7f8cf0102d30 2024-09-10T10:49:09.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.535+0000 7f8cedffb640 1 -- 172.21.15.143:0/755221845 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4018920 con 0x7f8cf0102d30 2024-09-10T10:49:09.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.536+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01ab1a0 con 0x7f8cf0102d30 2024-09-10T10:49:09.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cedffb640 1 -- 172.21.15.143:0/755221845 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8ce4010a90 con 0x7f8cf0102d30 2024-09-10T10:49:09.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 Processor -- start 2024-09-10T10:49:09.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 -- start start 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d09850 0x558330d09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 0x558330d144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d16e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558330c6e160 con 0x558330d0a170 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558330c15720 con 0x558330d14a30 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf949c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558330c6a120 con 0x558330d09850 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 0x558330d144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 0x558330d144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58382/0 (socket says 172.21.15.143:58382) 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.539+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3340198704 learned_addr learned my addr 172.21.15.143:0/3340198704 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8ceffff640 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d16e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3340198704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d09850 msgr2=0x558330d09c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:49:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d09850 0x558330d09c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3340198704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 msgr2=0x558330d16e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cedffb640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8ce00a2b80 0x7f8ce00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d16e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3340198704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583309dfba0 con 0x558330d0a170 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 0x558330d144f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40a7370 tx=0x7f8ce4098030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cf6051603c9aab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.540+0000 7f8cedffb640 1 -- 172.21.15.143:0/755221845 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8ce40cf7d0 con 0x7f8cf0102d30 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.541+0000 7f8cdaffd640 1 -- 172.21.15.143:0/3340198704 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce409b810 con 0x558330d0a170 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.541+0000 7f8cdaffd640 1 -- 172.21.15.143:0/3340198704 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce40a75a0 con 0x558330d0a170 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.541+0000 7f8cdaffd640 1 -- 172.21.15.143:0/3340198704 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40a73a0 con 0x558330d0a170 2024-09-10T10:49:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.541+0000 7f8cf5660640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8ce00a2b80 0x7f8ce00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.541+0000 7f8cf949c740 1 -- 172.21.15.143:0/3340198704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 msgr2=0x558330d144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.541+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 0x558330d144f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40a7370 tx=0x7f8ce4098030 comp rx=0 tx=0).stop 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf5660640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8ce00a2b80 0x7f8ce00a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8001190 tx=0x7f8ce8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 -- 172.21.15.143:0/3340198704 shutdown_connections 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d16e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d0a170 0x558330d144f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3340198704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d09850 0x558330d09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 -- 172.21.15.143:0/3340198704 >> 172.21.15.143:0/3340198704 conn(0x558330d05de0 msgr2=0x558330d061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 -- 172.21.15.143:0/3340198704 shutdown_connections 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 -- 172.21.15.143:0/3340198704 wait complete. 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 -- start start 2024-09-10T10:49:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.542+0000 7f8cf949c740 1 Processor -- start 2024-09-10T10:49:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 0x558330d119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d11f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d12450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558330d175a0 con 0x558330d09850 2024-09-10T10:49:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558330bf6900 con 0x558330d14a30 2024-09-10T10:49:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558330d17720 con 0x558330d0a170 2024-09-10T10:49:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 0x558330d119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf949c740 10 client.0 _ll_register_callbacks cb 0x558330bf4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 0x558330d119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58394/0 (socket says 172.21.15.143:58394) 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3954950339 learned_addr learned my addr 172.21.15.143:0/3954950339 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8ceffff640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d11f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf5660640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d12450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 msgr2=0x558330d11f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d11f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3954950339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 msgr2=0x558330d12450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d12450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.543+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/3954950339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558330c36c50 con 0x558330d09850 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 0x558330d119d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40a4410 tx=0x7f8ce4009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b5245d8bf9bc17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8ccffff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8ccffff640 1 -- 172.21.15.143:0/3954950339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40a7d10 con 0x558330d09850 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558330c7b890 con 0x558330d09850 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8ccffff640 1 -- 172.21.15.143:0/3954950339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8ce40d5730 con 0x558330d09850 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8ccffff640 1 -- 172.21.15.143:0/3954950339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ce409b630 con 0x558330d09850 2024-09-10T10:49:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8cf949c740 10 client.16359 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558330d175a0 con 0x558330d09850 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8ccffff640 1 -- 172.21.15.143:0/3954950339 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558330c7b890 con 0x558330d09850 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.544+0000 7f8ccffff640 1 -- 172.21.15.143:0/3954950339 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f8ce40a7140 con 0x558330d09850 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8cf949c740 10 client.16359 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8cf949c740 10 client.16359 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8cf949c740 20 client.16359 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8cf949c740 4 client.16359 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558330c75be0 con 0x558330d246c0 2024-09-10T10:49:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.545+0000 7f8ceffff640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8ccffff640 10 client.16359 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8ceffff640 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d26b10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bf7f175f64799f server_cookie=ebd07fbddba6eaf2 in_seq=0 out_seq=0 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8ccffff640 1 -- 172.21.15.143:0/3954950339 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558330c75be0 con 0x558330d246c0 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8ccffff640 10 client.16359 handle_command_reply: tid=0 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8cf949c740 1 client.16359 shutdown 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 msgr2=0x558330d119d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.546+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 0x558330d119d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40a4410 tx=0x7f8ce4009770 comp rx=0 tx=0).stop 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 shutdown_connections 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d14a30 0x558330d12450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d26b10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d11f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8ccffff640 0 client.16359 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 --2- 172.21.15.143:0/3954950339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d09850 0x558330d119d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 >> 172.21.15.143:0/3954950339 conn(0x558330d05de0 msgr2=0x558330d0bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 shutdown_connections 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 1 -- 172.21.15.143:0/3954950339 wait complete. 2024-09-10T10:49:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.547+0000 7f8cf949c740 20 client.16359 trim_cache size 0 max 16384 2024-09-10T10:49:09.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.571+0000 7f8cf949c740 1 Processor -- start 2024-09-10T10:49:09.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.571+0000 7f8cf949c740 1 -- start start 2024-09-10T10:49:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d00f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d2c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d2c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf949c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b52e0 con 0x558330d14a30 2024-09-10T10:49:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf949c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b6e50 con 0x558330d09850 2024-09-10T10:49:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf949c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b59b0 con 0x558330d0a170 2024-09-10T10:49:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d2c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d00f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d2c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58404/0 (socket says 172.21.15.143:58404) 2024-09-10T10:49:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 -- 172.21.15.143:0/1316515779 learned_addr learned my addr 172.21.15.143:0/1316515779 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8ceffff640 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d2c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 -- 172.21.15.143:0/1316515779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 msgr2=0x558330d2c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d2c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 -- 172.21.15.143:0/1316515779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 msgr2=0x558330d00f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d00f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.572+0000 7f8cf5660640 1 -- 172.21.15.143:0/1316515779 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558330d09fa0 con 0x558330d14a30 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.573+0000 7f8ceffff640 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d2c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.573+0000 7f8cf5660640 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d2c920 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009410 tx=0x7f8ce803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808efa8a538b594d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.573+0000 7f8cda7fc640 1 -- 172.21.15.143:0/1316515779 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8034070 con 0x558330d14a30 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.573+0000 7f8cda7fc640 1 -- 172.21.15.143:0/1316515779 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce8003e20 con 0x558330d14a30 2024-09-10T10:49:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.573+0000 7f8cda7fc640 1 -- 172.21.15.143:0/1316515779 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8030440 con 0x558330d14a30 2024-09-10T10:49:09.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 -- 172.21.15.143:0/1316515779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 msgr2=0x558330d2c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:09.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d2c920 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009410 tx=0x7f8ce803d000 comp rx=0 tx=0).stop 2024-09-10T10:49:09.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 -- 172.21.15.143:0/1316515779 shutdown_connections 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d2c920 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d2c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 --2- 172.21.15.143:0/1316515779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d00f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 -- 172.21.15.143:0/1316515779 >> 172.21.15.143:0/1316515779 conn(0x558330d05de0 msgr2=0x558330d0ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 -- 172.21.15.143:0/1316515779 shutdown_connections 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.574+0000 7f8cf949c740 1 -- 172.21.15.143:0/1316515779 wait complete. 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.575+0000 7f8cf949c740 1 -- start start 2024-09-10T10:49:09.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.575+0000 7f8cf949c740 1 Processor -- start 2024-09-10T10:49:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d1cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d18100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d18640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b6750 con 0x558330d14a30 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b6080 con 0x558330d09850 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01b32c0 con 0x558330d0a170 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d1cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf4e5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d1cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42574/0 (socket says 172.21.15.143:42574) 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d18100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36936/0 (socket says 172.21.15.143:36936) 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf4e5f640 1 -- 172.21.15.143:0/2312631386 learned_addr learned my addr 172.21.15.143:0/2312631386 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf5660640 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d18640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8cf949c740 10 client.0 _ll_register_callbacks cb 0x7f8cf006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 -- 172.21.15.143:0/2312631386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 msgr2=0x558330d1cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d1cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 -- 172.21.15.143:0/2312631386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 msgr2=0x558330d18640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d18640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.576+0000 7f8ceffff640 1 -- 172.21.15.143:0/2312631386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558330d18b80 con 0x558330d0a170 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8ceffff640 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d18100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x558330d07160 tx=0x7f8cdc00bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634db5eff892157b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8ccffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8ccffff640 1 -- 172.21.15.143:0/2312631386 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc01d620 con 0x558330d0a170 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ce8003e20 con 0x558330d0a170 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8ccffff640 1 -- 172.21.15.143:0/2312631386 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8cdc01d7c0 con 0x558330d0a170 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8ccffff640 1 -- 172.21.15.143:0/2312631386 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8cdc02e050 con 0x558330d0a170 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8cf949c740 10 client.25771 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.577+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558330d8efe0 con 0x558330d0a170 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8ccffff640 1 -- 172.21.15.143:0/2312631386 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce8003e20 con 0x558330d0a170 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8ccffff640 1 -- 172.21.15.143:0/2312631386 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f8cdc011a00 con 0x558330d0a170 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf949c740 10 client.25771 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf949c740 10 client.25771 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf949c740 20 client.25771 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf949c740 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf949c740 4 client.25771 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T10:49:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x558330bf1f40 con 0x558330d246c0 2024-09-10T10:49:09.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.578+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:09.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.579+0000 7f8ccffff640 10 client.25771 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:09.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:09.579+0000 7f8cf4e5f640 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d91860 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d48f1abcac13f69 server_cookie=9082e90d431ba62e in_seq=0 out_seq=0 2024-09-10T10:49:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:10.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 -- 172.21.15.143:0/3507194318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 msgr2=0x7f2920100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:10.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 --2- 172.21.15.143:0/3507194318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f2920100ed0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2910009930 tx=0x7f291002a420 comp rx=0 tx=0).stop 2024-09-10T10:49:10.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 -- 172.21.15.143:0/3507194318 shutdown_connections 2024-09-10T10:49:10.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 --2- 172.21.15.143:0/3507194318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 0x7f292010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 --2- 172.21.15.143:0/3507194318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f2920100ed0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 --2- 172.21.15.143:0/3507194318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 0x7f2920106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.113+0000 7f29287f0640 1 -- 172.21.15.143:0/3507194318 >> 172.21.15.143:0/3507194318 conn(0x7f29200fc790 msgr2=0x7f29200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:10.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.114+0000 7f29287f0640 1 -- 172.21.15.143:0/3507194318 shutdown_connections 2024-09-10T10:49:10.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.114+0000 7f29287f0640 1 -- 172.21.15.143:0/3507194318 wait complete. 2024-09-10T10:49:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.114+0000 7f29287f0640 1 Processor -- start 2024-09-10T10:49:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.114+0000 7f29287f0640 1 -- start start 2024-09-10T10:49:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f29287f0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f29200fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f29287f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 0x7f29200fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f29287f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 0x7f29200fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f29287f0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2920110d20 con 0x7f2920100a70 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f29287f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2920110ba0 con 0x7f2920101410 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f29287f0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2920110ea0 con 0x7f2920106a70 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f29200fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f29200fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58428/0 (socket says 172.21.15.143:58428) 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f2925d8b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 0x7f29200fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 -- 172.21.15.143:0/2928614864 learned_addr learned my addr 172.21.15.143:0/2928614864 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f2926d8d640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 0x7f29200fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 -- 172.21.15.143:0/2928614864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 msgr2=0x7f29200fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 0x7f29200fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 -- 172.21.15.143:0/2928614864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 msgr2=0x7f29200fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 0x7f29200fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 -- 172.21.15.143:0/2928614864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29201a6860 con 0x7f2920100a70 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f2926d8d640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 0x7f29200fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f2925d8b640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 0x7f29200fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.115+0000 7f292658c640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f29200fcef0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f291c00e9f0 tx=0x7f291c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.116+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c00cd80 con 0x7f2920100a70 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.116+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f291c0054d0 con 0x7f2920100a70 2024-09-10T10:49:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.116+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c010430 con 0x7f2920100a70 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.116+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29201a6b50 con 0x7f2920100a70 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.116+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a6fb0 con 0x7f2920100a70 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.117+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920100ed0 con 0x7f2920100a70 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.118+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f291c005020 con 0x7f2920100a70 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.119+0000 7f29177fe640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f290c0a2b80 0x7f290c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.119+0000 7f2925d8b640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f290c0a2b80 0x7f290c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.119+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f291c0cf700 con 0x7f2920100a70 2024-09-10T10:49:10.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.120+0000 7f2925d8b640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f290c0a2b80 0x7f290c0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2910009900 tx=0x7f2910000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:10.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.123+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f291c09c300 con 0x7f2920100a70 2024-09-10T10:49:10.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.380+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2920109a60 con 0x7f2920100a70 2024-09-10T10:49:10.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.381+0000 7f29177fe640 1 -- 172.21.15.143:0/2928614864 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7f291c09bc00 con 0x7f2920100a70 2024-09-10T10:49:10.382 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:49:10.382 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:49:10.382 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:49:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f290c0a2b80 msgr2=0x7f290c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f290c0a2b80 0x7f290c0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2910009900 tx=0x7f2910000bc0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 msgr2=0x7f29200fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f29200fcef0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f291c00e9f0 tx=0x7f291c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 shutdown_connections 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f290c0a2b80 0x7f290c0a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920106a70 0x7f29200fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920101410 0x7f29200fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 --2- 172.21.15.143:0/2928614864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2920100a70 0x7f29200fcef0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 >> 172.21.15.143:0/2928614864 conn(0x7f29200fc790 msgr2=0x7f292006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 shutdown_connections 2024-09-10T10:49:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:10.383+0000 7f29287f0640 1 -- 172.21.15.143:0/2928614864 wait complete. 2024-09-10T10:49:10.775 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:10 smithi143 ceph-mon[31057]: pgmap v474: 129 pgs: 129 active+clean; 390 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 61 op/s 2024-09-10T10:49:10.775 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:10 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2928614864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:10 smithi178 ceph-mon[34813]: pgmap v474: 129 pgs: 129 active+clean; 390 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 61 op/s 2024-09-10T10:49:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:10 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2928614864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:10 smithi144 ceph-mon[34230]: pgmap v474: 129 pgs: 129 active+clean; 390 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 61 op/s 2024-09-10T10:49:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:10 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2928614864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:11.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:49:11.517 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:12.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 -- 172.21.15.143:0/3833856715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c102ef0 msgr2=0x7f615c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 --2- 172.21.15.143:0/3833856715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c102ef0 0x7f615c10da40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6154009ca0 tx=0x7f615402a950 comp rx=0 tx=0).stop 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 -- 172.21.15.143:0/3833856715 shutdown_connections 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 --2- 172.21.15.143:0/3833856715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c102ef0 0x7f615c10da40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 --2- 172.21.15.143:0/3833856715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c102570 0x7f615c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 --2- 172.21.15.143:0/3833856715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c100e20 0x7f615c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 -- 172.21.15.143:0/3833856715 >> 172.21.15.143:0/3833856715 conn(0x7f615c0fc7d0 msgr2=0x7f615c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 -- 172.21.15.143:0/3833856715 shutdown_connections 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.116+0000 7f6160808640 1 -- 172.21.15.143:0/3833856715 wait complete. 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.117+0000 7f6160808640 1 Processor -- start 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.117+0000 7f6160808640 1 -- start start 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6160808640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 0x7f615c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6160808640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c102570 0x7f615c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6160808640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c102ef0 0x7f615c1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6160808640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f615c110e30 con 0x7f615c100e20 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6160808640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f615c110cb0 con 0x7f615c102570 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6160808640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f615c110fb0 con 0x7f615c102ef0 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 0x7f615c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 0x7f615c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58442/0 (socket says 172.21.15.143:58442) 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 -- 172.21.15.143:0/638836719 learned_addr learned my addr 172.21.15.143:0/638836719 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615ad9d640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c102ef0 0x7f615c1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f6159d9b640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c102570 0x7f615c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 -- 172.21.15.143:0/638836719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c102ef0 msgr2=0x7f615c1a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c102ef0 0x7f615c1a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 -- 172.21.15.143:0/638836719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c102570 msgr2=0x7f615c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c102570 0x7f615c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 -- 172.21.15.143:0/638836719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c1a4840 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615ad9d640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c102ef0 0x7f615c1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.118+0000 7f615a59c640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 0x7f615c199280 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f614c00e9f0 tx=0x7f614c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.119+0000 7f614b7fe640 1 -- 172.21.15.143:0/638836719 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c00cd80 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.119+0000 7f614b7fe640 1 -- 172.21.15.143:0/638836719 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f614c0054d0 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.119+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c1a4a10 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.119+0000 7f614b7fe640 1 -- 172.21.15.143:0/638836719 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c010490 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.119+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c1a4f00 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.121+0000 7f614b7fe640 1 -- 172.21.15.143:0/638836719 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f614c005020 con 0x7f615c100e20 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 Processor -- start 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 -- start start 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dcf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dcfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 0x55d0e2ddc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2d352f0 con 0x55d0e2dda370 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2cdccf0 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6162be1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2d312b0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f614b7fe640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f61440a2b80 0x7f61440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f615a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dcfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6159d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 0x55d0e2ddc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f615ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dcf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f615ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dcf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36962/0 (socket says 172.21.15.143:36962) 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f615a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dcfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42612/0 (socket says 172.21.15.143:42612) 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f6159d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 0x55d0e2ddc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58448/0 (socket says 172.21.15.143:58448) 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.122+0000 7f615ad9d640 1 -- 172.21.15.143:0/936325421 learned_addr learned my addr 172.21.15.143:0/936325421 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:12.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f61440a2b80 0x7f61440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f614b7fe640 1 -- 172.21.15.143:0/638836719 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f614c0cf730 con 0x7f615c100e20 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 -- 172.21.15.143:0/936325421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dcf5d0 msgr2=0x55d0e2dcfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dcfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 -- 172.21.15.143:0/936325421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 msgr2=0x55d0e2ddc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 0x55d0e2ddc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 -- 172.21.15.143:0/936325421 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e2cc13a0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f6159d9b640 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 0x55d0e2ddc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f615ad9d640 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dcf090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f615402fd10 tx=0x7f615402f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41a0f8bc5809cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f6148ff9640 1 -- 172.21.15.143:0/936325421 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154043050 con 0x55d0e2dd0b40 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f6148ff9640 1 -- 172.21.15.143:0/936325421 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6154044040 con 0x55d0e2dd0b40 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.123+0000 7f6148ff9640 1 -- 172.21.15.143:0/936325421 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154055070 con 0x55d0e2dd0b40 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f615ad9d640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f61440a2b80 0x7f61440a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6154052000 tx=0x7f615404e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 -- 172.21.15.143:0/936325421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 msgr2=0x55d0e2dcf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dcf090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f615402fd10 tx=0x7f615402f5c0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 -- 172.21.15.143:0/936325421 shutdown_connections 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dda370 0x55d0e2ddc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dcfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 --2- 172.21.15.143:0/936325421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dcf090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 -- 172.21.15.143:0/936325421 >> 172.21.15.143:0/936325421 conn(0x55d0e2d98310 msgr2=0x55d0e2dcbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 -- 172.21.15.143:0/936325421 shutdown_connections 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 -- 172.21.15.143:0/936325421 wait complete. 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 -- start start 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.124+0000 7f6162be1740 1 Processor -- start 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dd9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dd6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dda370 0x55d0e2dd7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2ddd070 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2cf0060 con 0x55d0e2dd0b40 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d0e2ddd1f0 con 0x55d0e2dda370 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 10 client.0 _ll_register_callbacks cb 0x55d0e2a438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dd6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dd9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dd6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42620/0 (socket says 172.21.15.143:42620) 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dd9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58452/0 (socket says 172.21.15.143:58452) 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 -- 172.21.15.143:0/872340020 learned_addr learned my addr 172.21.15.143:0/872340020 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f615ad9d640 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dda370 0x55d0e2dd7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 -- 172.21.15.143:0/872340020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dda370 msgr2=0x55d0e2dd7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dda370 0x55d0e2dd7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 -- 172.21.15.143:0/872340020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 msgr2=0x55d0e2dd9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dd9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 -- 172.21.15.143:0/872340020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e2aa63c0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6159d9b640 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dd6fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6150012700 tx=0x7f6150012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba6d4153cc36e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6141ffb640 1 -- 172.21.15.143:0/872340020 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000f840 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.125+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6154044040 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6141ffb640 1 -- 172.21.15.143:0/872340020 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f615000f9e0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6141ffb640 1 -- 172.21.15.143:0/872340020 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6150025050 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 10 client.25760 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0e2de8ba0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6141ffb640 1 -- 172.21.15.143:0/872340020 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6154044040 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6141ffb640 1 -- 172.21.15.143:0/872340020 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f6150010430 con 0x55d0e2dd0b40 2024-09-10T10:49:12.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 10 client.25760 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 10 client.25760 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 20 client.25760 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2dea020 0x55d0e2dec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 4 client.25760 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e2d42860 con 0x55d0e2dea020 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.126+0000 7f615a59c640 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2dea020 0x55d0e2dec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.127+0000 7f6141ffb640 10 client.25760 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.127+0000 7f615a59c640 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2dea020 0x55d0e2dec460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8db47b4a51451fef server_cookie=ffb610b915339a8e in_seq=0 out_seq=0 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.127+0000 7f6141ffb640 1 -- 172.21.15.143:0/872340020 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0e2d42860 con 0x55d0e2dea020 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.127+0000 7f6141ffb640 10 client.25760 handle_command_reply: tid=0 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 client.25760 shutdown 2024-09-10T10:49:12.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 msgr2=0x55d0e2dd6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dd6fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6150012700 tx=0x7f6150012bc0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 shutdown_connections 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2dea020 0x55d0e2dec460 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dda370 0x55d0e2dd7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dd0b40 0x55d0e2dd6fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 --2- 172.21.15.143:0/872340020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2dd9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 >> 172.21.15.143:0/872340020 conn(0x55d0e2d98310 msgr2=0x55d0e2dd1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6141ffb640 0 client.25760 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 shutdown_connections 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 1 -- 172.21.15.143:0/872340020 wait complete. 2024-09-10T10:49:12.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.128+0000 7f6162be1740 20 client.25760 trim_cache size 0 max 16384 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 Processor -- start 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 -- start start 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2df1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2df22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1ac970 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1ae4e0 con 0x55d0e2dda370 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6162be1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1ad040 con 0x55d0e2dd0b40 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58466/0 (socket says 172.21.15.143:58466) 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f615a59c640 1 -- 172.21.15.143:0/2833966545 learned_addr learned my addr 172.21.15.143:0/2833966545 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f6159d9b640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2df1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.142+0000 7f615ad9d640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2df22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615a59c640 1 -- 172.21.15.143:0/2833966545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 msgr2=0x55d0e2df1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615a59c640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2df1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615a59c640 1 -- 172.21.15.143:0/2833966545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 msgr2=0x55d0e2df22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615a59c640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2df22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615a59c640 1 -- 172.21.15.143:0/2833966545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e2dd78e0 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f6159d9b640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2df1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:12.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615ad9d640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2df22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f615a59c640 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de6340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f614c0cd240 tx=0x7f614c09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dbd2909c4c8b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f6143fff640 1 -- 172.21.15.143:0/2833966545 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c005680 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f6143fff640 1 -- 172.21.15.143:0/2833966545 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f614c09c330 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.143+0000 7f6143fff640 1 -- 172.21.15.143:0/2833966545 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c099fc0 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 -- 172.21.15.143:0/2833966545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 msgr2=0x55d0e2de6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de6340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f614c0cd240 tx=0x7f614c09f040 comp rx=0 tx=0).stop 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 -- 172.21.15.143:0/2833966545 shutdown_connections 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2df22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2df1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 --2- 172.21.15.143:0/2833966545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de6340 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 -- 172.21.15.143:0/2833966545 >> 172.21.15.143:0/2833966545 conn(0x55d0e2d98310 msgr2=0x55d0e2dd60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 -- 172.21.15.143:0/2833966545 shutdown_connections 2024-09-10T10:49:12.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.144+0000 7f6162be1740 1 -- 172.21.15.143:0/2833966545 wait complete. 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 -- start start 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 Processor -- start 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2de29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2ddf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1adde0 con 0x55d0e2dcf5d0 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1ad710 con 0x55d0e2dda370 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1aa950 con 0x55d0e2dd0b40 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6162be1740 10 client.0 _ll_register_callbacks cb 0x7f6150009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6159d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2de29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f615a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58480/0 (socket says 172.21.15.143:58480) 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f615a59c640 1 -- 172.21.15.143:0/3846225244 learned_addr learned my addr 172.21.15.143:0/3846225244 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f6159d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2de29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:36986/0 (socket says 172.21.15.143:36986) 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.145+0000 7f615ad9d640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2ddf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6159d9b640 1 -- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 msgr2=0x55d0e2ddf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6159d9b640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2ddf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6159d9b640 1 -- 172.21.15.143:0/3846225244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 msgr2=0x55d0e2de24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6159d9b640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6159d9b640 1 -- 172.21.15.143:0/3846225244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e2ddf9c0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f615a59c640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6159d9b640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2de29f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6150010800 tx=0x7f6150009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89f493280835e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6141ffb640 1 -- 172.21.15.143:0/3846225244 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500050b0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6141ffb640 1 -- 172.21.15.143:0/3846225244 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6150018030 con 0x55d0e2dd0b40 2024-09-10T10:49:12.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614c09c330 con 0x55d0e2dd0b40 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6141ffb640 1 -- 172.21.15.143:0/3846225244 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6150005250 con 0x55d0e2dd0b40 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6162be1740 10 client.25795 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0e2e55500 con 0x55d0e2dd0b40 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.146+0000 7f6141ffb640 1 -- 172.21.15.143:0/3846225244 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f614c09c330 con 0x55d0e2dd0b40 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6141ffb640 1 -- 172.21.15.143:0/3846225244 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f61500035f0 con 0x55d0e2dd0b40 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6162be1740 10 client.25795 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6162be1740 10 client.25795 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6162be1740 20 client.25795 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6162be1740 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2e55c70 0x55d0e2e58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6162be1740 4 client.25795 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0e2d3cd70 con 0x55d0e2e55c70 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f615a59c640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2e55c70 0x55d0e2e58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6141ffb640 10 client.25795 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f615a59c640 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2e55c70 0x55d0e2e58090 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3935d21b94138c6e server_cookie=2e55021064a1bd18 in_seq=0 out_seq=0 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6141ffb640 1 -- 172.21.15.143:0/3846225244 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0e2d3cd70 con 0x55d0e2e55c70 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.147+0000 7f6141ffb640 10 client.25795 handle_command_reply: tid=0 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 client.25795 shutdown 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 msgr2=0x55d0e2de29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2de29f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6150010800 tx=0x7f6150009fb0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 shutdown_connections 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55d0e2e55c70 0x55d0e2e58090 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0e2dda370 0x55d0e2ddf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6141ffb640 0 client.25795 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e2dd0b40 0x55d0e2de29f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 --2- 172.21.15.143:0/3846225244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d0e2dcf5d0 0x55d0e2de24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 >> 172.21.15.143:0/3846225244 conn(0x55d0e2d98310 msgr2=0x55d0e2dca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 shutdown_connections 2024-09-10T10:49:12.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 1 -- 172.21.15.143:0/3846225244 wait complete. 2024-09-10T10:49:12.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.148+0000 7f6162be1740 20 client.25795 trim_cache size 0 max 16384 2024-09-10T10:49:12.162 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f61440a2b80 msgr2=0x7f61440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f61440a2b80 0x7f61440a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6154052000 tx=0x7f615404e000 comp rx=0 tx=0).stop 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 msgr2=0x7f615c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 0x7f615c199280 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f614c00e9f0 tx=0x7f614c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 shutdown_connections 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f61440a2b80 0x7f61440a5040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f615c102ef0 0x7f615c1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c102570 0x7f615c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 --2- 172.21.15.143:0/638836719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f615c100e20 0x7f615c199280 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 >> 172.21.15.143:0/638836719 conn(0x7f615c0fc7d0 msgr2=0x7f615c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 shutdown_connections 2024-09-10T10:49:12.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:12.149+0000 7f6160808640 1 -- 172.21.15.143:0/638836719 wait complete. 2024-09-10T10:49:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:12 smithi178 ceph-mon[34813]: pgmap v475: 129 pgs: 129 active+clean; 392 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 59 op/s 2024-09-10T10:49:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:12 smithi144 ceph-mon[34230]: pgmap v475: 129 pgs: 129 active+clean; 392 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 59 op/s 2024-09-10T10:49:12.963 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:12 smithi143 ceph-mon[31057]: pgmap v475: 129 pgs: 129 active+clean; 392 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 59 op/s 2024-09-10T10:49:12.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:49:13.244 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:13.591 INFO:tasks.workunit.client.0.smithi143.stdout: 75 6301 50 4548 87 3676 33 2024-09-10T10:49:13.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.731+0000 7f96c604d640 1 -- 172.21.15.143:0/3911788998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0101de0 msgr2=0x7f96c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.731+0000 7f96c604d640 1 --2- 172.21.15.143:0/3911788998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0101de0 0x7f96c01021c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009ca0 tx=0x7f96b002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.731+0000 7f96c604d640 1 -- 172.21.15.143:0/3911788998 shutdown_connections 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 --2- 172.21.15.143:0/3911788998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c0106f10 0x7f96c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 --2- 172.21.15.143:0/3911788998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0102790 0x7f96c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 --2- 172.21.15.143:0/3911788998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0101de0 0x7f96c01021c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 -- 172.21.15.143:0/3911788998 >> 172.21.15.143:0/3911788998 conn(0x7f96c00fd8c0 msgr2=0x7f96c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 -- 172.21.15.143:0/3911788998 shutdown_connections 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 -- 172.21.15.143:0/3911788998 wait complete. 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 Processor -- start 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.732+0000 7f96c604d640 1 -- start start 2024-09-10T10:49:13.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0101de0 0x7f96c0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0102790 0x7f96c01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c0106f10 0x7f96c019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0111e20 con 0x7f96c0102790 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0111ca0 con 0x7f96c0106f10 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0111fa0 con 0x7f96c0101de0 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0101de0 0x7f96c0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0102790 0x7f96c01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0101de0 0x7f96c0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:37000/0 (socket says 172.21.15.143:37000) 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bf7fe640 1 -- 172.21.15.143:0/1988484441 learned_addr learned my addr 172.21.15.143:0/1988484441 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bffff640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c0106f10 0x7f96c019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 -- 172.21.15.143:0/1988484441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0101de0 msgr2=0x7f96c0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0101de0 0x7f96c0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 -- 172.21.15.143:0/1988484441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c0106f10 msgr2=0x7f96c019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c0106f10 0x7f96c019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 -- 172.21.15.143:0/1988484441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c019d310 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96beffd640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0102790 0x7f96c01988a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f96b400e9f0 tx=0x7f96b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bcff9640 1 -- 172.21.15.143:0/1988484441 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b400ccc0 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bcff9640 1 -- 172.21.15.143:0/1988484441 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96b4005020 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c019d600 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.733+0000 7f96bcff9640 1 -- 172.21.15.143:0/1988484441 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4004a00 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.734+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a4ee0 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.736+0000 7f96bcff9640 1 -- 172.21.15.143:0/1988484441 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f96b4004ba0 con 0x7f96c0102790 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.736+0000 7f96c8426740 1 Processor -- start 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.736+0000 7f96c8426740 1 -- start start 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bcff9640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96a80a2b80 0x7f96a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96c8426740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96c8426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e255d0 0x5646d1e25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96c8426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e30370 0x5646d1e32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96c8426740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5646d1d8b2f0 con 0x5646d1e26b40 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96c8426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646d1d32cf0 con 0x5646d1e255d0 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96c8426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5646d1d872b0 con 0x5646d1e30370 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bffff640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96a80a2b80 0x7f96a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bcff9640 1 -- 172.21.15.143:0/1988484441 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f96b40cf350 con 0x7f96c0102790 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e25090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58510/0 (socket says 172.21.15.143:58510) 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 -- 172.21.15.143:0/29755581 learned_addr learned my addr 172.21.15.143:0/29755581 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bffff640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e30370 0x5646d1e32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96beffd640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e255d0 0x5646d1e25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 -- 172.21.15.143:0/29755581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e30370 msgr2=0x5646d1e32760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e30370 0x5646d1e32760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 -- 172.21.15.143:0/29755581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e255d0 msgr2=0x5646d1e25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e255d0 0x5646d1e25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 -- 172.21.15.143:0/29755581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d1d173a0 con 0x5646d1e26b40 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bffff640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e30370 0x5646d1e32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96beffd640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e255d0 0x5646d1e25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bffff640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96a80a2b80 0x7f96a80a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009ca0 tx=0x7f96b8009830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.737+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e25090 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006010 tx=0x7f96b0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b9f690cef16b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96a5ffb640 1 -- 172.21.15.143:0/29755581 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0031d40 con 0x5646d1e26b40 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96a5ffb640 1 -- 172.21.15.143:0/29755581 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96b0036030 con 0x5646d1e26b40 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96a5ffb640 1 -- 172.21.15.143:0/29755581 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0004330 con 0x5646d1e26b40 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 -- 172.21.15.143:0/29755581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 msgr2=0x5646d1e25090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e25090 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006010 tx=0x7f96b0009a20 comp rx=0 tx=0).stop 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 -- 172.21.15.143:0/29755581 shutdown_connections 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e30370 0x5646d1e32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e255d0 0x5646d1e25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 --2- 172.21.15.143:0/29755581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e25090 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 -- 172.21.15.143:0/29755581 >> 172.21.15.143:0/29755581 conn(0x5646d1dee310 msgr2=0x5646d1e21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 -- 172.21.15.143:0/29755581 shutdown_connections 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.738+0000 7f96c8426740 1 -- 172.21.15.143:0/29755581 wait complete. 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 -- start start 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 Processor -- start 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e2fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e26b40 0x5646d1e2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 0x5646d1e2d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5646d1e372a0 con 0x5646d1e30370 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646d1d46060 con 0x5646d1e26b40 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5646d1e37420 con 0x5646d1e255d0 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.739+0000 7f96c8426740 10 client.0 _ll_register_callbacks cb 0x5646d1a998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 0x5646d1e2d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e2fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e2fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:37012/0 (socket says 172.21.15.143:37012) 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 0x5646d1e2d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58516/0 (socket says 172.21.15.143:58516) 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bf7fe640 1 -- 172.21.15.143:0/3463891319 learned_addr learned my addr 172.21.15.143:0/3463891319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96beffd640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e26b40 0x5646d1e2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 -- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 msgr2=0x5646d1e2fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e2fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 -- 172.21.15.143:0/3463891319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e26b40 msgr2=0x5646d1e2cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e26b40 0x5646d1e2cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 -- 172.21.15.143:0/3463891319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d1afc3c0 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96beffd640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e26b40 0x5646d1e2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96bffff640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 0x5646d1e2d5d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005f60 tx=0x7f96b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9675891da35d01b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f969effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f969effd640 1 -- 172.21.15.143:0/3463891319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b80118d0 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f969effd640 1 -- 172.21.15.143:0/3463891319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f96b8011a70 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b0036030 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f969effd640 1 -- 172.21.15.143:0/3463891319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96b8011c70 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96c8426740 10 client.16401 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.740+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5646d1e42d00 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f969effd640 1 -- 172.21.15.143:0/3463891319 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96b0036030 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f969effd640 1 -- 172.21.15.143:0/3463891319 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f96b801f070 con 0x5646d1e30370 2024-09-10T10:49:13.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96c8426740 10 client.16401 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96c8426740 10 client.16401 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96c8426740 20 client.16401 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96c8426740 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1e44210 0x5646d1e46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96c8426740 4 client.16401 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646d1d98860 con 0x5646d1e44210 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1e44210 0x5646d1e46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1e44210 0x5646d1e46690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88c181ff0d776888 server_cookie=fb56397fc65cf2f2 in_seq=0 out_seq=0 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.741+0000 7f969effd640 10 client.16401 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f969effd640 1 -- 172.21.15.143:0/3463891319 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646d1d98860 con 0x5646d1e44210 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f969effd640 10 client.16401 handle_command_reply: tid=0 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 client.16401 shutdown 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 msgr2=0x5646d1e2d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 0x5646d1e2d5d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005f60 tx=0x7f96b8004730 comp rx=0 tx=0).stop 2024-09-10T10:49:13.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 shutdown_connections 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1e44210 0x5646d1e46690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e30370 0x5646d1e2d5d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e26b40 0x5646d1e2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 --2- 172.21.15.143:0/3463891319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e2fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.743+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 >> 172.21.15.143:0/3463891319 conn(0x5646d1dee310 msgr2=0x5646d1e27960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.744+0000 7f969effd640 0 client.16401 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.744+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 shutdown_connections 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.744+0000 7f96c8426740 1 -- 172.21.15.143:0/3463891319 wait complete. 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.744+0000 7f96c8426740 20 client.16401 trim_cache size 0 max 16384 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.757+0000 7f96c8426740 1 Processor -- start 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.757+0000 7f96c8426740 1 -- start start 2024-09-10T10:49:13.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.757+0000 7f96c8426740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 0x5646d1e40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.757+0000 7f96c8426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 0x5646d1e2b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.757+0000 7f96c8426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e2be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.757+0000 7f96c8426740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ac850 con 0x5646d1e255d0 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96c8426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ae3c0 con 0x5646d1e30370 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96c8426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01acf20 con 0x5646d1e26b40 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 0x5646d1e40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 0x5646d1e2b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 0x5646d1e2b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:37016/0 (socket says 172.21.15.143:37016) 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96bffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e2be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 0x5646d1e40540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58522/0 (socket says 172.21.15.143:58522) 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 -- 172.21.15.143:0/1257395490 learned_addr learned my addr 172.21.15.143:0/1257395490 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 -- 172.21.15.143:0/1257395490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 msgr2=0x5646d1e2be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e2be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 -- 172.21.15.143:0/1257395490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 msgr2=0x5646d1e40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 0x5646d1e40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 -- 172.21.15.143:0/1257395490 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d1e21d70 con 0x5646d1e26b40 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 0x5646d1e40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96beffd640 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 0x5646d1e2b8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96b409b550 tx=0x7f96b4086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8affbe7e8c6a2630 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96a57fa640 1 -- 172.21.15.143:0/1257395490 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b401c070 con 0x5646d1e26b40 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96a57fa640 1 -- 172.21.15.143:0/1257395490 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96b40b4070 con 0x5646d1e26b40 2024-09-10T10:49:13.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.758+0000 7f96a57fa640 1 -- 172.21.15.143:0/1257395490 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b40051d0 con 0x5646d1e26b40 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 -- 172.21.15.143:0/1257395490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 msgr2=0x5646d1e2b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 0x5646d1e2b8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96b409b550 tx=0x7f96b4086040 comp rx=0 tx=0).stop 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 -- 172.21.15.143:0/1257395490 shutdown_connections 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e2be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e26b40 0x5646d1e2b8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 --2- 172.21.15.143:0/1257395490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e255d0 0x5646d1e40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 -- 172.21.15.143:0/1257395490 >> 172.21.15.143:0/1257395490 conn(0x5646d1dee310 msgr2=0x5646d1e2cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 -- 172.21.15.143:0/1257395490 shutdown_connections 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.759+0000 7f96c8426740 1 -- 172.21.15.143:0/1257395490 wait complete. 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 -- start start 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 Processor -- start 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e3c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e29290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01adcc0 con 0x5646d1e26b40 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ad5f0 con 0x5646d1e30370 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01aa830 con 0x5646d1e255d0 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96c8426740 10 client.0 _ll_register_callbacks cb 0x7f96b800f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e3c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96beffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96bffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e29290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96bf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e3c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:37020/0 (socket says 172.21.15.143:37020) 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.760+0000 7f96bf7fe640 1 -- 172.21.15.143:0/1175927224 learned_addr learned my addr 172.21.15.143:0/1175927224 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96bf7fe640 1 -- 172.21.15.143:0/1175927224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 msgr2=0x5646d1e29290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e29290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96bf7fe640 1 -- 172.21.15.143:0/1175927224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 msgr2=0x5646d1e3cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e3cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96bf7fe640 1 -- 172.21.15.143:0/1175927224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d1e2aeb0 con 0x5646d1e255d0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96bf7fe640 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e3c6b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96b0005e30 tx=0x7f96b002b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da83766b6f9cae34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f969effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f969effd640 1 -- 172.21.15.143:0/1175927224 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0055070 con 0x5646d1e255d0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b40b4070 con 0x5646d1e255d0 2024-09-10T10:49:13.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f969effd640 1 -- 172.21.15.143:0/1175927224 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f96b0003c30 con 0x5646d1e255d0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f969effd640 1 -- 172.21.15.143:0/1175927224 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96b0062050 con 0x5646d1e255d0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96c8426740 10 client.25813 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5646d1eae430 con 0x5646d1e255d0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.761+0000 7f969effd640 1 -- 172.21.15.143:0/1175927224 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96b40b4070 con 0x5646d1e255d0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f969effd640 1 -- 172.21.15.143:0/1175927224 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f96b0043070 con 0x5646d1e255d0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96c8426740 10 client.25813 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96c8426740 10 client.25813 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96c8426740 20 client.25813 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96c8426740 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1eae980 0x5646d1eb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96c8426740 4 client.25813 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646d1d92d70 con 0x5646d1eae980 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96beffd640 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1eae980 0x5646d1eb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f96beffd640 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1eae980 0x5646d1eb0da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc182359f914b844 server_cookie=835c922d57919a4b in_seq=0 out_seq=0 2024-09-10T10:49:13.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.762+0000 7f969effd640 10 client.25813 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f969effd640 1 -- 172.21.15.143:0/1175927224 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646d1d92d70 con 0x5646d1eae980 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f969effd640 10 client.25813 handle_command_reply: tid=0 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 client.25813 shutdown 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 msgr2=0x5646d1e3c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e3c6b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96b0005e30 tx=0x7f96b002b070 comp rx=0 tx=0).stop 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 shutdown_connections 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5646d1eae980 0x5646d1eb0da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646d1e30370 0x5646d1e29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f969effd640 0 client.25813 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646d1e26b40 0x5646d1e3cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 --2- 172.21.15.143:0/1175927224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5646d1e255d0 0x5646d1e3c6b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 >> 172.21.15.143:0/1175927224 conn(0x5646d1dee310 msgr2=0x5646d1e2b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 shutdown_connections 2024-09-10T10:49:13.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 1 -- 172.21.15.143:0/1175927224 wait complete. 2024-09-10T10:49:13.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.764+0000 7f96c8426740 20 client.25813 trim_cache size 0 max 16384 2024-09-10T10:49:13.780 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:49:13.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96a80a2b80 msgr2=0x7f96a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96a80a2b80 0x7f96a80a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009ca0 tx=0x7f96b8009830 comp rx=0 tx=0).stop 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0102790 msgr2=0x7f96c01988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0102790 0x7f96c01988a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f96b400e9f0 tx=0x7f96b400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 shutdown_connections 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f96a80a2b80 0x7f96a80a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c0106f10 0x7f96c019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96c0102790 0x7f96c01988a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 --2- 172.21.15.143:0/1988484441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96c0101de0 0x7f96c0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 >> 172.21.15.143:0/1988484441 conn(0x7f96c00fd8c0 msgr2=0x7f96c00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 shutdown_connections 2024-09-10T10:49:13.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:13.765+0000 7f96c604d640 1 -- 172.21.15.143:0/1988484441 wait complete. 2024-09-10T10:49:14.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:49:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:14 smithi178 ceph-mon[34813]: pgmap v476: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 142 op/s 2024-09-10T10:49:14.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:14 smithi144 ceph-mon[34230]: pgmap v476: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 142 op/s 2024-09-10T10:49:15.023 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:14 smithi143 ceph-mon[31057]: pgmap v476: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 142 op/s 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.686+0000 7f120ee48640 1 -- 172.21.15.143:0/362980124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12081027a0 msgr2=0x7f12081067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.686+0000 7f120ee48640 1 --2- 172.21.15.143:0/362980124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12081027a0 0x7f12081067f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009900 tx=0x7f11f402a410 comp rx=0 tx=0).stop 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 -- 172.21.15.143:0/362980124 shutdown_connections 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 --2- 172.21.15.143:0/362980124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208106f20 0x7f120810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 --2- 172.21.15.143:0/362980124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12081027a0 0x7f12081067f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 --2- 172.21.15.143:0/362980124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1208101df0 0x7f12081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 -- 172.21.15.143:0/362980124 >> 172.21.15.143:0/362980124 conn(0x7f12080fd8b0 msgr2=0x7f12080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 -- 172.21.15.143:0/362980124 shutdown_connections 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 -- 172.21.15.143:0/362980124 wait complete. 2024-09-10T10:49:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 Processor -- start 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.687+0000 7f120ee48640 1 -- start start 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120ee48640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208101df0 0x7f12081983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120ee48640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12081027a0 0x7f1208198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120ee48640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1208106f20 0x7f120819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f1207fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12081027a0 0x7f1208198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120cbe4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208101df0 0x7f12081983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f1207fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12081027a0 0x7f1208198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:37038/0 (socket says 172.21.15.143:37038) 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f1207fff640 1 -- 172.21.15.143:0/1647949971 learned_addr learned my addr 172.21.15.143:0/1647949971 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120d3e5640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1208106f20 0x7f120819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111df0 con 0x7f1208106f20 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111c70 con 0x7f1208101df0 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111f70 con 0x7f12081027a0 2024-09-10T10:49:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120cbe4640 1 -- 172.21.15.143:0/1647949971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12081027a0 msgr2=0x7f1208198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12081027a0 0x7f1208198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120cbe4640 1 -- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1208106f20 msgr2=0x7f120819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.688+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1208106f20 0x7f120819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f120cbe4640 1 -- 172.21.15.143:0/1647949971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120819d380 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208101df0 0x7f12081983d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9f0 tx=0x7f11fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f1205ffb640 1 -- 172.21.15.143:0/1647949971 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00ccc0 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120819d670 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f1205ffb640 1 -- 172.21.15.143:0/1647949971 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11fc0108e0 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f1205ffb640 1 -- 172.21.15.143:0/1647949971 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc018880 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.689+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a4f50 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.692+0000 7f1205ffb640 1 -- 172.21.15.143:0/1647949971 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f11fc010430 con 0x7f1208101df0 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 Processor -- start 2024-09-10T10:49:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1205ffb640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11f80a2b80 0x7f11f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 -- start start 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4319090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4319a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4326760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628e427f2f0 con 0x5628e4324370 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628e4226cf0 con 0x5628e431ab40 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f1211221740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5628e427b2b0 con 0x5628e43195d0 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.693+0000 7f120d3e5640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11f80a2b80 0x7f11f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4319090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4319090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42734/0 (socket says 172.21.15.143:42734) 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 -- 172.21.15.143:0/3851395787 learned_addr learned my addr 172.21.15.143:0/3851395787 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120d3e5640 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4326760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f1207fff640 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4319a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f1205ffb640 1 -- 172.21.15.143:0/1647949971 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f11fc0cf6b0 con 0x7f1208101df0 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 -- 172.21.15.143:0/3851395787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 msgr2=0x5628e4319a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4319a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 -- 172.21.15.143:0/3851395787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 msgr2=0x5628e4326760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4326760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 -- 172.21.15.143:0/3851395787 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e420b3a0 con 0x5628e431ab40 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f120cbe4640 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4319090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11fc098630 tx=0x7f11fc04a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99514614e85d69e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f11f2ffd640 1 -- 172.21.15.143:0/3851395787 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc010720 con 0x5628e431ab40 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f11f2ffd640 1 -- 172.21.15.143:0/3851395787 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11fc09bc60 con 0x5628e431ab40 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.694+0000 7f11f2ffd640 1 -- 172.21.15.143:0/3851395787 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0982d0 con 0x5628e431ab40 2024-09-10T10:49:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.695+0000 7f120d3e5640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11f80a2b80 0x7f11f80a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5628e431a070 tx=0x7f1200009b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.695+0000 7f1211221740 1 -- 172.21.15.143:0/3851395787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 msgr2=0x5628e4319090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.695+0000 7f1211221740 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4319090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11fc098630 tx=0x7f11fc04a770 comp rx=0 tx=0).stop 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 -- 172.21.15.143:0/3851395787 shutdown_connections 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4326760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4319a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 --2- 172.21.15.143:0/3851395787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4319090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 -- 172.21.15.143:0/3851395787 >> 172.21.15.143:0/3851395787 conn(0x5628e42e2310 msgr2=0x5628e4315c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 -- 172.21.15.143:0/3851395787 shutdown_connections 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 -- 172.21.15.143:0/3851395787 wait complete. 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 -- start start 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.696+0000 7f1211221740 1 Processor -- start 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4323d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4321040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4321650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628e432b420 con 0x5628e4324370 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628e423a060 con 0x5628e431ab40 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1207fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4321040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4323d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1207fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4321040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42740/0 (socket says 172.21.15.143:42740) 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1207fff640 1 -- 172.21.15.143:0/2354890897 learned_addr learned my addr 172.21.15.143:0/2354890897 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5628e432b5a0 con 0x5628e43195d0 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120d3e5640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4321650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 10 client.0 _ll_register_callbacks cb 0x5628e3f8d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 -- 172.21.15.143:0/2354890897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 msgr2=0x5628e4321040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4321040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 -- 172.21.15.143:0/2354890897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 msgr2=0x5628e4321650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4321650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 -- 172.21.15.143:0/2354890897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e3ff03c0 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc09bc60 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120d3e5640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4321650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f120cbe4640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4323d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0cd1c0 tx=0x7f11fc09a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d119db412caf0ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.697+0000 7f11e7fff640 10 client.25825 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f11e7fff640 1 -- 172.21.15.143:0/2354890897 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc07f050 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f11e7fff640 1 -- 172.21.15.143:0/2354890897 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f11fc04b050 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f11e7fff640 1 -- 172.21.15.143:0/2354890897 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f11fc0b2070 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 10 client.25825 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5628e432b420 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f11e7fff640 1 -- 172.21.15.143:0/2354890897 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f11fc0ba2a0 con 0x5628e43195d0 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 10 client.25825 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 10 client.25825 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:49:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 20 client.25825 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e4338230 0x5628e433a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 4 client.25825 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e428c860 con 0x5628e4338230 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.698+0000 7f1207fff640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e4338230 0x5628e433a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.699+0000 7f1207fff640 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e4338230 0x5628e433a660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6e93a8dd864b1ec server_cookie=a47ff91a89794716 in_seq=0 out_seq=0 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.699+0000 7f11e7fff640 10 client.25825 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.699+0000 7f11e7fff640 1 -- 172.21.15.143:0/2354890897 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628e428c860 con 0x5628e4338230 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.699+0000 7f11e7fff640 10 client.25825 handle_command_reply: tid=0 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.699+0000 7f1211221740 1 client.25825 shutdown 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 msgr2=0x5628e4323d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4323d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0cd1c0 tx=0x7f11fc09a140 comp rx=0 tx=0).stop 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 shutdown_connections 2024-09-10T10:49:15.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e4338230 0x5628e433a660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e4324370 0x5628e4321650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e4321040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 --2- 172.21.15.143:0/2354890897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e4323d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 >> 172.21.15.143:0/2354890897 conn(0x5628e42e2310 msgr2=0x5628e431b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f11e7fff640 0 client.25825 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 shutdown_connections 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 1 -- 172.21.15.143:0/2354890897 wait complete. 2024-09-10T10:49:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.700+0000 7f1211221740 20 client.25825 trim_cache size 0 max 16384 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 Processor -- start 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 -- start start 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e43195d0 0x5628e431c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e431c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e4324370 0x5628e433ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ac8a0 con 0x5628e43195d0 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ae410 con 0x5628e431ab40 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.716+0000 7f1211221740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12081acf70 con 0x5628e4324370 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e431c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e431c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42746/0 (socket says 172.21.15.143:42746) 2024-09-10T10:49:15.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 -- 172.21.15.143:0/1348095687 learned_addr learned my addr 172.21.15.143:0/1348095687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e43195d0 0x5628e431c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f120d3e5640 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e4324370 0x5628e433ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 -- 172.21.15.143:0/1348095687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e4324370 msgr2=0x5628e433ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e4324370 0x5628e433ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 -- 172.21.15.143:0/1348095687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e43195d0 msgr2=0x5628e431c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e43195d0 0x5628e431c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 -- 172.21.15.143:0/1348095687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e4316960 con 0x5628e431ab40 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f1207fff640 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e431c620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11f40096c0 tx=0x7f11f40058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13132f2e54b0c429 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.717+0000 7f11f27fc640 1 -- 172.21.15.143:0/1348095687 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f404a020 con 0x5628e431ab40 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.718+0000 7f11f27fc640 1 -- 172.21.15.143:0/1348095687 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f40054f0 con 0x5628e431ab40 2024-09-10T10:49:15.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.718+0000 7f11f27fc640 1 -- 172.21.15.143:0/1348095687 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4035040 con 0x5628e431ab40 2024-09-10T10:49:15.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.718+0000 7f1211221740 1 -- 172.21.15.143:0/1348095687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 msgr2=0x5628e431c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.718+0000 7f1211221740 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e431c620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11f40096c0 tx=0x7f11f40058d0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 -- 172.21.15.143:0/1348095687 shutdown_connections 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e4324370 0x5628e433ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e431ab40 0x5628e431c620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 --2- 172.21.15.143:0/1348095687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e43195d0 0x5628e431c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 -- 172.21.15.143:0/1348095687 >> 172.21.15.143:0/1348095687 conn(0x5628e42e2310 msgr2=0x5628e4314d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 -- 172.21.15.143:0/1348095687 shutdown_connections 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 -- 172.21.15.143:0/1348095687 wait complete. 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 -- start start 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.719+0000 7f1211221740 1 Processor -- start 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e43278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e431ab40 0x5628e4327de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e4324370 0x5628e432f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f12081add10 con 0x5628e431ab40 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad640 con 0x5628e4324370 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12081aa880 con 0x5628e43195d0 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1211221740 10 client.0 _ll_register_callbacks cb 0x7f1200009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120cbe4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e43278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120cbe4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e43278a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:37082/0 (socket says 172.21.15.143:37082) 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120cbe4640 1 -- 172.21.15.143:0/1856963245 learned_addr learned my addr 172.21.15.143:0/1856963245 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120d3e5640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e4324370 0x5628e432f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f1207fff640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e431ab40 0x5628e4327de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120cbe4640 1 -- 172.21.15.143:0/1856963245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e4324370 msgr2=0x5628e432f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e4324370 0x5628e432f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.720+0000 7f120cbe4640 1 -- 172.21.15.143:0/1856963245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e431ab40 msgr2=0x5628e4327de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e431ab40 0x5628e4327de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f120cbe4640 1 -- 172.21.15.143:0/1856963245 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e4321fe0 con 0x5628e43195d0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f120d3e5640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e4324370 0x5628e432f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f1207fff640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e431ab40 0x5628e4327de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f120cbe4640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e43278a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9c0 tx=0x7f11fc097860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdbe4aed395d0791 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f11e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f11e7fff640 1 -- 172.21.15.143:0/1856963245 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc07f040 con 0x5628e43195d0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f11e7fff640 1 -- 172.21.15.143:0/1856963245 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f11fc09b2b0 con 0x5628e43195d0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f40054f0 con 0x5628e43195d0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.721+0000 7f11e7fff640 1 -- 172.21.15.143:0/1856963245 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f11fc010720 con 0x5628e43195d0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 10 client.25831 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5628e4330f70 con 0x5628e43195d0 2024-09-10T10:49:15.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f11e7fff640 1 -- 172.21.15.143:0/1856963245 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11fc09bc60 con 0x5628e43195d0 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f11e7fff640 1 -- 172.21.15.143:0/1856963245 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f11fc09ab20 con 0x5628e43195d0 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 10 client.25831 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 10 client.25831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 20 client.25831 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e43a7700 0x5628e43a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 4 client.25831 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f11f40054f0 con 0x5628e43a7700 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.722+0000 7f1207fff640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e43a7700 0x5628e43a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f1207fff640 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e43a7700 0x5628e43a9af0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25af7d2431c4425c server_cookie=6cf430c7f38e29d1 in_seq=0 out_seq=0 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f11e7fff640 10 client.25831 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f11e7fff640 1 -- 172.21.15.143:0/1856963245 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f11f40054f0 con 0x5628e43a7700 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f11e7fff640 10 client.25831 handle_command_reply: tid=0 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f1211221740 1 client.25831 shutdown 2024-09-10T10:49:15.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 msgr2=0x5628e43278a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.723+0000 7f1211221740 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e43278a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9c0 tx=0x7f11fc097860 comp rx=0 tx=0).stop 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 shutdown_connections 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5628e43a7700 0x5628e43a9af0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628e4324370 0x5628e432f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628e431ab40 0x5628e4327de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 --2- 172.21.15.143:0/1856963245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5628e43195d0 0x5628e43278a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 >> 172.21.15.143:0/1856963245 conn(0x5628e42e2310 msgr2=0x5628e43228a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f11e7fff640 0 client.25831 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 shutdown_connections 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 1 -- 172.21.15.143:0/1856963245 wait complete. 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.724+0000 7f1211221740 20 client.25831 trim_cache size 0 max 16384 2024-09-10T10:49:15.731 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:49:15.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11f80a2b80 msgr2=0x7f11f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11f80a2b80 0x7f11f80a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5628e431a070 tx=0x7f1200009b70 comp rx=0 tx=0).stop 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208101df0 msgr2=0x7f12081983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208101df0 0x7f12081983d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9f0 tx=0x7f11fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 shutdown_connections 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f11f80a2b80 0x7f11f80a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1208106f20 0x7f120819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12081027a0 0x7f1208198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 --2- 172.21.15.143:0/1647949971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1208101df0 0x7f12081983d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 >> 172.21.15.143:0/1647949971 conn(0x7f12080fd8b0 msgr2=0x7f12080fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 shutdown_connections 2024-09-10T10:49:15.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.725+0000 7f120ee48640 1 -- 172.21.15.143:0/1647949971 wait complete. 2024-09-10T10:49:15.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8ccffff640 1 -- 172.21.15.143:0/2312631386 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558330bf1f40 con 0x558330d246c0 2024-09-10T10:49:15.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8ccffff640 10 client.25771 handle_command_reply: tid=0 2024-09-10T10:49:15.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 client.25771 shutdown 2024-09-10T10:49:15.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 msgr2=0x558330d18100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d18100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x558330d07160 tx=0x7f8cdc00bde0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 shutdown_connections 2024-09-10T10:49:15.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558330d14a30 0x558330d18640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x558330d246c0 0x558330d91860 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558330d0a170 0x558330d18100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 --2- 172.21.15.143:0/2312631386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558330d09850 0x558330d1cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8ccffff640 0 client.25771 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:15.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.739+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 >> 172.21.15.143:0/2312631386 conn(0x558330d05de0 msgr2=0x558330d0b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.740+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 shutdown_connections 2024-09-10T10:49:15.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.740+0000 7f8cf949c740 1 -- 172.21.15.143:0/2312631386 wait complete. 2024-09-10T10:49:15.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.740+0000 7f8cf949c740 20 client.25771 trim_cache size 0 max 16384 2024-09-10T10:49:15.745 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:15.746 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 32, 2024-09-10T10:49:15.746 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:49:15.746 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 10, 2024-09-10T10:49:15.746 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:49:15.746 INFO:teuthology.orchestra.run.smithi143.stdout: "544fb307": { 2024-09-10T10:49:15.747 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 10, 2024-09-10T10:49:15.747 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:49:15.747 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:49:15.747 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:49:15.747 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:49:15.747 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:49:15.748 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:49:15.748 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 142.8, 2024-09-10T10:49:15.748 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:49:15.748 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:49:15.748 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:49:15.748 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:49:15.749 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:49:15.749 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:49:15.749 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:15.749 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:15.749 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:15.750 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:15.750 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:15.750 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:15.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8ce00a2b80 msgr2=0x7f8ce00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8ce00a2b80 0x7f8ce00a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8001190 tx=0x7f8ce8005d80 comp rx=0 tx=0).stop 2024-09-10T10:49:15.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 msgr2=0x7f8cf0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 0x7f8cf0100990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400e9f0 tx=0x7f8ce400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 shutdown_connections 2024-09-10T10:49:15.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8ce00a2b80 0x7f8ce00a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cf010c160 0x7f8cf0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cf0103650 0x7f8cf0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 --2- 172.21.15.143:0/755221845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8cf0102d30 0x7f8cf0100990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 >> 172.21.15.143:0/755221845 conn(0x7f8cf00fc790 msgr2=0x7f8cf010e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 shutdown_connections 2024-09-10T10:49:15.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:15.741+0000 7f8cf70c3640 1 -- 172.21.15.143:0/755221845 wait complete. 2024-09-10T10:49:16.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:16 smithi143 ceph-mon[31057]: pgmap v477: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 134 op/s 2024-09-10T10:49:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:16 smithi178 ceph-mon[34813]: pgmap v477: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 134 op/s 2024-09-10T10:49:16.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:16 smithi144 ceph-mon[34230]: pgmap v477: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 134 op/s 2024-09-10T10:49:16.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:49:17.602 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '544fb307', quiesce took 12.4 seconds. Will release after: 10.35 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.179+0000 7f8e05434640 1 -- 172.21.15.143:0/2065433002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e001036e0 msgr2=0x7f8e00107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.179+0000 7f8e05434640 1 --2- 172.21.15.143:0/2065433002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e001036e0 0x7f8e00107b90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009930 tx=0x7f8df402a420 comp rx=0 tx=0).stop 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 -- 172.21.15.143:0/2065433002 shutdown_connections 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 --2- 172.21.15.143:0/2065433002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e001082a0 0x7f8e0010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 --2- 172.21.15.143:0/2065433002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e001036e0 0x7f8e00107b90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 --2- 172.21.15.143:0/2065433002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e00102d30 0x7f8e00103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 -- 172.21.15.143:0/2065433002 >> 172.21.15.143:0/2065433002 conn(0x7f8e000fc7d0 msgr2=0x7f8e000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 -- 172.21.15.143:0/2065433002 shutdown_connections 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.180+0000 7f8e05434640 1 -- 172.21.15.143:0/2065433002 wait complete. 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 Processor -- start 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 -- start start 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 0x7f8e00100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e001036e0 0x7f8e00100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 0x7f8e00100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e001082a0 0x7f8e00101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 0x7f8e00100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44230/0 (socket says 172.21.15.143:44230) 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1319381482 learned_addr learned my addr 172.21.15.143:0/1319381482 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:18.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00111f50 con 0x7f8e00102d30 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00111dd0 con 0x7f8e001082a0 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001120d0 con 0x7f8e001036e0 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.181+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e001036e0 0x7f8e00100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dff7fe640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e001082a0 0x7f8e00101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1319381482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e001036e0 msgr2=0x7f8e00100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e001036e0 0x7f8e00100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1319381482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e001082a0 msgr2=0x7f8e00101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e001082a0 0x7f8e00101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1319381482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e001aae60 con 0x7f8e00102d30 2024-09-10T10:49:18.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e001036e0 0x7f8e00100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 0x7f8e00100990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e9f0 tx=0x7f8df000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8de7fff640 1 -- 172.21.15.143:0/1319381482 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000cd80 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e001ab090 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8de7fff640 1 -- 172.21.15.143:0/1319381482 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df00108e0 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.182+0000 7f8de7fff640 1 -- 172.21.15.143:0/1319381482 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0018920 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.183+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e001ab5a0 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.185+0000 7f8de7fff640 1 -- 172.21.15.143:0/1319381482 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8df0010a90 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.186+0000 7f8de7fff640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8de80a2b80 0x7f8de80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8de7fff640 1 -- 172.21.15.143:0/1319381482 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8df00cf780 con 0x7f8e00102d30 2024-09-10T10:49:18.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 Processor -- start 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 -- start start 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8de80a2b80 0x7f8de80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c89922b40 0x556c89921090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89921a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c8992c370 0x556c8992e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556c898872f0 con 0x556c8992c370 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c8982ecf0 con 0x556c89922b40 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8e0780d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c898832b0 con 0x556c899215d0 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89921a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89921a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47720/0 (socket says 172.21.15.143:47720) 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c89922b40 0x556c89921090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/1318736207 learned_addr learned my addr 172.21.15.143:0/1318736207 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8de80a2b80 0x7f8de80a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8df4031440 tx=0x7f8df4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.187+0000 7f8dff7fe640 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c8992c370 0x556c8992e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/1318736207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c89922b40 msgr2=0x556c89921090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c89922b40 0x556c89921090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/1318736207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c8992c370 msgr2=0x556c8992e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c8992c370 0x556c8992e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/1318736207 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c898133a0 con 0x556c899215d0 2024-09-10T10:49:18.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dff7fe640 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c8992c370 0x556c8992e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89921a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8df4033000 tx=0x7f8df4042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d810e9571d1980 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8de57fa640 1 -- 172.21.15.143:0/1318736207 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4035040 con 0x556c899215d0 2024-09-10T10:49:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8de57fa640 1 -- 172.21.15.143:0/1318736207 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4042070 con 0x556c899215d0 2024-09-10T10:49:18.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.188+0000 7f8de57fa640 1 -- 172.21.15.143:0/1318736207 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40035f0 con 0x556c899215d0 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 -- 172.21.15.143:0/1318736207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 msgr2=0x556c89921a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89921a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8df4033000 tx=0x7f8df4042000 comp rx=0 tx=0).stop 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 -- 172.21.15.143:0/1318736207 shutdown_connections 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c8992c370 0x556c8992e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89921a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1318736207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c89922b40 0x556c89921090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 -- 172.21.15.143:0/1318736207 >> 172.21.15.143:0/1318736207 conn(0x556c898ea310 msgr2=0x556c8991dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 -- 172.21.15.143:0/1318736207 shutdown_connections 2024-09-10T10:49:18.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.189+0000 7f8e0780d740 1 -- 172.21.15.143:0/1318736207 wait complete. 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.190+0000 7f8e0780d740 1 -- start start 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.190+0000 7f8e0780d740 1 Processor -- start 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c899215d0 0x556c8992bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c89922b40 0x556c89929030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89929640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556c8992f010 con 0x556c899215d0 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c89842060 con 0x556c8992c370 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c899215d0 0x556c8992bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c899215d0 0x556c8992bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44248/0 (socket says 172.21.15.143:44248) 2024-09-10T10:49:18.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c89922b40 0x556c89929030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8dfeffd640 1 -- 172.21.15.143:0/3822613522 learned_addr learned my addr 172.21.15.143:0/3822613522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556c8992f190 con 0x556c89922b40 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8dff7fe640 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89929640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.191+0000 7f8e0780d740 10 client.0 _ll_register_callbacks cb 0x556c895958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 msgr2=0x556c89929640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89929640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/3822613522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c899215d0 msgr2=0x556c8992bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c899215d0 0x556c8992bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8dfe7fc640 1 -- 172.21.15.143:0/3822613522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c895f83c0 con 0x556c89922b40 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c89922b40 0x556c89929030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8df402a940 tx=0x7f8df4004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de460053926a88c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8de2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3822613522 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df405d060 con 0x556c89922b40 2024-09-10T10:49:18.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3822613522 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8df40055c0 con 0x556c89922b40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df4042070 con 0x556c89922b40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3822613522 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8df4004980 con 0x556c89922b40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8e0780d740 10 client.25843 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.192+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c8992f060 con 0x556c89922b40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3822613522 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4042070 con 0x556c89922b40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3822613522 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f8df4003ba0 con 0x556c89922b40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8e0780d740 10 client.25843 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8e0780d740 10 client.25843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8e0780d740 20 client.25843 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c8993e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8e0780d740 4 client.25843 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c89894860 con 0x556c8993bd40 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.193+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c8993e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.194+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c8993e130 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99dde3a605a4295b server_cookie=811fd3e11ca1b208 in_seq=0 out_seq=0 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.194+0000 7f8de2ff5640 10 client.25843 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.194+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3822613522 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c89894860 con 0x556c8993bd40 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.194+0000 7f8de2ff5640 10 client.25843 handle_command_reply: tid=0 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.194+0000 7f8e0780d740 1 client.25843 shutdown 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c89922b40 msgr2=0x556c89929030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c89922b40 0x556c89929030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8df402a940 tx=0x7f8df4004fb0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 shutdown_connections 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c8993e130 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8de2ff5640 0 client.25843 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89929640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c89922b40 0x556c89929030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3822613522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c899215d0 0x556c8992bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 >> 172.21.15.143:0/3822613522 conn(0x556c898ea310 msgr2=0x556c89923960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 shutdown_connections 2024-09-10T10:49:18.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 1 -- 172.21.15.143:0/3822613522 wait complete. 2024-09-10T10:49:18.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.195+0000 7f8e0780d740 20 client.25843 trim_cache size 0 max 16384 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.213+0000 7f8e0780d740 1 Processor -- start 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.213+0000 7f8e0780d740 1 -- start start 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.213+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c899215d0 0x556c899382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.213+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89928010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.213+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 0x556c89943850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8e0780d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001b50a0 con 0x556c89922b40 2024-09-10T10:49:18.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8e0780d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001b6c10 con 0x556c899215d0 2024-09-10T10:49:18.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8e0780d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001b5770 con 0x556c8992c370 2024-09-10T10:49:18.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 0x556c89943850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89928010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 0x556c89943850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47738/0 (socket says 172.21.15.143:47738) 2024-09-10T10:49:18.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dff7fe640 1 -- 172.21.15.143:0/1923027478 learned_addr learned my addr 172.21.15.143:0/1923027478 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:18.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c899215d0 0x556c899382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1923027478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 msgr2=0x556c89943850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 0x556c89943850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1923027478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 msgr2=0x556c89928010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89928010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 -- 172.21.15.143:0/1923027478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c89929f80 con 0x556c899215d0 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89928010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dff7fe640 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 0x556c89943850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c899215d0 0x556c899382e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8df0098080 tx=0x7f8df00ccd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae1fa97089177b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.214+0000 7f8de4ff9640 1 -- 172.21.15.143:0/1923027478 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00b9070 con 0x556c899215d0 2024-09-10T10:49:18.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.215+0000 7f8de4ff9640 1 -- 172.21.15.143:0/1923027478 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df00a4890 con 0x556c899215d0 2024-09-10T10:49:18.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.215+0000 7f8de4ff9640 1 -- 172.21.15.143:0/1923027478 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df009b2d0 con 0x556c899215d0 2024-09-10T10:49:18.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.215+0000 7f8e0780d740 1 -- 172.21.15.143:0/1923027478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c899215d0 msgr2=0x556c899382e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.215+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c899215d0 0x556c899382e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8df0098080 tx=0x7f8df00ccd90 comp rx=0 tx=0).stop 2024-09-10T10:49:18.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.215+0000 7f8e0780d740 1 -- 172.21.15.143:0/1923027478 shutdown_connections 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c8992c370 0x556c89943850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89928010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 --2- 172.21.15.143:0/1923027478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c899215d0 0x556c899382e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 -- 172.21.15.143:0/1923027478 >> 172.21.15.143:0/1923027478 conn(0x556c898ea310 msgr2=0x556c89927900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 -- 172.21.15.143:0/1923027478 shutdown_connections 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 -- 172.21.15.143:0/1923027478 wait complete. 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 -- start start 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.216+0000 7f8e0780d740 1 Processor -- start 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89934ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89925cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89926200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001b6510 con 0x556c89922b40 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001b5e40 con 0x556c8992c370 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001b3080 con 0x556c899215d0 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8e0780d740 10 client.0 _ll_register_callbacks cb 0x7f8df0097880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89925cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89934ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89934ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47754/0 (socket says 172.21.15.143:47754) 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 -- 172.21.15.143:0/3324388330 learned_addr learned my addr 172.21.15.143:0/3324388330 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89925cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44278/0 (socket says 172.21.15.143:44278) 2024-09-10T10:49:18.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dff7fe640 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89926200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 -- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 msgr2=0x556c89926200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89926200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 -- 172.21.15.143:0/3324388330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 msgr2=0x556c89925cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89925cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 -- 172.21.15.143:0/3324388330 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c89926a30 con 0x556c899215d0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8dfeffd640 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89934ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c6a0 tx=0x7f8df00a0110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e78c6c49ca2b20b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.217+0000 7f8de2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3324388330 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00a9060 con 0x556c899215d0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3324388330 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8df0010430 con 0x556c899215d0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df00a4890 con 0x556c899215d0 2024-09-10T10:49:18.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3324388330 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8df00b9040 con 0x556c899215d0 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8e0780d740 10 client.25849 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c899a6760 con 0x556c899215d0 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3324388330 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df0010650 con 0x556c899215d0 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.218+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3324388330 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f8df00b4070 con 0x556c899215d0 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 10 client.25849 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 10 client.25849 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 20 client.25849 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c899a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 4 client.25849 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8df00a4890 con 0x556c8993bd40 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c899a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8de2ff5640 10 client.25849 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8dfe7fc640 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c899a8e40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17dac17dd63fc28b server_cookie=5eed210f4877e4f3 in_seq=0 out_seq=0 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8de2ff5640 1 -- 172.21.15.143:0/3324388330 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8df00a4890 con 0x556c8993bd40 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8de2ff5640 10 client.25849 handle_command_reply: tid=0 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 client.25849 shutdown 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 msgr2=0x556c89934ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89934ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c6a0 tx=0x7f8df00a0110 comp rx=0 tx=0).stop 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 shutdown_connections 2024-09-10T10:49:18.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556c8993bd40 0x556c899a8e40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c8992c370 0x556c89926200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556c89922b40 0x556c89925cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 --2- 172.21.15.143:0/3324388330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556c899215d0 0x556c89934ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8de2ff5640 0 client.25849 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.219+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 >> 172.21.15.143:0/3324388330 conn(0x556c898ea310 msgr2=0x556c89928120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 shutdown_connections 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e0780d740 1 -- 172.21.15.143:0/3324388330 wait complete. 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e0780d740 20 client.25849 trim_cache size 0 max 16384 2024-09-10T10:49:18.231 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:49:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8de80a2b80 msgr2=0x7f8de80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8de80a2b80 0x7f8de80a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8df4031440 tx=0x7f8df4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 msgr2=0x7f8e00100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 0x7f8e00100990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8df000e9f0 tx=0x7f8df000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 shutdown_connections 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8de80a2b80 0x7f8de80a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e001082a0 0x7f8e00101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e001036e0 0x7f8e00100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 --2- 172.21.15.143:0/1319381482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e00102d30 0x7f8e00100990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.220+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 >> 172.21.15.143:0/1319381482 conn(0x7f8e000fc7d0 msgr2=0x7f8e00105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.221+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 shutdown_connections 2024-09-10T10:49:18.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:18.221+0000 7f8e05434640 1 -- 172.21.15.143:0/1319381482 wait complete. 2024-09-10T10:49:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:18 smithi178 ceph-mon[34813]: pgmap v478: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2024-09-10T10:49:18.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:18 smithi144 ceph-mon[34230]: pgmap v478: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2024-09-10T10:49:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:18 smithi143 ceph-mon[31057]: pgmap v478: 129 pgs: 129 active+clean; 397 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2024-09-10T10:49:19.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:49:19.459 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:19.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:19 smithi178 ceph-mon[34813]: pgmap v479: 129 pgs: 129 active+clean; 401 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-09-10T10:49:19.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:19 smithi144 ceph-mon[34230]: pgmap v479: 129 pgs: 129 active+clean; 401 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-09-10T10:49:19.974 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:19 smithi143 ceph-mon[31057]: pgmap v479: 129 pgs: 129 active+clean; 401 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-09-10T10:49:20.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 -- 172.21.15.143:0/935498208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4112a20 msgr2=0x7f31c4114e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/935498208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4112a20 0x7f31c4114e10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009ca0 tx=0x7f31bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 -- 172.21.15.143:0/935498208 shutdown_connections 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/935498208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4112a20 0x7f31c4114e10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/935498208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c4077040 0x7f31c41124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/935498208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 0x7f31c4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 -- 172.21.15.143:0/935498208 >> 172.21.15.143:0/935498208 conn(0x7f31c40ffac0 msgr2=0x7f31c4101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 -- 172.21.15.143:0/935498208 shutdown_connections 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.084+0000 7f31cc0f6640 1 -- 172.21.15.143:0/935498208 wait complete. 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 Processor -- start 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 -- start start 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 0x7f31c41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 0x7f31c41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c4112a20 0x7f31c419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f31c4118430 con 0x7f31c4076720 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41182b0 con 0x7f31c4112a20 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31cc0f6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41185b0 con 0x7f31c4077040 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 0x7f31c41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31c9691640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 0x7f31c41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 0x7f31c41a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44290/0 (socket says 172.21.15.143:44290) 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31c9e92640 1 -- 172.21.15.143:0/242903796 learned_addr learned my addr 172.21.15.143:0/242903796 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31c9691640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 0x7f31c41a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47770/0 (socket says 172.21.15.143:47770) 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.085+0000 7f31ca693640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c4112a20 0x7f31c419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31c9691640 1 -- 172.21.15.143:0/242903796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c4112a20 msgr2=0x7f31c419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31c9691640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c4112a20 0x7f31c419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31c9691640 1 -- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 msgr2=0x7f31c41a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31c9691640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 0x7f31c41a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31c9691640 1 -- 172.21.15.143:0/242903796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c419efe0 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31c9691640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 0x7f31c41a4c00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f31c000e9f0 tx=0x7f31c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31baffd640 1 -- 172.21.15.143:0/242903796 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c000ccc0 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c419f2d0 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31baffd640 1 -- 172.21.15.143:0/242903796 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31c00108e0 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31baffd640 1 -- 172.21.15.143:0/242903796 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0018880 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.086+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c41ab520 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.089+0000 7f31baffd640 1 -- 172.21.15.143:0/242903796 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f31c0010a90 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31baffd640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f31b00a2c00 0x7f31b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 Processor -- start 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31c9e92640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f31b00a2c00 0x7f31b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31baffd640 1 -- 172.21.15.143:0/242903796 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f31c00cf620 con 0x7f31c4077040 2024-09-10T10:49:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 -- start start 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31c9e92640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f31b00a2c00 0x7f31b00a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009c90 tx=0x7f31b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2c5d0 0x55c489f2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f37370 0x55c489f39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c489e922f0 con 0x55c489f37370 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c489e39cf0 con 0x55c489f2db40 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.090+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c489e8e2b0 con 0x55c489f2c5d0 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f2c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42588/0 (socket says 172.21.15.143:42588) 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9691640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2c5d0 0x55c489f2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 -- 172.21.15.143:0/1339019508 learned_addr learned my addr 172.21.15.143:0/1339019508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31ca693640 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f37370 0x55c489f39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 -- 172.21.15.143:0/1339019508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2c5d0 msgr2=0x55c489f2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2c5d0 0x55c489f2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 -- 172.21.15.143:0/1339019508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f37370 msgr2=0x55c489f39760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f37370 0x55c489f39760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 -- 172.21.15.143:0/1339019508 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c489e1e3a0 con 0x55c489f2db40 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31c9e92640 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f2c090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009fc0 tx=0x7f31b4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b707ce23e6dc7d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.091+0000 7f31affff640 1 -- 172.21.15.143:0/1339019508 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b402b070 con 0x55c489f2db40 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.092+0000 7f31affff640 1 -- 172.21.15.143:0/1339019508 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31b4026ad0 con 0x55c489f2db40 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.092+0000 7f31affff640 1 -- 172.21.15.143:0/1339019508 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b400ea90 con 0x55c489f2db40 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.092+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1339019508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 msgr2=0x55c489f2c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.092+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f2c090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009fc0 tx=0x7f31b4014000 comp rx=0 tx=0).stop 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1339019508 shutdown_connections 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f37370 0x55c489f39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2c5d0 0x55c489f2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1339019508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f2c090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1339019508 >> 172.21.15.143:0/1339019508 conn(0x55c489ef5310 msgr2=0x55c489f28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1339019508 shutdown_connections 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1339019508 wait complete. 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.093+0000 7f31ce4cf740 1 -- start start 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 Processor -- start 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f36d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 0x55c489f33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 0x55c489f345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c489f3a0c0 con 0x55c489f2c5d0 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c489e4d060 con 0x55c489f37370 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c489f3a240 con 0x55c489f2db40 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.094+0000 7f31ce4cf740 10 client.0 _ll_register_callbacks cb 0x55c489ba08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f36d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 0x55c489f33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31ca693640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 0x55c489f345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f36d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44318/0 (socket says 172.21.15.143:44318) 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 0x55c489f33fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47792/0 (socket says 172.21.15.143:47792) 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31ca693640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 0x55c489f345d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42590/0 (socket says 172.21.15.143:42590) 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9e92640 1 -- 172.21.15.143:0/2650301785 learned_addr learned my addr 172.21.15.143:0/2650301785 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 -- 172.21.15.143:0/2650301785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 msgr2=0x55c489f345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 0x55c489f345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 -- 172.21.15.143:0/2650301785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 msgr2=0x55c489f36d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f36d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 -- 172.21.15.143:0/2650301785 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c489c033c0 con 0x55c489f2db40 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31ca693640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 0x55c489f345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9e92640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31c9691640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 0x55c489f33fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31c009bb00 tx=0x7f31c0010f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d9b0e6c474b14e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2650301785 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00a4730 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.095+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31b4026ad0 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2650301785 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f31c00cf300 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2650301785 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f31c0010430 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 10 client.25861 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c489f45b70 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2650301785 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31b4026ad0 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2650301785 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f31c009aa30 con 0x55c489f2db40 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 10 client.25861 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 20 client.25861 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489f46f80 0x55c489f493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 4 client.25861 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c489e9f860 con 0x55c489f46f80 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.096+0000 7f31c9e92640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489f46f80 0x55c489f493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.097+0000 7f31ad7fa640 10 client.25861 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.097+0000 7f31c9e92640 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489f46f80 0x55c489f493e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a56cfd6699f3f5 server_cookie=dbe7887e0bc9607a in_seq=0 out_seq=0 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.097+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2650301785 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c489e9f860 con 0x55c489f46f80 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.097+0000 7f31ad7fa640 10 client.25861 handle_command_reply: tid=0 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.097+0000 7f31ce4cf740 1 client.25861 shutdown 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 msgr2=0x55c489f33fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 0x55c489f33fe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31c009bb00 tx=0x7f31c0010f40 comp rx=0 tx=0).stop 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 shutdown_connections 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f37370 0x55c489f345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489f46f80 0x55c489f493e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f2db40 0x55c489f33fe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2650301785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ad7fa640 0 client.25861 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 >> 172.21.15.143:0/2650301785 conn(0x55c489ef5310 msgr2=0x55c489f2e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 shutdown_connections 2024-09-10T10:49:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2650301785 wait complete. 2024-09-10T10:49:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.098+0000 7f31ce4cf740 20 client.25861 trim_cache size 0 max 16384 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 Processor -- start 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 -- start start 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2c5d0 0x55c489f338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 0x55c489f2f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f2f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b51d0 con 0x55c489f2db40 2024-09-10T10:49:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b6d40 con 0x55c489f2c5d0 2024-09-10T10:49:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b58a0 con 0x55c489f37370 2024-09-10T10:49:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31c9691640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 0x55c489f2f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ca693640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f2f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31c9691640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 0x55c489f2f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44326/0 (socket says 172.21.15.143:44326) 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31ca693640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f2f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47802/0 (socket says 172.21.15.143:47802) 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31c9691640 1 -- 172.21.15.143:0/1364435418 learned_addr learned my addr 172.21.15.143:0/1364435418 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.118+0000 7f31c9e92640 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2c5d0 0x55c489f338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31c9691640 1 -- 172.21.15.143:0/1364435418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 msgr2=0x55c489f2f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31c9691640 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f2f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31c9691640 1 -- 172.21.15.143:0/1364435418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2c5d0 msgr2=0x55c489f338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31c9691640 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2c5d0 0x55c489f338a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31c9691640 1 -- 172.21.15.143:0/1364435418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c489f32f90 con 0x55c489f2db40 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31ca693640 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f2f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31c9691640 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 0x55c489f2f120 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f31c009b870 tx=0x7f31c009ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de2931cd4c826c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31af7fe640 1 -- 172.21.15.143:0/1364435418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c009b4d0 con 0x55c489f2db40 2024-09-10T10:49:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31af7fe640 1 -- 172.21.15.143:0/1364435418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31c00b4070 con 0x55c489f2db40 2024-09-10T10:49:20.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.119+0000 7f31af7fe640 1 -- 172.21.15.143:0/1364435418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00a1120 con 0x55c489f2db40 2024-09-10T10:49:20.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1364435418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 msgr2=0x55c489f2f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 0x55c489f2f120 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f31c009b870 tx=0x7f31c009ba70 comp rx=0 tx=0).stop 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1364435418 shutdown_connections 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f2f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2db40 0x55c489f2f120 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/1364435418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2c5d0 0x55c489f338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1364435418 >> 172.21.15.143:0/1364435418 conn(0x55c489ef5310 msgr2=0x55c489f36060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1364435418 shutdown_connections 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 -- 172.21.15.143:0/1364435418 wait complete. 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.120+0000 7f31ce4cf740 1 -- start start 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 Processor -- start 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f3b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f3b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b6640 con 0x55c489f2c5d0 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b5f70 con 0x55c489f2db40 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f31c41b31b0 con 0x55c489f37370 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ce4cf740 10 client.0 _ll_register_callbacks cb 0x7f31c000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f3b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31c9e92640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f3b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44342/0 (socket says 172.21.15.143:44342) 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31c9e92640 1 -- 172.21.15.143:0/2262868655 learned_addr learned my addr 172.21.15.143:0/2262868655 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31ca693640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.121+0000 7f31c9691640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f3b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9e92640 1 -- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 msgr2=0x55c489f3be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9e92640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f3be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9e92640 1 -- 172.21.15.143:0/2262868655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 msgr2=0x55c489f3b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9e92640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f3b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9e92640 1 -- 172.21.15.143:0/2262868655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c489f370a0 con 0x55c489f2c5d0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9691640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f3b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31ca693640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f3be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31c9e92640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f3b380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f31b4011ca0 tx=0x7f31b4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4adb6f2feffb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2262868655 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4042060 con 0x55c489f2c5d0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2262868655 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f31b400e6a0 con 0x55c489f2c5d0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.122+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c00b4070 con 0x55c489f2c5d0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2262868655 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f31b402b040 con 0x55c489f2c5d0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 10 client.16461 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c489fb1ed0 con 0x55c489f2c5d0 2024-09-10T10:49:20.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2262868655 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31c00b4070 con 0x55c489f2c5d0 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2262868655 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f31b403a070 con 0x55c489f2c5d0 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 10 client.16461 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 20 client.16461 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489fb2350 0x55c489fb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 4 client.16461 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:49:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c489e99d70 con 0x55c489fb2350 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.123+0000 7f31c9691640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489fb2350 0x55c489fb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ad7fa640 10 client.16461 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31c9691640 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489fb2350 0x55c489fb4770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8af1f3a5fbe48a server_cookie=92c8ef62c0973dba in_seq=0 out_seq=0 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ad7fa640 1 -- 172.21.15.143:0/2262868655 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c489e99d70 con 0x55c489fb2350 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ad7fa640 10 client.16461 handle_command_reply: tid=0 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 client.16461 shutdown 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 msgr2=0x55c489f3b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f3b380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f31b4011ca0 tx=0x7f31b4005ab0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 shutdown_connections 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55c489fb2350 0x55c489fb4770 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c489f37370 0x55c489f3be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c489f2db40 0x55c489f3b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 --2- 172.21.15.143:0/2262868655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c489f2c5d0 0x55c489f3b380 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.124+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 >> 172.21.15.143:0/2262868655 conn(0x55c489ef5310 msgr2=0x55c489f36060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31ad7fa640 0 client.16461 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:20.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 shutdown_connections 2024-09-10T10:49:20.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31ce4cf740 1 -- 172.21.15.143:0/2262868655 wait complete. 2024-09-10T10:49:20.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31ce4cf740 20 client.16461 trim_cache size 0 max 16384 2024-09-10T10:49:20.131 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:49:20.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f31b00a2c00 msgr2=0x7f31b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f31b00a2c00 0x7f31b00a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009c90 tx=0x7f31b4005f90 comp rx=0 tx=0).stop 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 msgr2=0x7f31c41a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.125+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 0x7f31c41a4c00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f31c000e9f0 tx=0x7f31c000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 shutdown_connections 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f31b00a2c00 0x7f31b00a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c4112a20 0x7f31c419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c4077040 0x7f31c41a4c00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 --2- 172.21.15.143:0/242903796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31c4076720 0x7f31c41a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 >> 172.21.15.143:0/242903796 conn(0x7f31c40ffac0 msgr2=0x7f31c41140a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 shutdown_connections 2024-09-10T10:49:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:20.126+0000 7f31cc0f6640 1 -- 172.21.15.143:0/242903796 wait complete. 2024-09-10T10:49:22.351 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:49:22.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:49:22.703 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:22.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:22 smithi143 ceph-mon[31057]: pgmap v480: 129 pgs: 129 active+clean; 401 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1008 KiB/s wr, 384 op/s 2024-09-10T10:49:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:22 smithi178 ceph-mon[34813]: pgmap v480: 129 pgs: 129 active+clean; 401 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1008 KiB/s wr, 384 op/s 2024-09-10T10:49:22.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:22 smithi144 ceph-mon[34230]: pgmap v480: 129 pgs: 129 active+clean; 401 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1008 KiB/s wr, 384 op/s 2024-09-10T10:49:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.255+0000 7f268d96e640 1 -- 172.21.15.143:0/952862874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688100a90 msgr2=0x7f2688100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.255+0000 7f268d96e640 1 --2- 172.21.15.143:0/952862874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688100a90 0x7f2688100ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d940 tx=0x7f267c02c3b0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.255+0000 7f268d96e640 1 -- 172.21.15.143:0/952862874 shutdown_connections 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.255+0000 7f268d96e640 1 --2- 172.21.15.143:0/952862874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 0x7f268810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.255+0000 7f268d96e640 1 --2- 172.21.15.143:0/952862874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688100a90 0x7f2688100ef0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.255+0000 7f268d96e640 1 --2- 172.21.15.143:0/952862874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688106a90 0x7f2688106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.256+0000 7f268d96e640 1 -- 172.21.15.143:0/952862874 >> 172.21.15.143:0/952862874 conn(0x7f26880fc7d0 msgr2=0x7f26880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.256+0000 7f268d96e640 1 -- 172.21.15.143:0/952862874 shutdown_connections 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.256+0000 7f268d96e640 1 -- 172.21.15.143:0/952862874 wait complete. 2024-09-10T10:49:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.256+0000 7f268d96e640 1 Processor -- start 2024-09-10T10:49:23.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 -- start start 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 0x7f26881961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 0x7f2688196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688106a90 0x7f268819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f268810fc30 con 0x7f2688106a90 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f268810fab0 con 0x7f2688101430 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f268d96e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f268810fdb0 con 0x7f2688100a90 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 0x7f26881961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 0x7f26881961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47826/0 (socket says 172.21.15.143:47826) 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 -- 172.21.15.143:0/2396607763 learned_addr learned my addr 172.21.15.143:0/2396607763 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f26877fe640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688106a90 0x7f268819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f26867fc640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 0x7f2688196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 -- 172.21.15.143:0/2396607763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 msgr2=0x7f2688196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 0x7f2688196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 -- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688106a90 msgr2=0x7f268819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688106a90 0x7f268819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f2686ffd640 1 -- 172.21.15.143:0/2396607763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268819b180 con 0x7f2688100a90 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.257+0000 7f26867fc640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 0x7f2688196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.258+0000 7f2686ffd640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 0x7f26881961d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9f0 tx=0x7f267800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.258+0000 7f268c96c640 1 -- 172.21.15.143:0/2396607763 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267800cd80 con 0x7f2688100a90 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.258+0000 7f268c96c640 1 -- 172.21.15.143:0/2396607763 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26780054d0 con 0x7f2688100a90 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.258+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268819b470 con 0x7f2688100a90 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.258+0000 7f268c96c640 1 -- 172.21.15.143:0/2396607763 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678010490 con 0x7f2688100a90 2024-09-10T10:49:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.258+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881a7160 con 0x7f2688100a90 2024-09-10T10:49:23.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.261+0000 7f268c96c640 1 -- 172.21.15.143:0/2396607763 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2678005020 con 0x7f2688100a90 2024-09-10T10:49:23.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268c96c640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f26700a2c80 0x7f26700a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 Processor -- start 2024-09-10T10:49:23.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f26867fc640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f26700a2c80 0x7f26700a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 -- start start 2024-09-10T10:49:23.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 0x562ccfbc5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc6480 0x562ccfbd0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfbd3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562ccfb2b280 con 0x562ccfbc5b60 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ccfad2b10 con 0x562ccfbc6480 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.262+0000 7f268fd47740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ccfb27240 con 0x562ccfbd0d40 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f268c96c640 1 -- 172.21.15.143:0/2396607763 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f26780cf770 con 0x7f2688100a90 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 0x562ccfbc5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f26877fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfbd3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 0x562ccfbc5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44354/0 (socket says 172.21.15.143:44354) 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 -- 172.21.15.143:0/346469774 learned_addr learned my addr 172.21.15.143:0/346469774 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 -- 172.21.15.143:0/346469774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 msgr2=0x562ccfbd3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfbd3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 -- 172.21.15.143:0/346469774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc6480 msgr2=0x562ccfbd0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f26867fc640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f26700a2c80 0x7f26700a5140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f267c006010 tx=0x7f267c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc6480 0x562ccfbd0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 -- 172.21.15.143:0/346469774 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ccfadf1f0 con 0x562ccfbc5b60 2024-09-10T10:49:23.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f2686ffd640 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 0x562ccfbc5f40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f26780985e0 tx=0x7f267809b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a03efbee8f9926 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f266dffb640 1 -- 172.21.15.143:0/346469774 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267809c310 con 0x562ccfbc5b60 2024-09-10T10:49:23.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f266dffb640 1 -- 172.21.15.143:0/346469774 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267809bc10 con 0x562ccfbc5b60 2024-09-10T10:49:23.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.263+0000 7f266dffb640 1 -- 172.21.15.143:0/346469774 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678098280 con 0x562ccfbc5b60 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.264+0000 7f268fd47740 1 -- 172.21.15.143:0/346469774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 msgr2=0x562ccfbc5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.264+0000 7f268fd47740 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 0x562ccfbc5f40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f26780985e0 tx=0x7f267809b630 comp rx=0 tx=0).stop 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 -- 172.21.15.143:0/346469774 shutdown_connections 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfbd3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc6480 0x562ccfbd0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 --2- 172.21.15.143:0/346469774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc5b60 0x562ccfbc5f40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 -- 172.21.15.143:0/346469774 >> 172.21.15.143:0/346469774 conn(0x562ccfbc20f0 msgr2=0x562ccfbc24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 -- 172.21.15.143:0/346469774 shutdown_connections 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 -- 172.21.15.143:0/346469774 wait complete. 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 -- start start 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.265+0000 7f268fd47740 1 Processor -- start 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbcdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562ccfbd3910 con 0x562ccfbc6480 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ccfae5f10 con 0x562ccfbd0d40 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ccfbd3a90 con 0x562ccfbc5b60 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbcdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f268fd47740 10 client.0 _ll_register_callbacks cb 0x562ccfaaea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbcdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47854/0 (socket says 172.21.15.143:47854) 2024-09-10T10:49:23.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 -- 172.21.15.143:0/591125254 learned_addr learned my addr 172.21.15.143:0/591125254 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f26867fc640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 -- 172.21.15.143:0/591125254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 msgr2=0x562ccfbce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 -- 172.21.15.143:0/591125254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 msgr2=0x562ccfbce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 -- 172.21.15.143:0/591125254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccf89c6a0 con 0x562ccfbc5b60 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f26867fc640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2686ffd640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbcdc00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2678098020 tx=0x7f26780cf650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddce128911112c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.266+0000 7f2666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f2666ffd640 1 -- 172.21.15.143:0/591125254 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780a8070 con 0x562ccfbc5b60 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f2666ffd640 1 -- 172.21.15.143:0/591125254 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f26780a43f0 con 0x562ccfbc5b60 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267809bc10 con 0x562ccfbc5b60 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f2666ffd640 1 -- 172.21.15.143:0/591125254 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26780ba050 con 0x562ccfbc5b60 2024-09-10T10:49:23.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 10 client.25879 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ccfbd3910 con 0x562ccfbc5b60 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f2666ffd640 1 -- 172.21.15.143:0/591125254 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267809bc10 con 0x562ccfbc5b60 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f2666ffd640 1 -- 172.21.15.143:0/591125254 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f26780a4ad0 con 0x562ccfbc5b60 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 10 client.25879 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 20 client.25879 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfbe0670 0x562ccfbe2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 4 client.25879 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.267+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ccfb387f0 con 0x562ccfbe0670 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.268+0000 7f26867fc640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfbe0670 0x562ccfbe2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.268+0000 7f26867fc640 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfbe0670 0x562ccfbe2a60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822525eda127d031 server_cookie=6e5aa0ea468bfb14 in_seq=0 out_seq=0 2024-09-10T10:49:23.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.268+0000 7f2666ffd640 10 client.25879 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:23.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f2666ffd640 1 -- 172.21.15.143:0/591125254 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ccfb387f0 con 0x562ccfbe0670 2024-09-10T10:49:23.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f2666ffd640 10 client.25879 handle_command_reply: tid=0 2024-09-10T10:49:23.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 client.25879 shutdown 2024-09-10T10:49:23.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 msgr2=0x562ccfbcdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbcdc00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2678098020 tx=0x7f26780cf650 comp rx=0 tx=0).stop 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 shutdown_connections 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfbe0670 0x562ccfbe2a60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 --2- 172.21.15.143:0/591125254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbcdc00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 >> 172.21.15.143:0/591125254 conn(0x562ccfbc20f0 msgr2=0x562ccfbc1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f2666ffd640 0 client.25879 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 shutdown_connections 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.269+0000 7f268fd47740 1 -- 172.21.15.143:0/591125254 wait complete. 2024-09-10T10:49:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.270+0000 7f268fd47740 20 client.25879 trim_cache size 0 max 16384 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.294+0000 7f268fd47740 1 Processor -- start 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.294+0000 7f268fd47740 1 -- start start 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.294+0000 7f268fd47740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbc85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f268fd47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbdc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f268fd47740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbe56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f268fd47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b0df0 con 0x562ccfbc6480 2024-09-10T10:49:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f268fd47740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b2960 con 0x562ccfbd0d40 2024-09-10T10:49:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f268fd47740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b14c0 con 0x562ccfbc5b60 2024-09-10T10:49:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f26867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbdc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbc85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f26877fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbe56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f26867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbdc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44370/0 (socket says 172.21.15.143:44370) 2024-09-10T10:49:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f26867fc640 1 -- 172.21.15.143:0/1443562181 learned_addr learned my addr 172.21.15.143:0/1443562181 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 -- 172.21.15.143:0/1443562181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 msgr2=0x562ccfbe56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbe56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 -- 172.21.15.143:0/1443562181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 msgr2=0x562ccfbdc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbdc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 -- 172.21.15.143:0/1443562181 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ccfbc62b0 con 0x562ccfbc5b60 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.295+0000 7f2686ffd640 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbc85c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2678098070 tx=0x7f26780048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab3cd2679f5c6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.296+0000 7f266d7fa640 1 -- 172.21.15.143:0/1443562181 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780a8030 con 0x562ccfbc5b60 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.296+0000 7f266d7fa640 1 -- 172.21.15.143:0/1443562181 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267809c310 con 0x562ccfbc5b60 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.296+0000 7f266d7fa640 1 -- 172.21.15.143:0/1443562181 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780a4ad0 con 0x562ccfbc5b60 2024-09-10T10:49:23.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 -- 172.21.15.143:0/1443562181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 msgr2=0x562ccfbc85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbc85c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2678098070 tx=0x7f26780048d0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 -- 172.21.15.143:0/1443562181 shutdown_connections 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbd0d40 0x562ccfbe56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbdc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 --2- 172.21.15.143:0/1443562181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbc5b60 0x562ccfbc85c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 -- 172.21.15.143:0/1443562181 >> 172.21.15.143:0/1443562181 conn(0x562ccfbc20f0 msgr2=0x562ccfbdbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 -- 172.21.15.143:0/1443562181 shutdown_connections 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.297+0000 7f268fd47740 1 -- 172.21.15.143:0/1443562181 wait complete. 2024-09-10T10:49:23.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.298+0000 7f268fd47740 1 -- start start 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.298+0000 7f268fd47740 1 Processor -- start 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f268fd47740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc5b60 0x562ccfbe2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f268fd47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbe3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f26867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbe3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f26867fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbe3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44378/0 (socket says 172.21.15.143:44378) 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc5b60 0x562ccfbe2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc5b60 0x562ccfbe2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:42678/0 (socket says 172.21.15.143:42678) 2024-09-10T10:49:23.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f268fd47740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfc4a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f26867fc640 1 -- 172.21.15.143:0/750412263 learned_addr learned my addr 172.21.15.143:0/750412263 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.299+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b2260 con 0x562ccfbc6480 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b1b90 con 0x562ccfbc5b60 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26881aedd0 con 0x562ccfbd0d40 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfc4a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f268fd47740 10 client.0 _ll_register_callbacks cb 0x7f2680009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 -- 172.21.15.143:0/750412263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc5b60 msgr2=0x562ccfbe2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc5b60 0x562ccfbe2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 -- 172.21.15.143:0/750412263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 msgr2=0x562ccfbe3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbe3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccfc4ae10 con 0x562ccfbd0d40 2024-09-10T10:49:23.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26867fc640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbe3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267809c310 con 0x562ccfbd0d40 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f26877fe640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfc4a770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x562ccfbc3470 tx=0x7f2680005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfab2735fcc2205 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.300+0000 7f2666ffd640 10 client.25885 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f2666ffd640 1 -- 172.21.15.143:0/750412263 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680003d80 con 0x562ccfbd0d40 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f2666ffd640 1 -- 172.21.15.143:0/750412263 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2680004050 con 0x562ccfbd0d40 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f2666ffd640 1 -- 172.21.15.143:0/750412263 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2680004290 con 0x562ccfbd0d40 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 10 client.25885 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=32}) -- 0x562ccfc53320 con 0x562ccfbd0d40 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f2666ffd640 1 -- 172.21.15.143:0/750412263 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f2680004430 con 0x562ccfbd0d40 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 20 client.25885 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:23.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfc539d0 0x562ccfc55dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 4 client.25885 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:49:23.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ccfb32d00 con 0x562ccfc539d0 2024-09-10T10:49:23.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.301+0000 7f2686ffd640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfc539d0 0x562ccfc55dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.302+0000 7f2686ffd640 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfc539d0 0x562ccfc55dc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f01190f8624d015 server_cookie=3b6321bfe4edc1b7 in_seq=0 out_seq=0 2024-09-10T10:49:23.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.302+0000 7f2666ffd640 10 client.25885 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:23.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f2666ffd640 1 -- 172.21.15.143:0/750412263 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ccfb32d00 con 0x562ccfc539d0 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f2666ffd640 10 client.25885 handle_command_reply: tid=0 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 client.25885 shutdown 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 msgr2=0x562ccfc4a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfc4a770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x562ccfbc3470 tx=0x7f2680005410 comp rx=0 tx=0).stop 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 shutdown_connections 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ccfc539d0 0x562ccfc55dc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ccfbd0d40 0x562ccfc4a770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ccfbc6480 0x562ccfbe3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 --2- 172.21.15.143:0/750412263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccfbc5b60 0x562ccfbe2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 >> 172.21.15.143:0/750412263 conn(0x562ccfbc20f0 msgr2=0x562ccfbdc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f2666ffd640 0 client.25885 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 shutdown_connections 2024-09-10T10:49:23.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 1 -- 172.21.15.143:0/750412263 wait complete. 2024-09-10T10:49:23.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.303+0000 7f268fd47740 20 client.25885 trim_cache size 0 max 16384 2024-09-10T10:49:23.308 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:23.308 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:49:23.308 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "8da4408a-e433-47a1-842d-bae134b4da51", 2024-09-10T10:49:23.308 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:49:23.308 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:23.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.304+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f26700a2c80 msgr2=0x7f26700a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.304+0000 7f268d96e640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f26700a2c80 0x7f26700a5140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f267c006010 tx=0x7f267c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.304+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 msgr2=0x7f26881961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.304+0000 7f268d96e640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 0x7f26881961d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9f0 tx=0x7f267800edf0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 shutdown_connections 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f26700a2c80 0x7f26700a5140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2688106a90 0x7f268819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2688101430 0x7f2688196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 --2- 172.21.15.143:0/2396607763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2688100a90 0x7f26881961d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 >> 172.21.15.143:0/2396607763 conn(0x7f26880fc7d0 msgr2=0x7f26880fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 shutdown_connections 2024-09-10T10:49:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:23.305+0000 7f268d96e640 1 -- 172.21.15.143:0/2396607763 wait complete. 2024-09-10T10:49:23.591 INFO:tasks.workunit.client.0.smithi143.stdout: 75 7001 0 4589 0 2875 0 2024-09-10T10:49:23.889 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:23 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:49:23.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:23 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:49:23.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:23 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:49:24.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:49:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:24 smithi178 ceph-mon[34813]: pgmap v481: 129 pgs: 129 active+clean; 402 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 969 KiB/s wr, 619 op/s 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:24 smithi144 ceph-mon[34230]: pgmap v481: 129 pgs: 129 active+clean; 402 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 969 KiB/s wr, 619 op/s 2024-09-10T10:49:25.002 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:24 smithi143 ceph-mon[31057]: pgmap v481: 129 pgs: 129 active+clean; 402 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 969 KiB/s wr, 619 op/s 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 -- 172.21.15.143:0/1966796689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c109100 msgr2=0x7f989c110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 --2- 172.21.15.143:0/1966796689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c109100 0x7f989c110c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9894009ca0 tx=0x7f989402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 -- 172.21.15.143:0/1966796689 shutdown_connections 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 --2- 172.21.15.143:0/1966796689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c109100 0x7f989c110c80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 --2- 172.21.15.143:0/1966796689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 0x7f989c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 --2- 172.21.15.143:0/1966796689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c103fd0 0x7f989c1043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 -- 172.21.15.143:0/1966796689 >> 172.21.15.143:0/1966796689 conn(0x7f989c0ffa30 msgr2=0x7f989c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 -- 172.21.15.143:0/1966796689 shutdown_connections 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.704+0000 7f98a1285640 1 -- 172.21.15.143:0/1966796689 wait complete. 2024-09-10T10:49:25.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 Processor -- start 2024-09-10T10:49:25.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 -- start start 2024-09-10T10:49:25.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 0x7f989c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 0x7f989c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f989affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 0x7f989c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c109100 0x7f989c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f989c113fc0 con 0x7f989c103fd0 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989c113e40 con 0x7f989c109100 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f989affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 0x7f989c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44386/0 (socket says 172.21.15.143:44386) 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f98a1285640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f989c114140 con 0x7f989c104980 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.705+0000 7f989affd640 1 -- 172.21.15.143:0/3424661556 learned_addr learned my addr 172.21.15.143:0/3424661556 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989a7fc640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 0x7f989c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c109100 0x7f989c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989affd640 1 -- 172.21.15.143:0/3424661556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 msgr2=0x7f989c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989affd640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 0x7f989c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989affd640 1 -- 172.21.15.143:0/3424661556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c109100 msgr2=0x7f989c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989affd640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c109100 0x7f989c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989affd640 1 -- 172.21.15.143:0/3424661556 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c19f470 con 0x7f989c103fd0 2024-09-10T10:49:25.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989a7fc640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 0x7f989c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c109100 0x7f989c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.706+0000 7f989affd640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 0x7f989c19a4c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f988c00d950 tx=0x7f988c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.707+0000 7f9883fff640 1 -- 172.21.15.143:0/3424661556 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c017030 con 0x7f989c103fd0 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.707+0000 7f9883fff640 1 -- 172.21.15.143:0/3424661556 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c004020 con 0x7f989c103fd0 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.707+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c19f760 con 0x7f989c103fd0 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.707+0000 7f9883fff640 1 -- 172.21.15.143:0/3424661556 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0047a0 con 0x7f989c103fd0 2024-09-10T10:49:25.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.707+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c1a7040 con 0x7f989c103fd0 2024-09-10T10:49:25.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.710+0000 7f9883fff640 1 -- 172.21.15.143:0/3424661556 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f988c0041d0 con 0x7f989c103fd0 2024-09-10T10:49:25.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 Processor -- start 2024-09-10T10:49:25.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 -- start start 2024-09-10T10:49:25.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f9883fff640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98840a2c00 0x7f98840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c86880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 0x562732c91140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c93a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562732bec2e0 con 0x562732c86dc0 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562732b93ce0 con 0x562732c864a0 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f98a365e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562732be82a0 con 0x562732c91680 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f9883fff640 1 -- 172.21.15.143:0/3424661556 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f988c0cedf0 con 0x7f989c103fd0 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98840a2c00 0x7f98840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.711+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 0x562732c91140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 0x562732c91140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44394/0 (socket says 172.21.15.143:44394) 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 -- 172.21.15.143:0/2358671417 learned_addr learned my addr 172.21.15.143:0/2358671417 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:25.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989affd640 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c86880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989b7fe640 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c93a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 -- 172.21.15.143:0/2358671417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 msgr2=0x562732c93a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c93a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 -- 172.21.15.143:0/2358671417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 msgr2=0x562732c86880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c86880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.712+0000 7f989a7fc640 1 -- 172.21.15.143:0/2358671417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562732b78440 con 0x562732c86dc0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.713+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98840a2c00 0x7f98840a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9894004af0 tx=0x7f9894005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.713+0000 7f989a7fc640 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 0x562732c91140 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9890011470 tx=0x7f9890011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2070f0e7abecbdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.713+0000 7f98817fa640 1 -- 172.21.15.143:0/2358671417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989001e070 con 0x562732c86dc0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.713+0000 7f98817fa640 1 -- 172.21.15.143:0/2358671417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98900035f0 con 0x562732c86dc0 2024-09-10T10:49:25.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.713+0000 7f98817fa640 1 -- 172.21.15.143:0/2358671417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98900044e0 con 0x562732c86dc0 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 -- 172.21.15.143:0/2358671417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 msgr2=0x562732c91140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 0x562732c91140 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9890011470 tx=0x7f9890011930 comp rx=0 tx=0).stop 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 -- 172.21.15.143:0/2358671417 shutdown_connections 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c93a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c86dc0 0x562732c91140 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 --2- 172.21.15.143:0/2358671417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c86880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 -- 172.21.15.143:0/2358671417 >> 172.21.15.143:0/2358671417 conn(0x562732c82a30 msgr2=0x562732c82e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 -- 172.21.15.143:0/2358671417 shutdown_connections 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.714+0000 7f98a365e740 1 -- 172.21.15.143:0/2358671417 wait complete. 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 -- start start 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 Processor -- start 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 0x562732c90fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c8e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c8e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562732c94250 con 0x562732c864a0 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562732ba7050 con 0x562732c86dc0 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562732c943d0 con 0x562732c91680 2024-09-10T10:49:25.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f98a365e740 10 client.0 _ll_register_callbacks cb 0x562732be7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f989affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 0x562732c90fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.715+0000 7f989affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 0x562732c90fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44398/0 (socket says 172.21.15.143:44398) 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989affd640 1 -- 172.21.15.143:0/3089042173 learned_addr learned my addr 172.21.15.143:0/3089042173 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c8e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989a7fc640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c8e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 -- 172.21.15.143:0/3089042173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 msgr2=0x562732c8e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c8e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 -- 172.21.15.143:0/3089042173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 msgr2=0x562732c90fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 0x562732c90fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 -- 172.21.15.143:0/3089042173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273295d3c0 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989b7fe640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c8e7b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9894031040 tx=0x7f9894004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1774da8593d7607 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f989affd640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 0x562732c90fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f987eff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.716+0000 7f987eff5640 1 -- 172.21.15.143:0/3089042173 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894050020 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98900035f0 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f987eff5640 1 -- 172.21.15.143:0/3089042173 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9894004450 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f987eff5640 1 -- 172.21.15.143:0/3089042173 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f989405b050 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 10 client.25897 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562732c9fc00 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f987eff5640 1 -- 172.21.15.143:0/3089042173 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9894036040 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f987eff5640 1 -- 172.21.15.143:0/3089042173 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f989402f4c0 con 0x562732c91680 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 10 client.25897 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:25.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 20 client.25897 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732ca1330 0x562732ca3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 4 client.25897 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.717+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f98900035f0 con 0x562732ca1330 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.718+0000 7f989affd640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732ca1330 0x562732ca3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.718+0000 7f987eff5640 10 client.25897 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.718+0000 7f989affd640 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732ca1330 0x562732ca3740 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3a8d46893aa8c2 server_cookie=df8bd9cf2ee59a46 in_seq=0 out_seq=0 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f987eff5640 1 -- 172.21.15.143:0/3089042173 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f98900035f0 con 0x562732ca1330 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f987eff5640 10 client.25897 handle_command_reply: tid=0 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 client.25897 shutdown 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 msgr2=0x562732c8e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c8e7b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9894031040 tx=0x7f9894004a60 comp rx=0 tx=0).stop 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 shutdown_connections 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732ca1330 0x562732ca3740 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c91680 0x562732c8e7b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c8e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 --2- 172.21.15.143:0/3089042173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c864a0 0x562732c90fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 >> 172.21.15.143:0/3089042173 conn(0x562732c82a30 msgr2=0x562732c81b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.719+0000 7f987eff5640 0 client.25897 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.720+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 shutdown_connections 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.720+0000 7f98a365e740 1 -- 172.21.15.143:0/3089042173 wait complete. 2024-09-10T10:49:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.720+0000 7f98a365e740 20 client.25897 trim_cache size 0 max 16384 2024-09-10T10:49:25.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.743+0000 7f98a365e740 1 Processor -- start 2024-09-10T10:49:25.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.743+0000 7f98a365e740 1 -- start start 2024-09-10T10:49:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.743+0000 7f98a365e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 0x562732c9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.743+0000 7f98a365e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f98a365e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732c901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f98a365e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f989c071210 con 0x562732c91680 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f98a365e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989c072d80 con 0x562732c86dc0 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f98a365e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f989c0718e0 con 0x562732c864a0 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 0x562732c9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 0x562732c9d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47920/0 (socket says 172.21.15.143:47920) 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 -- 172.21.15.143:0/197680234 learned_addr learned my addr 172.21.15.143:0/197680234 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989b7fe640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732c901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989a7fc640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 -- 172.21.15.143:0/197680234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 msgr2=0x562732c82e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c82e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 -- 172.21.15.143:0/197680234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 msgr2=0x562732c901e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732c901e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 -- 172.21.15.143:0/197680234 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562732c8d220 con 0x562732c864a0 2024-09-10T10:49:25.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989affd640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 0x562732c9d4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f988c005170 tx=0x7f988c0bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbce8f123cef868 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989a7fc640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:25.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.744+0000 7f989b7fe640 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732c901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.745+0000 7f9880ff9640 1 -- 172.21.15.143:0/197680234 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c095e60 con 0x562732c864a0 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.745+0000 7f9880ff9640 1 -- 172.21.15.143:0/197680234 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c014040 con 0x562732c864a0 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.745+0000 7f9880ff9640 1 -- 172.21.15.143:0/197680234 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c09f3c0 con 0x562732c864a0 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.745+0000 7f98a365e740 1 -- 172.21.15.143:0/197680234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 msgr2=0x562732c9d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.745+0000 7f98a365e740 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 0x562732c9d4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f988c005170 tx=0x7f988c0bb040 comp rx=0 tx=0).stop 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 -- 172.21.15.143:0/197680234 shutdown_connections 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732c901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c86dc0 0x562732c82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 --2- 172.21.15.143:0/197680234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c864a0 0x562732c9d4b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 -- 172.21.15.143:0/197680234 >> 172.21.15.143:0/197680234 conn(0x562732c82a30 msgr2=0x562732c8fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 -- 172.21.15.143:0/197680234 shutdown_connections 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 -- 172.21.15.143:0/197680234 wait complete. 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 -- start start 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 Processor -- start 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c94b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.746+0000 7f98a365e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 0x562732c950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f98a365e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732d0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c94b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 0x562732c950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 0x562732c950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:47928/0 (socket says 172.21.15.143:47928) 2024-09-10T10:49:25.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 -- 172.21.15.143:0/697255839 learned_addr learned my addr 172.21.15.143:0/697255839 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f98a365e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f989c072680 con 0x562732c91680 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989c071fb0 con 0x562732c864a0 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f989c06f1f0 con 0x562732c86dc0 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989b7fe640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732d0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f98a365e740 10 client.0 _ll_register_callbacks cb 0x7f9894009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 -- 172.21.15.143:0/697255839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 msgr2=0x562732c94b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c94b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 -- 172.21.15.143:0/697255839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 msgr2=0x562732d0a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732d0a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 -- 172.21.15.143:0/697255839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562732d0a600 con 0x562732c86dc0 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989b7fe640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732d0a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f989a7fc640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 0x562732c950c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9890011470 tx=0x7f98900039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5784c096baee5105 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.747+0000 7f987eff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:25.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f987eff5640 1 -- 172.21.15.143:0/697255839 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890026020 con 0x562732c86dc0 2024-09-10T10:49:25.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c014040 con 0x562732c86dc0 2024-09-10T10:49:25.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f987eff5640 1 -- 172.21.15.143:0/697255839 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f98900050b0 con 0x562732c86dc0 2024-09-10T10:49:25.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f987eff5640 1 -- 172.21.15.143:0/697255839 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9890031050 con 0x562732c86dc0 2024-09-10T10:49:25.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f98a365e740 10 client.25903 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:25.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562732d0b930 con 0x562732c86dc0 2024-09-10T10:49:25.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f987eff5640 1 -- 172.21.15.143:0/697255839 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c014040 con 0x562732c86dc0 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f987eff5640 1 -- 172.21.15.143:0/697255839 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f989000fa90 con 0x562732c86dc0 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.748+0000 7f98a365e740 10 client.25903 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f98a365e740 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f98a365e740 20 client.25903 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f98a365e740 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732d0bf20 0x562732d0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f98a365e740 4 client.25903 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562732c835e0 con 0x562732d0bf20 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f989affd640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732d0bf20 0x562732d0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f987eff5640 10 client.25903 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.749+0000 7f989affd640 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732d0bf20 0x562732d0e340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9eb2f53b741cf2 server_cookie=83b7964d04fd2be9 in_seq=0 out_seq=0 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f987eff5640 1 -- 172.21.15.143:0/697255839 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562732c835e0 con 0x562732d0bf20 2024-09-10T10:49:25.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f987eff5640 10 client.25903 handle_command_reply: tid=0 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 client.25903 shutdown 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 msgr2=0x562732c950c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 0x562732c950c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9890011470 tx=0x7f98900039d0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 shutdown_connections 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562732c91680 0x562732d0a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562732d0bf20 0x562732d0e340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562732c86dc0 0x562732c950c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 --2- 172.21.15.143:0/697255839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562732c864a0 0x562732c94b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 >> 172.21.15.143:0/697255839 conn(0x562732c82a30 msgr2=0x562732c81b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f987eff5640 0 client.25903 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:25.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 shutdown_connections 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.750+0000 7f98a365e740 1 -- 172.21.15.143:0/697255839 wait complete. 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a365e740 20 client.25903 trim_cache size 0 max 16384 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout: "8da4408a-e433-47a1-842d-bae134b4da51": { 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "8da4408a-e433-47a1-842d-bae134b4da51", 2024-09-10T10:49:25.756 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:49:25.757 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:25.757 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:25.757 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:25.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98840a2c00 msgr2=0x7f98840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98840a2c00 0x7f98840a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9894004af0 tx=0x7f9894005ef0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 msgr2=0x7f989c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 0x7f989c19a4c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f988c00d950 tx=0x7f988c00de10 comp rx=0 tx=0).stop 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 shutdown_connections 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f98840a2c00 0x7f98840a50c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c109100 0x7f989c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f989c104980 0x7f989c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 --2- 172.21.15.143:0/3424661556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f989c103fd0 0x7f989c19a4c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 >> 172.21.15.143:0/3424661556 conn(0x7f989c0ffa30 msgr2=0x7f989c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.751+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 shutdown_connections 2024-09-10T10:49:25.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:25.752+0000 7f98a1285640 1 -- 172.21.15.143:0/3424661556 wait complete. 2024-09-10T10:49:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:26 smithi178 ceph-mon[34813]: pgmap v482: 129 pgs: 129 active+clean; 405 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 815 KiB/s wr, 593 op/s 2024-09-10T10:49:26.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:26 smithi144 ceph-mon[34230]: pgmap v482: 129 pgs: 129 active+clean; 405 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 815 KiB/s wr, 593 op/s 2024-09-10T10:49:27.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:26 smithi143 ceph-mon[31057]: pgmap v482: 129 pgs: 129 active+clean; 405 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 815 KiB/s wr, 593 op/s 2024-09-10T10:49:27.459 INFO:tasks.cephfs.filesystem:scrub status for tag:8da4408a-e433-47a1-842d-bae134b4da51 - {'path': '/', 'tag': '8da4408a-e433-47a1-842d-bae134b4da51', 'options': 'recursive,force'} 2024-09-10T10:49:27.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:27 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:27 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:27.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:27 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:49:27.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:49:28.296 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:28.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:28 smithi143 ceph-mon[31057]: pgmap v483: 129 pgs: 129 active+clean; 408 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 815 op/s 2024-09-10T10:49:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:28 smithi178 ceph-mon[34813]: pgmap v483: 129 pgs: 129 active+clean; 408 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 815 op/s 2024-09-10T10:49:28.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:28 smithi144 ceph-mon[34230]: pgmap v483: 129 pgs: 129 active+clean; 408 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 815 op/s 2024-09-10T10:49:28.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.936+0000 7fa85159a640 1 -- 172.21.15.143:0/697800919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440035f0 con 0x7fa854109450 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.936+0000 7fa858bca640 1 -- 172.21.15.143:0/697800919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 msgr2=0x7fa854109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.936+0000 7fa858bca640 1 --2- 172.21.15.143:0/697800919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 0x7fa854109830 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa844009900 tx=0x7fa84402a440 comp rx=0 tx=0).stop 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.940+0000 7fa858bca640 1 -- 172.21.15.143:0/697800919 shutdown_connections 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.940+0000 7fa858bca640 1 --2- 172.21.15.143:0/697800919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854074990 0x7fa85407b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.940+0000 7fa858bca640 1 --2- 172.21.15.143:0/697800919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa8540742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.940+0000 7fa858bca640 1 --2- 172.21.15.143:0/697800919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 0x7fa854109830 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.940+0000 7fa858bca640 1 -- 172.21.15.143:0/697800919 >> 172.21.15.143:0/697800919 conn(0x7fa85406e140 msgr2=0x7fa85406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:28.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.942+0000 7fa858bca640 1 -- 172.21.15.143:0/697800919 shutdown_connections 2024-09-10T10:49:28.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.948+0000 7fa858bca640 1 -- 172.21.15.143:0/697800919 wait complete. 2024-09-10T10:49:28.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 Processor -- start 2024-09-10T10:49:28.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 -- start start 2024-09-10T10:49:28.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854074990 0x7fa85407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 0x7fa854076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa854076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa85407e690 con 0x7fa854109450 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa85407e510 con 0x7fa854109e00 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.949+0000 7fa858bca640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa85407e810 con 0x7fa854074990 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.950+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa854076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.950+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa854076670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:36350/0 (socket says 172.21.15.143:36350) 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.950+0000 7fa852d9d640 1 -- 172.21.15.143:0/2018093536 learned_addr learned my addr 172.21.15.143:0/2018093536 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.951+0000 7fa851d9b640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 0x7fa854076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.952+0000 7fa85259c640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854074990 0x7fa85407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.953+0000 7fa852d9d640 1 -- 172.21.15.143:0/2018093536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854074990 msgr2=0x7fa85407b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.953+0000 7fa852d9d640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854074990 0x7fa85407b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.953+0000 7fa852d9d640 1 -- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 msgr2=0x7fa854076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.953+0000 7fa852d9d640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 0x7fa854076130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.953+0000 7fa852d9d640 1 -- 172.21.15.143:0/2018093536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa854076da0 con 0x7fa854109e00 2024-09-10T10:49:28.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.956+0000 7fa852d9d640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa854076670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa84800d750 tx=0x7fa84800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.957+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848017070 con 0x7fa854109e00 2024-09-10T10:49:28.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.957+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8480054d0 con 0x7fa854109e00 2024-09-10T10:49:28.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.957+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848004c60 con 0x7fa854109e00 2024-09-10T10:49:28.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.957+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa854130250 con 0x7fa854109e00 2024-09-10T10:49:28.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.957+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541307c0 con 0x7fa854109e00 2024-09-10T10:49:28.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.962+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa854073b00 con 0x7fa854109e00 2024-09-10T10:49:28.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.962+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa848004e00 con 0x7fa854109e00 2024-09-10T10:49:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.964+0000 7fa8437fe640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa83c0a2d90 0x7fa83c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.966+0000 7fa85259c640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa83c0a2d90 0x7fa83c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.966+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa8480cecf0 con 0x7fa854109e00 2024-09-10T10:49:28.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.967+0000 7fa85259c640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa83c0a2d90 0x7fa83c0a5250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa8440096c0 tx=0x7fa844000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:28.971+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa84809b890 con 0x7fa854109e00 2024-09-10T10:49:29.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.292+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa854080560 con 0x7fa854109e00 2024-09-10T10:49:29.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.293+0000 7fa8437fe640 1 -- 172.21.15.143:0/2018093536 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7fa84809b190 con 0x7fa854109e00 2024-09-10T10:49:29.294 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:49:29.294 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:49:29.294 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.295+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa83c0a2d90 msgr2=0x7fa83c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.295+0000 7fa858bca640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa83c0a2d90 0x7fa83c0a5250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa8440096c0 tx=0x7fa844000bc0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 msgr2=0x7fa854076670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa854076670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa84800d750 tx=0x7fa84800dc10 comp rx=0 tx=0).stop 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 shutdown_connections 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa83c0a2d90 0x7fa83c0a5250 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa854109e00 0x7fa854076670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa854109450 0x7fa854076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 --2- 172.21.15.143:0/2018093536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa854074990 0x7fa85407b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 >> 172.21.15.143:0/2018093536 conn(0x7fa85406e140 msgr2=0x7fa8540724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 shutdown_connections 2024-09-10T10:49:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:29.296+0000 7fa858bca640 1 -- 172.21.15.143:0/2018093536 wait complete. 2024-09-10T10:49:29.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:29 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2018093536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:29.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:29 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2018093536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:30.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:29 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2018093536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:30.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:30 smithi178 ceph-mon[34813]: pgmap v484: 129 pgs: 129 active+clean; 404 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 1.08k op/s 2024-09-10T10:49:30.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:30 smithi144 ceph-mon[34230]: pgmap v484: 129 pgs: 129 active+clean; 404 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 1.08k op/s 2024-09-10T10:49:31.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:30 smithi143 ceph-mon[31057]: pgmap v484: 129 pgs: 129 active+clean; 404 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 1.08k op/s 2024-09-10T10:49:32.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:32 smithi178 ceph-mon[34813]: pgmap v485: 129 pgs: 129 active+clean; 393 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.11k op/s 2024-09-10T10:49:32.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:32 smithi144 ceph-mon[34230]: pgmap v485: 129 pgs: 129 active+clean; 393 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.11k op/s 2024-09-10T10:49:33.058 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:32 smithi143 ceph-mon[31057]: pgmap v485: 129 pgs: 129 active+clean; 393 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.11k op/s 2024-09-10T10:49:33.591 INFO:tasks.workunit.client.0.smithi143.stdout: 75 10036 0 7188 0 3469 0 2024-09-10T10:49:34.840 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 544fb307 --release --await' 2024-09-10T10:49:34.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 544fb307 --release --await 2024-09-10T10:49:34.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:34 smithi143 ceph-mon[31057]: pgmap v486: 129 pgs: 129 active+clean; 397 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 1.18k op/s 2024-09-10T10:49:34.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:34 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:49:34.868 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:34 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:49:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:34 smithi178 ceph-mon[34813]: pgmap v486: 129 pgs: 129 active+clean; 397 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 1.18k op/s 2024-09-10T10:49:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:34 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:49:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:34 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:49:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:34 smithi144 ceph-mon[34230]: pgmap v486: 129 pgs: 129 active+clean; 397 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 1.18k op/s 2024-09-10T10:49:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:34 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:49:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:34 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:49:35.281 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:36 smithi178 ceph-mon[34813]: pgmap v487: 129 pgs: 129 active+clean; 397 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 1.02k op/s 2024-09-10T10:49:36.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:36 smithi144 ceph-mon[34230]: pgmap v487: 129 pgs: 129 active+clean; 397 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 1.02k op/s 2024-09-10T10:49:37.057 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:36 smithi143 ceph-mon[31057]: pgmap v487: 129 pgs: 129 active+clean; 397 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 1.02k op/s 2024-09-10T10:49:38.887 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:38 smithi143 ceph-mon[31057]: pgmap v488: 129 pgs: 129 active+clean; 389 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-09-10T10:49:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:38 smithi178 ceph-mon[34813]: pgmap v488: 129 pgs: 129 active+clean; 389 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-09-10T10:49:38.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:38 smithi144 ceph-mon[34230]: pgmap v488: 129 pgs: 129 active+clean; 389 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 1.17k op/s 2024-09-10T10:49:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 -- 172.21.15.143:0/1600664687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 msgr2=0x7f5ad410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/1600664687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad410da40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009ca0 tx=0x7f5acc02a950 comp rx=0 tx=0).stop 2024-09-10T10:49:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 -- 172.21.15.143:0/1600664687 shutdown_connections 2024-09-10T10:49:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/1600664687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad410da40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/1600664687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4102570 0x7f5ad41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/1600664687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad4100e20 0x7f5ad4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 -- 172.21.15.143:0/1600664687 >> 172.21.15.143:0/1600664687 conn(0x7f5ad40fc7d0 msgr2=0x7f5ad40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 -- 172.21.15.143:0/1600664687 shutdown_connections 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 -- 172.21.15.143:0/1600664687 wait complete. 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 Processor -- start 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.738+0000 7f5ad9e14640 1 -- start start 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad9e14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4100e20 0x7f5ad41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad9e14640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad4102570 0x7f5ad41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad9e14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad9e14640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4110e30 con 0x7f5ad4102ef0 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad9e14640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4110cb0 con 0x7f5ad4102570 2024-09-10T10:49:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad9e14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4110fb0 con 0x7f5ad4100e20 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59736/0 (socket says 172.21.15.143:59736) 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2748845753 learned_addr learned my addr 172.21.15.143:0/2748845753 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad2ffd640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad4102570 0x7f5ad41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4100e20 0x7f5ad41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2748845753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4100e20 msgr2=0x7f5ad41972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4100e20 0x7f5ad41972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2748845753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad4102570 msgr2=0x7f5ad41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad4102570 0x7f5ad41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2748845753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad419c260 con 0x7f5ad4102ef0 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad419bb80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5acc02ae70 tx=0x7f5acc02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad0ff9640 1 -- 172.21.15.143:0/2748845753 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc034040 con 0x7f5ad4102ef0 2024-09-10T10:49:39.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.739+0000 7f5ad0ff9640 1 -- 172.21.15.143:0/2748845753 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5acc0308f0 con 0x7f5ad4102ef0 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.740+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad419c4f0 con 0x7f5ad4102ef0 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.740+0000 7f5ad0ff9640 1 -- 172.21.15.143:0/2748845753 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc004780 con 0x7f5ad4102ef0 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.740+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad419c9c0 con 0x7f5ad4102ef0 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5ad0ff9640 1 -- 172.21.15.143:0/2748845753 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5acc037020 con 0x7f5ad4102ef0 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 Processor -- start 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 -- start start 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5ad0ff9640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5abc0a2c00 0x7f5abc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 0x559f86454300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86463110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86463650 0x559f86465a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559f863b9620 con 0x559f86453f20 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f86360e50 con 0x559f86463650 2024-09-10T10:49:39.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f863b55e0 con 0x559f86454840 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5ad0ff9640 1 -- 172.21.15.143:0/2748845753 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5acc0f1c20 con 0x7f5ad4102ef0 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.742+0000 7f5ad2ffd640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5abc0a2c00 0x7f5abc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 0x559f86454300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 0x559f86454300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59744/0 (socket says 172.21.15.143:59744) 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 -- 172.21.15.143:0/3671825732 learned_addr learned my addr 172.21.15.143:0/3671825732 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad2ffd640 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86463110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86463650 0x559f86465a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 -- 172.21.15.143:0/3671825732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 msgr2=0x559f86463110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86463110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 -- 172.21.15.143:0/3671825732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86463650 msgr2=0x559f86465a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86463650 0x559f86465a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 -- 172.21.15.143:0/3671825732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f86351d40 con 0x559f86453f20 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad2ffd640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5abc0a2c00 0x7f5abc0a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40ffe10 tx=0x7f5ac800bb70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 0x559f86454300 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400bcd0 tx=0x7f5ac40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e454c1a4df9ae365 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ab9ffb640 1 -- 172.21.15.143:0/3671825732 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4005020 con 0x559f86453f20 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ab9ffb640 1 -- 172.21.15.143:0/3671825732 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ac40047a0 con 0x559f86453f20 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.743+0000 7f5ab9ffb640 1 -- 172.21.15.143:0/3671825732 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40116a0 con 0x559f86453f20 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 -- 172.21.15.143:0/3671825732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 msgr2=0x559f86454300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 0x559f86454300 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400bcd0 tx=0x7f5ac40107a0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 -- 172.21.15.143:0/3671825732 shutdown_connections 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86463650 0x559f86465a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86463110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/3671825732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86453f20 0x559f86454300 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 -- 172.21.15.143:0/3671825732 >> 172.21.15.143:0/3671825732 conn(0x559f864505f0 msgr2=0x559f864509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 -- 172.21.15.143:0/3671825732 shutdown_connections 2024-09-10T10:49:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 -- 172.21.15.143:0/3671825732 wait complete. 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 32, 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 12, 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "544fb307": { 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 12, 2024-09-10T10:49:39.817 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 142.8, 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:49:39.818 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 -- start start 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.744+0000 7f5adc1ed740 1 Processor -- start 2024-09-10T10:49:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86460b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86461050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559f86466220 con 0x559f86463650 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f86341f10 con 0x559f86453f20 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559f864663a0 con 0x559f86454840 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 10 client.0 _ll_register_callbacks cb 0x559f86365db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86461050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86461050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59750/0 (socket says 172.21.15.143:59750) 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 -- 172.21.15.143:0/4209530052 learned_addr learned my addr 172.21.15.143:0/4209530052 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad2ffd640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86460b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 -- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 msgr2=0x559f86460b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86460b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 -- 172.21.15.143:0/4209530052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 msgr2=0x559f864605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 -- 172.21.15.143:0/4209530052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f860c8580 con 0x559f86463650 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86461050 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0be160 tx=0x7f5acc0bddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38cfa3c6ad0c4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/4209530052 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0be1d0 con 0x559f86463650 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/4209530052 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5acc0c2120 con 0x559f86463650 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.745+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac40047a0 con 0x559f86463650 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/4209530052 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5acc0c24d0 con 0x559f86463650 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 10 client.16515 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f86192960 con 0x559f86463650 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/4209530052 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ac40047a0 con 0x559f86463650 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/4209530052 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f5acc0be430 con 0x559f86463650 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 10 client.16515 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 20 client.16515 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f86473320 0x559f86475740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 4 client.16515 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f863c6b90 con 0x559f86473320 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.746+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f86473320 0x559f86475740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f86473320 0x559f86475740 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b0397dba5bfd6f5 server_cookie=46aeb34e54420da5 in_seq=0 out_seq=0 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5ab2ffd640 10 client.16515 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/4209530052 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f863c6b90 con 0x559f86473320 2024-09-10T10:49:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5ab2ffd640 10 client.16515 handle_command_reply: tid=0 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 client.16515 shutdown 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 msgr2=0x559f86461050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86461050 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0be160 tx=0x7f5acc0bddd0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 shutdown_connections 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86461050 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f86473320 0x559f86475740 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86460b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/4209530052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 >> 172.21.15.143:0/4209530052 conn(0x559f864505f0 msgr2=0x559f86456350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5ab2ffd640 0 client.16515 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 shutdown_connections 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.747+0000 7f5adc1ed740 1 -- 172.21.15.143:0/4209530052 wait complete. 2024-09-10T10:49:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.748+0000 7f5adc1ed740 20 client.16515 trim_cache size 0 max 16384 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.764+0000 7f5adc1ed740 1 Processor -- start 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.764+0000 7f5adc1ed740 1 -- start start 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86453f20 0x559f86462540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86454840 0x559f8647b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f8647b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41ac810 con 0x559f86463650 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41ae380 con 0x559f86454840 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41acee0 con 0x559f86453f20 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f8647b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f8647b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59758/0 (socket says 172.21.15.143:59758) 2024-09-10T10:49:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 -- 172.21.15.143:0/448209242 learned_addr learned my addr 172.21.15.143:0/448209242 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad2ffd640 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86454840 0x559f8647b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86453f20 0x559f86462540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 -- 172.21.15.143:0/448209242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86453f20 msgr2=0x559f86462540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86453f20 0x559f86462540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 -- 172.21.15.143:0/448209242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86454840 msgr2=0x559f8647b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86454840 0x559f8647b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 -- 172.21.15.143:0/448209242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f86462f40 con 0x559f86463650 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f8647b540 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0bc480 tx=0x7f5acc005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9b50719f11b114 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ab97fa640 1 -- 172.21.15.143:0/448209242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0bdfa0 con 0x559f86463650 2024-09-10T10:49:39.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ab97fa640 1 -- 172.21.15.143:0/448209242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5acc0be1d0 con 0x559f86463650 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.765+0000 7f5ab97fa640 1 -- 172.21.15.143:0/448209242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0c2120 con 0x559f86463650 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 -- 172.21.15.143:0/448209242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 msgr2=0x559f8647b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f8647b540 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0bc480 tx=0x7f5acc005fa0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 -- 172.21.15.143:0/448209242 shutdown_connections 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f8647b540 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86454840 0x559f8647b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/448209242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86453f20 0x559f86462540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 -- 172.21.15.143:0/448209242 >> 172.21.15.143:0/448209242 conn(0x559f864505f0 msgr2=0x559f86461e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 -- 172.21.15.143:0/448209242 shutdown_connections 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 -- 172.21.15.143:0/448209242 wait complete. 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.766+0000 7f5adc1ed740 1 -- start start 2024-09-10T10:49:39.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 Processor -- start 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86467930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86467e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41adc80 con 0x559f86463650 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41ad5b0 con 0x559f86453f20 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41aa7f0 con 0x559f86454840 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5adc1ed740 10 client.0 _ll_register_callbacks cb 0x7f5acc0ba500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86467e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86467e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59764/0 (socket says 172.21.15.143:59764) 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86467930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2434660440 learned_addr learned my addr 172.21.15.143:0/2434660440 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 msgr2=0x559f86467930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86467930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2434660440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 msgr2=0x559f864673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 -- 172.21.15.143:0/2434660440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f864dd050 con 0x559f86463650 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ad3fff640 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86467e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0baba0 tx=0x7f5acc0be6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e961fe969b9e5c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ab2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.767+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/2434660440 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0bdfa0 con 0x559f86463650 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/2434660440 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5acc0bd620 con 0x559f86463650 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5acc0be1d0 con 0x559f86463650 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/2434660440 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5acc0c6d90 con 0x559f86463650 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 10 client.16521 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:39.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f864dd050 con 0x559f86463650 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/2434660440 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5acc0be1d0 con 0x559f86463650 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/2434660440 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f5acc0bd090 con 0x559f86463650 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 10 client.16521 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 10 client.16521 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 20 client.16521 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f864de600 0x559f864e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 4 client.16521 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "544fb307", "release": true, "await": true}] 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "set_id": "544fb307", "release": true, "await": true}) -- 0x559f863c10a0 con 0x559f864de600 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f864de600 0x559f864e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ab2ffd640 10 client.16521 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:38.768+0000 7f5ad37fe640 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f864de600 0x559f864e09f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107614042e6027b1 server_cookie=6e28327fe977d23d in_seq=0 out_seq=0 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5ab2ffd640 1 -- 172.21.15.143:0/2434660440 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559f863c10a0 con 0x559f864de600 2024-09-10T10:49:39.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5ab2ffd640 10 client.16521 handle_command_reply: tid=0 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 client.16521 shutdown 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 msgr2=0x559f86467e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86467e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0baba0 tx=0x7f5acc0be6b0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 shutdown_connections 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559f86463650 0x559f86467e70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559f864de600 0x559f864e09f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559f86454840 0x559f86467930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 --2- 172.21.15.143:0/2434660440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f86453f20 0x559f864673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5ab2ffd640 0 client.16521 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 >> 172.21.15.143:0/2434660440 conn(0x559f864505f0 msgr2=0x559f86462650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 shutdown_connections 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 1 -- 172.21.15.143:0/2434660440 wait complete. 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.389+0000 7f5adc1ed740 20 client.16521 trim_cache size 0 max 16384 2024-09-10T10:49:39.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5abc0a2c00 msgr2=0x7f5abc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5abc0a2c00 0x7f5abc0a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40ffe10 tx=0x7f5ac800bb70 comp rx=0 tx=0).stop 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 msgr2=0x7f5ad419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad419bb80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5acc02ae70 tx=0x7f5acc02f5d0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 shutdown_connections 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5abc0a2c00 0x7f5abc0a50c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ad4102ef0 0x7f5ad419bb80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad4102570 0x7f5ad41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 --2- 172.21.15.143:0/2748845753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4100e20 0x7f5ad41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 >> 172.21.15.143:0/2748845753 conn(0x7f5ad40fc7d0 msgr2=0x7f5ad40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 shutdown_connections 2024-09-10T10:49:39.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:39.390+0000 7f5ad9e14640 1 -- 172.21.15.143:0/2748845753 wait complete. 2024-09-10T10:49:39.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:39 smithi178 ceph-mon[34813]: pgmap v489: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 1.18k op/s 2024-09-10T10:49:39.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:39 smithi144 ceph-mon[34230]: pgmap v489: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 1.18k op/s 2024-09-10T10:49:40.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:39 smithi143 ceph-mon[31057]: pgmap v489: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 1.18k op/s 2024-09-10T10:49:40.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '544fb307', total seconds elapsed: 35.7 2024-09-10T10:49:40.871 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T10:49:40.871 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:49:41.274 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 -- 172.21.15.143:0/2601471618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 msgr2=0x7ff8dc100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 --2- 172.21.15.143:0/2601471618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc100ef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009930 tx=0x7ff8d002a420 comp rx=0 tx=0).stop 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 -- 172.21.15.143:0/2601471618 shutdown_connections 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 --2- 172.21.15.143:0/2601471618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc101430 0x7ff8dc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 --2- 172.21.15.143:0/2601471618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc100ef0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 --2- 172.21.15.143:0/2601471618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc106a90 0x7ff8dc106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 -- 172.21.15.143:0/2601471618 >> 172.21.15.143:0/2601471618 conn(0x7ff8dc0fc7d0 msgr2=0x7ff8dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 -- 172.21.15.143:0/2601471618 shutdown_connections 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 -- 172.21.15.143:0/2601471618 wait complete. 2024-09-10T10:49:42.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 Processor -- start 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.237+0000 7ff8e2342640 1 -- start start 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e2342640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e2342640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc101430 0x7ff8dc06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e2342640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 0x7ff8dc06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e2342640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc110d90 con 0x7ff8dc100a90 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e2342640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc110c10 con 0x7ff8dc101430 2024-09-10T10:49:42.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e2342640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc110f10 con 0x7ff8dc106a90 2024-09-10T10:49:42.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8db7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc101430 0x7ff8dc06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 0x7ff8dc06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 0x7ff8dc06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33678/0 (socket says 172.21.15.143:33678) 2024-09-10T10:49:42.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 -- 172.21.15.143:0/3364016970 learned_addr learned my addr 172.21.15.143:0/3364016970 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8dbfff640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 -- 172.21.15.143:0/3364016970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc101430 msgr2=0x7ff8dc06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc101430 0x7ff8dc06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 -- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 msgr2=0x7ff8dc073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8e08df640 1 -- 172.21.15.143:0/3364016970 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc06d6a0 con 0x7ff8dc106a90 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.238+0000 7ff8dbfff640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.239+0000 7ff8e08df640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 0x7ff8dc06d060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00e970 tx=0x7ff8cc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.239+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00ccc0 con 0x7ff8dc106a90 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.239+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc071570 con 0x7ff8dc106a90 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.239+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc071ad0 con 0x7ff8dc106a90 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.239+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8cc00ce60 con 0x7ff8dc106a90 2024-09-10T10:49:42.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.239+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc010430 con 0x7ff8dc106a90 2024-09-10T10:49:42.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.240+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc06d930 con 0x7ff8dc106a90 2024-09-10T10:49:42.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.242+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff8cc044b30 con 0x7ff8dc106a90 2024-09-10T10:49:42.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.242+0000 7ff8d97fa640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff8c40a2b80 0x7ff8c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.243+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff8cc0cf8f0 con 0x7ff8dc106a90 2024-09-10T10:49:42.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.243+0000 7ff8dbfff640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff8c40a2b80 0x7ff8c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.243+0000 7ff8dbfff640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff8c40a2b80 0x7ff8c40a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4005fd0 tx=0x7ff8d4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.245+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8cc09c490 con 0x7ff8dc106a90 2024-09-10T10:49:42.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.595+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8dc1021d0 con 0x7ff8dc106a90 2024-09-10T10:49:42.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.598+0000 7ff8d97fa640 1 -- 172.21.15.143:0/3364016970 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7ff8cc044e20 con 0x7ff8dc106a90 2024-09-10T10:49:42.600 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:49:42.600 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:49:42.600 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:49:42.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff8c40a2b80 msgr2=0x7ff8c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff8c40a2b80 0x7ff8c40a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4005fd0 tx=0x7ff8d4009500 comp rx=0 tx=0).stop 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 msgr2=0x7ff8dc06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 0x7ff8dc06d060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00e970 tx=0x7ff8cc00ed70 comp rx=0 tx=0).stop 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 shutdown_connections 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff8c40a2b80 0x7ff8c40a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8dc106a90 0x7ff8dc06d060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8dc101430 0x7ff8dc06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 --2- 172.21.15.143:0/3364016970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8dc100a90 0x7ff8dc073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.601+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 >> 172.21.15.143:0/3364016970 conn(0x7ff8dc0fc7d0 msgr2=0x7ff8dc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.602+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 shutdown_connections 2024-09-10T10:49:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:42.602+0000 7ff8e2342640 1 -- 172.21.15.143:0/3364016970 wait complete. 2024-09-10T10:49:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:42 smithi178 ceph-mon[34813]: pgmap v490: 129 pgs: 129 active+clean; 384 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 1.01k op/s 2024-09-10T10:49:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:42 smithi144 ceph-mon[34230]: pgmap v490: 129 pgs: 129 active+clean; 384 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 1.01k op/s 2024-09-10T10:49:43.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:42 smithi143 ceph-mon[31057]: pgmap v490: 129 pgs: 129 active+clean; 384 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 1.01k op/s 2024-09-10T10:49:43.591 INFO:tasks.workunit.client.0.smithi143.stdout: 75 9436 38 6144 8 4006 40 2024-09-10T10:49:43.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:43 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3364016970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:43.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:43 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3364016970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:44.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:43 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3364016970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:49:44.642 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T10:49:44.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T10:49:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:44 smithi178 ceph-mon[34813]: pgmap v491: 129 pgs: 129 active+clean; 386 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.15k op/s 2024-09-10T10:49:44.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:44 smithi144 ceph-mon[34230]: pgmap v491: 129 pgs: 129 active+clean; 386 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.15k op/s 2024-09-10T10:49:44.995 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:44 smithi143 ceph-mon[31057]: pgmap v491: 129 pgs: 129 active+clean; 386 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.15k op/s 2024-09-10T10:49:45.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 -- 172.21.15.143:0/536796077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 msgr2=0x7f28341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:45.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 --2- 172.21.15.143:0/536796077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341029b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2828009890 tx=0x7f282802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 -- 172.21.15.143:0/536796077 shutdown_connections 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 --2- 172.21.15.143:0/536796077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2834102ef0 0x7f283410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 --2- 172.21.15.143:0/536796077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341029b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 --2- 172.21.15.143:0/536796077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f2834101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 -- 172.21.15.143:0/536796077 >> 172.21.15.143:0/536796077 conn(0x7f28340fc7d0 msgr2=0x7f28340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 -- 172.21.15.143:0/536796077 shutdown_connections 2024-09-10T10:49:45.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.553+0000 7f2839ec3640 1 -- 172.21.15.143:0/536796077 wait complete. 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 Processor -- start 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 -- start start 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f28341a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2834102ef0 0x7f283410cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f283406a830 con 0x7f2834100e20 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f283406a6b0 con 0x7f2834102ef0 2024-09-10T10:49:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2839ec3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f283406a9b0 con 0x7f2834102570 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f28337fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f28341a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f28337fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f28341a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59796/0 (socket says 172.21.15.143:59796) 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f28337fe640 1 -- 172.21.15.143:0/4206940962 learned_addr learned my addr 172.21.15.143:0/4206940962 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2833fff640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2834102ef0 0x7f283410cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.554+0000 7f2832ffd640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2832ffd640 1 -- 172.21.15.143:0/4206940962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2834102ef0 msgr2=0x7f283410cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2832ffd640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2834102ef0 0x7f283410cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2832ffd640 1 -- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 msgr2=0x7f28341a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2832ffd640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f28341a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2832ffd640 1 -- 172.21.15.143:0/4206940962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283410d2f0 con 0x7f2834102570 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f28337fe640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f28341a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2832ffd640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341a0800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f282802a910 tx=0x7f2828032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2830ff9640 1 -- 172.21.15.143:0/4206940962 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828040020 con 0x7f2834102570 2024-09-10T10:49:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283410d580 con 0x7f2834102570 2024-09-10T10:49:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.555+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28341a7120 con 0x7f2834102570 2024-09-10T10:49:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.556+0000 7f2830ff9640 1 -- 172.21.15.143:0/4206940962 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2828004380 con 0x7f2834102570 2024-09-10T10:49:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.556+0000 7f2830ff9640 1 -- 172.21.15.143:0/4206940962 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828037040 con 0x7f2834102570 2024-09-10T10:49:45.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.558+0000 7f2830ff9640 1 -- 172.21.15.143:0/4206940962 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2828043020 con 0x7f2834102570 2024-09-10T10:49:45.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.558+0000 7f283c29c740 1 Processor -- start 2024-09-10T10:49:45.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.558+0000 7f283c29c740 1 -- start start 2024-09-10T10:49:45.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f283c29c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f283c29c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 0x55f4006c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f283c29c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006c1a30 0x55f4006c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f283c29c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f40061b160 con 0x55f4006c1a30 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f283c29c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4005c2720 con 0x55f4006b7170 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f283c29c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f400617120 con 0x55f4006b6850 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 0x55f4006c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 0x55f4006c14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:37028/0 (socket says 172.21.15.143:37028) 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 -- 172.21.15.143:0/853566672 learned_addr learned my addr 172.21.15.143:0/853566672 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f2833fff640 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 -- 172.21.15.143:0/853566672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 msgr2=0x55f4006b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 -- 172.21.15.143:0/853566672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006c1a30 msgr2=0x55f4006c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f2832ffd640 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006c1a30 0x55f4006c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006c1a30 0x55f4006c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f28337fe640 1 -- 172.21.15.143:0/853566672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f40038cba0 con 0x55f4006b7170 2024-09-10T10:49:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.559+0000 7f2832ffd640 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006c1a30 0x55f4006c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f28337fe640 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 0x55f4006c14f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f28240115c0 tx=0x7f2824011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd553bd67845729d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f2830ff9640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f281c0a2c80 0x7f281c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f2819ffb640 1 -- 172.21.15.143:0/853566672 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282400f6c0 con 0x55f4006b7170 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f2819ffb640 1 -- 172.21.15.143:0/853566672 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f282400fd10 con 0x55f4006b7170 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f2833fff640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f281c0a2c80 0x7f281c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f2830ff9640 1 -- 172.21.15.143:0/4206940962 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f28280eed80 con 0x7f2834102570 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.560+0000 7f2819ffb640 1 -- 172.21.15.143:0/853566672 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824019e40 con 0x55f4006b7170 2024-09-10T10:49:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 -- 172.21.15.143:0/853566672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 msgr2=0x55f4006c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f2833fff640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f281c0a2c80 0x7f281c0a5140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f282c005fd0 tx=0x7f282c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 0x55f4006c14f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f28240115c0 tx=0x7f2824011a80 comp rx=0 tx=0).stop 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 -- 172.21.15.143:0/853566672 shutdown_connections 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006c1a30 0x55f4006c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b7170 0x55f4006c14f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 --2- 172.21.15.143:0/853566672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 -- 172.21.15.143:0/853566672 >> 172.21.15.143:0/853566672 conn(0x55f4006b2de0 msgr2=0x55f4006b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 -- 172.21.15.143:0/853566672 shutdown_connections 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 -- 172.21.15.143:0/853566672 wait complete. 2024-09-10T10:49:45.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.561+0000 7f283c29c740 1 -- start start 2024-09-10T10:49:45.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 Processor -- start 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 0x55f4006be850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006c1a30 0x55f4006bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f4006c46a0 con 0x55f4006b7170 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4005a3900 con 0x55f4006b6850 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4006c4820 con 0x55f4006c1a30 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 0x55f4006be850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f283c29c740 10 client.0 _ll_register_callbacks cb 0x55f4005a1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 0x55f4006be850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:37032/0 (socket says 172.21.15.143:37032) 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59802/0 (socket says 172.21.15.143:59802) 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 -- 172.21.15.143:0/2028698352 learned_addr learned my addr 172.21.15.143:0/2028698352 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f2833fff640 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006c1a30 0x55f4006bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 -- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006c1a30 msgr2=0x55f4006bf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006c1a30 0x55f4006bf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 -- 172.21.15.143:0/2028698352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 msgr2=0x55f4006bed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.562+0000 7f28337fe640 1 -- 172.21.15.143:0/2028698352 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4005e3c50 con 0x55f4006b6850 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f28337fe640 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 0x55f4006be850 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2824011510 tx=0x7f2824019d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6a4a01c49a8588 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f2812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:49:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f2812ffd640 1 -- 172.21.15.143:0/2028698352 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824027020 con 0x55f4006b6850 2024-09-10T10:49:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f2812ffd640 1 -- 172.21.15.143:0/2028698352 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2824004250 con 0x55f4006b6850 2024-09-10T10:49:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f282400fd10 con 0x55f4006b6850 2024-09-10T10:49:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f2812ffd640 1 -- 172.21.15.143:0/2028698352 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2824032050 con 0x55f4006b6850 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f283c29c740 10 client.25901 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4006d0280 con 0x55f4006b6850 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f2812ffd640 1 -- 172.21.15.143:0/2028698352 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f282400fd10 con 0x55f4006b6850 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f2812ffd640 1 -- 172.21.15.143:0/2028698352 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f2824005040 con 0x55f4006b6850 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f283c29c740 10 client.25901 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f283c29c740 10 client.25901 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.563+0000 7f283c29c740 20 client.25901 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.564+0000 7f283c29c740 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f4006d16e0 0x55f4006d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.564+0000 7f283c29c740 4 client.25901 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.564+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f400628890 con 0x55f4006d16e0 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.564+0000 7f2832ffd640 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f4006d16e0 0x55f4006d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.564+0000 7f2832ffd640 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f4006d16e0 0x55f4006d3ad0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8fc9dd41e02335 server_cookie=fc2f93e3be7240b9 in_seq=0 out_seq=0 2024-09-10T10:49:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.564+0000 7f2812ffd640 10 client.25901 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f2812ffd640 1 -- 172.21.15.143:0/2028698352 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f400628890 con 0x55f4006d16e0 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f2812ffd640 10 client.25901 handle_command_reply: tid=0 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 client.25901 shutdown 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 msgr2=0x55f4006be850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 0x55f4006be850 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2824011510 tx=0x7f2824019d70 comp rx=0 tx=0).stop 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 shutdown_connections 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f4006d16e0 0x55f4006d3ad0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f2812ffd640 0 client.25901 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006c1a30 0x55f4006bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 --2- 172.21.15.143:0/2028698352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006b6850 0x55f4006be850 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 >> 172.21.15.143:0/2028698352 conn(0x55f4006b2de0 msgr2=0x55f4006b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 shutdown_connections 2024-09-10T10:49:45.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 1 -- 172.21.15.143:0/2028698352 wait complete. 2024-09-10T10:49:45.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.565+0000 7f283c29c740 20 client.25901 trim_cache size 0 max 16384 2024-09-10T10:49:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 Processor -- start 2024-09-10T10:49:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 -- start start 2024-09-10T10:49:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f282000faa0 con 0x55f4006b7170 2024-09-10T10:49:45.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2820011610 con 0x55f4006c1a30 2024-09-10T10:49:45.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f283c29c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2820010170 con 0x55f4006b6850 2024-09-10T10:49:45.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f28337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f28337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33722/0 (socket says 172.21.15.143:33722) 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59804/0 (socket says 172.21.15.143:59804) 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f28337fe640 1 -- 172.21.15.143:0/689381816 learned_addr learned my addr 172.21.15.143:0/689381816 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.591+0000 7f2833fff640 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28337fe640 1 -- 172.21.15.143:0/689381816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 msgr2=0x55f4006d92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28337fe640 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006d92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28337fe640 1 -- 172.21.15.143:0/689381816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 msgr2=0x55f4006bd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28337fe640 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28337fe640 1 -- 172.21.15.143:0/689381816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4006b6fa0 con 0x55f4006b6850 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f2832ffd640 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28337fe640 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b92b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f28240053e0 tx=0x7f2824012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908912dfec4a748d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28197fa640 1 -- 172.21.15.143:0/689381816 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824027040 con 0x55f4006b6850 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28197fa640 1 -- 172.21.15.143:0/689381816 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28240197d0 con 0x55f4006b6850 2024-09-10T10:49:45.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.592+0000 7f28197fa640 1 -- 172.21.15.143:0/689381816 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004d60 con 0x55f4006b6850 2024-09-10T10:49:45.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 -- 172.21.15.143:0/689381816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 msgr2=0x55f4006b92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b92b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f28240053e0 tx=0x7f2824012f90 comp rx=0 tx=0).stop 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 -- 172.21.15.143:0/689381816 shutdown_connections 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006d92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 --2- 172.21.15.143:0/689381816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006b92b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 -- 172.21.15.143:0/689381816 >> 172.21.15.143:0/689381816 conn(0x55f4006b2de0 msgr2=0x55f4006bf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 -- 172.21.15.143:0/689381816 shutdown_connections 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.593+0000 7f283c29c740 1 -- 172.21.15.143:0/689381816 wait complete. 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 -- start start 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 Processor -- start 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2820010f10 con 0x55f4006b7170 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2820010840 con 0x55f4006c1a30 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f282000da80 con 0x55f4006b6850 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f283c29c740 10 client.0 _ll_register_callbacks cb 0x7f282c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f28337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f28337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33738/0 (socket says 172.21.15.143:33738) 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f28337fe640 1 -- 172.21.15.143:0/845312311 learned_addr learned my addr 172.21.15.143:0/845312311 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.594+0000 7f2833fff640 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f28337fe640 1 -- 172.21.15.143:0/845312311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 msgr2=0x55f4006bbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f28337fe640 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006bbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f28337fe640 1 -- 172.21.15.143:0/845312311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 msgr2=0x55f4006bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f28337fe640 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f28337fe640 1 -- 172.21.15.143:0/845312311 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4006c7f30 con 0x55f4006b6850 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f28337fe640 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006ca3a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2824005d60 tx=0x7f2824019980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76358e2e933ea670 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f2812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f2812ffd640 1 -- 172.21.15.143:0/845312311 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824027070 con 0x55f4006b6850 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f2812ffd640 1 -- 172.21.15.143:0/845312311 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f28240049c0 con 0x55f4006b6850 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28240197d0 con 0x55f4006b6850 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f2812ffd640 1 -- 172.21.15.143:0/845312311 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2824058050 con 0x55f4006b6850 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f283c29c740 10 client.25951 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.595+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f40073c460 con 0x55f4006b6850 2024-09-10T10:49:45.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f2812ffd640 1 -- 172.21.15.143:0/845312311 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28240197d0 con 0x55f4006b6850 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f2812ffd640 1 -- 172.21.15.143:0/845312311 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f282403e540 con 0x55f4006b6850 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f283c29c740 10 client.25951 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f283c29c740 10 client.25951 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f283c29c740 20 client.25951 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f283c29c740 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f40073cb60 0x55f40073ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f283c29c740 4 client.25951 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T10:49:45.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55f400622be0 con 0x55f40073cb60 2024-09-10T10:49:45.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f2832ffd640 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f40073cb60 0x55f40073ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:45.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f2812ffd640 10 client.25951 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:45.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:45.596+0000 7f2832ffd640 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f40073cb60 0x55f40073ef80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e729c51231eeec74 server_cookie=462d526c7a48a772 in_seq=0 out_seq=0 2024-09-10T10:49:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:46 smithi178 ceph-mon[34813]: pgmap v492: 129 pgs: 129 active+clean; 386 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 993 op/s 2024-09-10T10:49:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:46 smithi144 ceph-mon[34230]: pgmap v492: 129 pgs: 129 active+clean; 386 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 993 op/s 2024-09-10T10:49:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:46 smithi143 ceph-mon[31057]: pgmap v492: 129 pgs: 129 active+clean; 386 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 993 op/s 2024-09-10T10:49:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:48 smithi178 ceph-mon[34813]: pgmap v493: 129 pgs: 129 active+clean; 376 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.12k op/s 2024-09-10T10:49:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:48 smithi144 ceph-mon[34230]: pgmap v493: 129 pgs: 129 active+clean; 376 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.12k op/s 2024-09-10T10:49:49.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:48 smithi143 ceph-mon[31057]: pgmap v493: 129 pgs: 129 active+clean; 376 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.12k op/s 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f2812ffd640 1 -- 172.21.15.143:0/845312311 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f400622be0 con 0x55f40073cb60 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f2812ffd640 10 client.25951 handle_command_reply: tid=0 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 client.25951 shutdown 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 msgr2=0x55f4006ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006ca3a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2824005d60 tx=0x7f2824019980 comp rx=0 tx=0).stop 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 shutdown_connections 2024-09-10T10:49:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4006c1a30 0x55f4006bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4006b7170 0x55f4006bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f40073cb60 0x55f40073ef80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 --2- 172.21.15.143:0/845312311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4006b6850 0x55f4006ca3a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f2812ffd640 0 client.25951 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.632+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 >> 172.21.15.143:0/845312311 conn(0x55f4006b2de0 msgr2=0x55f4006be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.633+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 shutdown_connections 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.633+0000 7f283c29c740 1 -- 172.21.15.143:0/845312311 wait complete. 2024-09-10T10:49:50.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.633+0000 7f283c29c740 20 client.25951 trim_cache size 0 max 16384 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 32, 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 14, 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "8ac87a67": { 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 14, 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:49:50.637 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 142.8, 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:50.638 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f281c0a2c80 msgr2=0x7f281c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f281c0a2c80 0x7f281c0a5140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f282c005fd0 tx=0x7f282c009500 comp rx=0 tx=0).stop 2024-09-10T10:49:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 msgr2=0x7f28341a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341a0800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f282802a910 tx=0x7f2828032da0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 shutdown_connections 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f281c0a2c80 0x7f281c0a5140 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2834102ef0 0x7f283410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2834102570 0x7f28341a0800 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 --2- 172.21.15.143:0/4206940962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2834100e20 0x7f28341a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 >> 172.21.15.143:0/4206940962 conn(0x7f28340fc7d0 msgr2=0x7f2834110570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 shutdown_connections 2024-09-10T10:49:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:50.634+0000 7f2839ec3640 1 -- 172.21.15.143:0/4206940962 wait complete. 2024-09-10T10:49:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:50 smithi178 ceph-mon[34813]: pgmap v494: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 1.02k op/s 2024-09-10T10:49:50.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:50 smithi144 ceph-mon[34230]: pgmap v494: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 1.02k op/s 2024-09-10T10:49:50.963 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:50 smithi143 ceph-mon[31057]: pgmap v494: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 1.02k op/s 2024-09-10T10:49:51.419 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8ac87a67', quiesce took 10.5 seconds. Will release after: 12.25 2024-09-10T10:49:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:52 smithi178 ceph-mon[34813]: pgmap v495: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 812 op/s 2024-09-10T10:49:52.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:52 smithi144 ceph-mon[34230]: pgmap v495: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 812 op/s 2024-09-10T10:49:53.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:52 smithi143 ceph-mon[31057]: pgmap v495: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 812 op/s 2024-09-10T10:49:53.591 INFO:tasks.workunit.client.0.smithi143.stdout: 76 4539 25 3153 13 1494 31 2024-09-10T10:49:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:54 smithi178 ceph-mon[34813]: pgmap v496: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 977 KiB/s wr, 766 op/s 2024-09-10T10:49:54.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:54 smithi144 ceph-mon[34230]: pgmap v496: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 977 KiB/s wr, 766 op/s 2024-09-10T10:49:55.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:54 smithi143 ceph-mon[31057]: pgmap v496: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 977 KiB/s wr, 766 op/s 2024-09-10T10:49:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:56 smithi178 ceph-mon[34813]: pgmap v497: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 831 KiB/s wr, 492 op/s 2024-09-10T10:49:56.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:56 smithi144 ceph-mon[34230]: pgmap v497: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 831 KiB/s wr, 492 op/s 2024-09-10T10:49:57.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:56 smithi143 ceph-mon[31057]: pgmap v497: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 831 KiB/s wr, 492 op/s 2024-09-10T10:49:57.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:49:57.814 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 -- 172.21.15.143:0/213663231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580101de0 msgr2=0x7f75801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 --2- 172.21.15.143:0/213663231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580101de0 0x7f75801021c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7570009ca0 tx=0x7f757002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 -- 172.21.15.143:0/213663231 shutdown_connections 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 --2- 172.21.15.143:0/213663231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580106f10 0x7f758010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 --2- 172.21.15.143:0/213663231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580102790 0x7f75801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 --2- 172.21.15.143:0/213663231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580101de0 0x7f75801021c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 -- 172.21.15.143:0/213663231 >> 172.21.15.143:0/213663231 conn(0x7f75800fd8c0 msgr2=0x7f75800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 -- 172.21.15.143:0/213663231 shutdown_connections 2024-09-10T10:49:58.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.333+0000 7f758752f640 1 -- 172.21.15.143:0/213663231 wait complete. 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 Processor -- start 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 -- start start 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 0x7f75800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580102790 0x7f7580074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580106f10 0x7f7580074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7580111e20 con 0x7f7580102790 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7580111ca0 con 0x7f7580106f10 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f758752f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7580111fa0 con 0x7f7580101de0 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f7585acc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580106f10 0x7f7580074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f75852cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 0x7f75800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f75852cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 0x7f75800744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35398/0 (socket says 172.21.15.143:35398) 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.334+0000 7f75852cb640 1 -- 172.21.15.143:0/3603730156 learned_addr learned my addr 172.21.15.143:0/3603730156 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:58.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f7584aca640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580102790 0x7f7580074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f75852cb640 1 -- 172.21.15.143:0/3603730156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580106f10 msgr2=0x7f7580074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f75852cb640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580106f10 0x7f7580074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f75852cb640 1 -- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580102790 msgr2=0x7f7580074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f75852cb640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580102790 0x7f7580074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f75852cb640 1 -- 172.21.15.143:0/3603730156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75801a6820 con 0x7f7580101de0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f7584aca640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580102790 0x7f7580074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f75852cb640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 0x7f75800744f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f757002ade0 tx=0x7f7570032ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f757e7fc640 1 -- 172.21.15.143:0/3603730156 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570003c70 con 0x7f7580101de0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f757e7fc640 1 -- 172.21.15.143:0/3603730156 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7570003e10 con 0x7f7580101de0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f757e7fc640 1 -- 172.21.15.143:0/3603730156 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570030c30 con 0x7f7580101de0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75801a6ab0 con 0x7f7580101de0 2024-09-10T10:49:58.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.335+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801a6fa0 con 0x7f7580101de0 2024-09-10T10:49:58.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.338+0000 7f757e7fc640 1 -- 172.21.15.143:0/3603730156 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7570004040 con 0x7f7580101de0 2024-09-10T10:49:58.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 Processor -- start 2024-09-10T10:49:58.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f757e7fc640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75680a2c00 0x7f75680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 -- start start 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f757e7fc640 1 -- 172.21.15.143:0/3603730156 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f75700f0850 con 0x7f7580101de0 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 0x560ff6b98880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7585acc640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75680a2c00 0x7f75680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b98dc0 0x560ff6ba3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6ba3680 0x560ff6ba5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560ff6afe2e0 con 0x560ff6b984a0 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ff6aa5ce0 con 0x560ff6ba3680 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7589908740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560ff6afa2a0 con 0x560ff6b98dc0 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f75852cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 0x560ff6b98880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f75852cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 0x560ff6b98880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42720/0 (socket says 172.21.15.143:42720) 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f75852cb640 1 -- 172.21.15.143:0/205899325 learned_addr learned my addr 172.21.15.143:0/205899325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.339+0000 7f7585acc640 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6ba3680 0x560ff6ba5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7584aca640 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b98dc0 0x560ff6ba3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 -- 172.21.15.143:0/205899325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b98dc0 msgr2=0x560ff6ba3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b98dc0 0x560ff6ba3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 -- 172.21.15.143:0/205899325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 msgr2=0x560ff6b98880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 0x560ff6b98880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 -- 172.21.15.143:0/205899325 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff6a8a440 con 0x560ff6ba3680 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f75852cb640 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 0x560ff6b98880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75680a2c00 0x7f75680a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7578009f80 tx=0x7f7578011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f7585acc640 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6ba3680 0x560ff6ba5a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7578009a20 tx=0x7f7578020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a49da3aa430138 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f75677fe640 1 -- 172.21.15.143:0/205899325 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757801a040 con 0x560ff6ba3680 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f75677fe640 1 -- 172.21.15.143:0/205899325 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7578024070 con 0x560ff6ba3680 2024-09-10T10:49:58.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.340+0000 7f75677fe640 1 -- 172.21.15.143:0/205899325 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578020770 con 0x560ff6ba3680 2024-09-10T10:49:58.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 -- 172.21.15.143:0/205899325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6ba3680 msgr2=0x560ff6ba5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6ba3680 0x560ff6ba5a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7578009a20 tx=0x7f7578020000 comp rx=0 tx=0).stop 2024-09-10T10:49:58.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 -- 172.21.15.143:0/205899325 shutdown_connections 2024-09-10T10:49:58.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6ba3680 0x560ff6ba5a70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b98dc0 0x560ff6ba3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 --2- 172.21.15.143:0/205899325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b984a0 0x560ff6b98880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 -- 172.21.15.143:0/205899325 >> 172.21.15.143:0/205899325 conn(0x560ff6b94a30 msgr2=0x560ff6b94e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 -- 172.21.15.143:0/205899325 shutdown_connections 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.341+0000 7f7589908740 1 -- 172.21.15.143:0/205899325 wait complete. 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.342+0000 7f7589908740 1 -- start start 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.342+0000 7f7589908740 1 Processor -- start 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6ba2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b98dc0 0x560ff6ba0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6ba3680 0x560ff6ba07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6ba2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560ff6ba6250 con 0x560ff6ba3680 2024-09-10T10:49:58.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6ba2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35408/0 (socket says 172.21.15.143:35408) 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 -- 172.21.15.143:0/4259524526 learned_addr learned my addr 172.21.15.143:0/4259524526 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ff6ab9050 con 0x560ff6b98dc0 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560ff6ba63d0 con 0x560ff6b984a0 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7584aca640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b98dc0 0x560ff6ba0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7589908740 10 client.0 _ll_register_callbacks cb 0x560ff6af9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7585acc640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6ba3680 0x560ff6ba07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 -- 172.21.15.143:0/4259524526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b98dc0 msgr2=0x560ff6ba0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b98dc0 0x560ff6ba0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 -- 172.21.15.143:0/4259524526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6ba3680 msgr2=0x560ff6ba07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6ba3680 0x560ff6ba07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 -- 172.21.15.143:0/4259524526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff686f3c0 con 0x560ff6b984a0 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7585acc640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6ba3680 0x560ff6ba07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:58.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f75852cb640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6ba2fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7570048040 tx=0x7f75700305c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c0c1e7bef1bfaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.343+0000 7f7564ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7564ff9640 1 -- 172.21.15.143:0/4259524526 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700bcd80 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7564ff9640 1 -- 172.21.15.143:0/4259524526 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f75700b93f0 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7578024070 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7564ff9640 1 -- 172.21.15.143:0/4259524526 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f75700c1130 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7589908740 10 client.25963 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ff6ba6250 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7564ff9640 1 -- 172.21.15.143:0/4259524526 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7578024070 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7564ff9640 1 -- 172.21.15.143:0/4259524526 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f75700c54d0 con 0x560ff6b984a0 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7589908740 10 client.25963 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7589908740 10 client.25963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.344+0000 7f7589908740 20 client.25963 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.345+0000 7f7589908740 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6bb30b0 0x560ff6bb5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.345+0000 7f7589908740 4 client.25963 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:58.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.345+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ff6b0b850 con 0x560ff6bb30b0 2024-09-10T10:49:58.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.345+0000 7f7584aca640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6bb30b0 0x560ff6bb5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.345+0000 7f7584aca640 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6bb30b0 0x560ff6bb5530 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cb566ff805171e server_cookie=c1d14f0721f4c452 in_seq=0 out_seq=0 2024-09-10T10:49:58.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.345+0000 7f7564ff9640 10 client.25963 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7564ff9640 1 -- 172.21.15.143:0/4259524526 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ff6b0b850 con 0x560ff6bb30b0 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7564ff9640 10 client.25963 handle_command_reply: tid=0 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 client.25963 shutdown 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 msgr2=0x560ff6ba2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6ba2fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7570048040 tx=0x7f75700305c0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 shutdown_connections 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6ba3680 0x560ff6ba07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b98dc0 0x560ff6ba0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6bb30b0 0x560ff6bb5530 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 --2- 172.21.15.143:0/4259524526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6ba2fc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 >> 172.21.15.143:0/4259524526 conn(0x560ff6b94a30 msgr2=0x560ff6b9a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7564ff9640 0 client.25963 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 shutdown_connections 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.346+0000 7f7589908740 1 -- 172.21.15.143:0/4259524526 wait complete. 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.347+0000 7f7589908740 20 client.25963 trim_cache size 0 max 16384 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 Processor -- start 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 -- start start 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b94a30 0x560ff6baf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6bbae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6bbb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b0d40 con 0x560ff6b98dc0 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b28b0 con 0x560ff6b94a30 2024-09-10T10:49:58.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7589908740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b1410 con 0x560ff6b984a0 2024-09-10T10:49:58.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7585acc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6bbb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7585acc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6bbb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42740/0 (socket says 172.21.15.143:42740) 2024-09-10T10:49:58.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7585acc640 1 -- 172.21.15.143:0/1763156252 learned_addr learned my addr 172.21.15.143:0/1763156252 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f75852cb640 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b94a30 0x560ff6baf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7584aca640 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6bbae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f75852cb640 1 -- 172.21.15.143:0/1763156252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 msgr2=0x560ff6bbae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f75852cb640 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6bbae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f75852cb640 1 -- 172.21.15.143:0/1763156252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 msgr2=0x560ff6bbb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f75852cb640 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6bbb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f75852cb640 1 -- 172.21.15.143:0/1763156252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff6b9e7c0 con 0x560ff6b94a30 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.370+0000 7f7585acc640 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6bbb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:58.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.371+0000 7f75852cb640 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b94a30 0x560ff6baf500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75700c5430 tx=0x7f757002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00efbb7ef276181 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.371+0000 7f7566ffd640 1 -- 172.21.15.143:0/1763156252 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700bcd80 con 0x560ff6b94a30 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.371+0000 7f7566ffd640 1 -- 172.21.15.143:0/1763156252 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7570036030 con 0x560ff6b94a30 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.371+0000 7f7566ffd640 1 -- 172.21.15.143:0/1763156252 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700bc6e0 con 0x560ff6b94a30 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 -- 172.21.15.143:0/1763156252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b94a30 msgr2=0x560ff6baf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b94a30 0x560ff6baf500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75700c5430 tx=0x7f757002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 -- 172.21.15.143:0/1763156252 shutdown_connections 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6bbb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b984a0 0x560ff6bbae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 --2- 172.21.15.143:0/1763156252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b94a30 0x560ff6baf500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 -- 172.21.15.143:0/1763156252 >> 172.21.15.143:0/1763156252 conn(0x560ff6ba0910 msgr2=0x560ff6b9eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 -- 172.21.15.143:0/1763156252 shutdown_connections 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 -- 172.21.15.143:0/1763156252 wait complete. 2024-09-10T10:49:58.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 -- start start 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.372+0000 7f7589908740 1 Processor -- start 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b94a30 0x560ff6ba75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 0x560ff6ba7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6ba8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b21b0 con 0x560ff6b98dc0 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b1ae0 con 0x560ff6b984a0 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75801aed20 con 0x560ff6b94a30 2024-09-10T10:49:58.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7589908740 10 client.0 _ll_register_callbacks cb 0x7f75700eded0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7584aca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 0x560ff6ba7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f75852cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b94a30 0x560ff6ba75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7584aca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 0x560ff6ba7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:51402/0 (socket says 172.21.15.143:51402) 2024-09-10T10:49:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f75852cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b94a30 0x560ff6ba75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35422/0 (socket says 172.21.15.143:35422) 2024-09-10T10:49:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7584aca640 1 -- 172.21.15.143:0/3448496324 learned_addr learned my addr 172.21.15.143:0/3448496324 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:49:58.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7585acc640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6ba8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7584aca640 1 -- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b94a30 msgr2=0x560ff6ba75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7584aca640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b94a30 0x560ff6ba75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.373+0000 7f7584aca640 1 -- 172.21.15.143:0/3448496324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 msgr2=0x560ff6ba8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7584aca640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6ba8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7584aca640 1 -- 172.21.15.143:0/3448496324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff6c1cfb0 con 0x560ff6b984a0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7585acc640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6ba8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7584aca640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 0x560ff6ba7b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7580075f80 tx=0x7f757400b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a8b2e5b652e48c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7564ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7564ff9640 1 -- 172.21.15.143:0/3448496324 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574022070 con 0x560ff6b984a0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7564ff9640 1 -- 172.21.15.143:0/3448496324 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f757401d5b0 con 0x560ff6b984a0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7570036030 con 0x560ff6b984a0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7564ff9640 1 -- 172.21.15.143:0/3448496324 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f757402e050 con 0x560ff6b984a0 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7589908740 10 client.25925 fetch_fsmap learned FSMap version 32 2024-09-10T10:49:58.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ff6c1e330 con 0x560ff6b984a0 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.374+0000 7f7564ff9640 1 -- 172.21.15.143:0/3448496324 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7574003e70 con 0x560ff6b984a0 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7564ff9640 1 -- 172.21.15.143:0/3448496324 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f7574011aa0 con 0x560ff6b984a0 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7589908740 10 client.25925 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7589908740 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7589908740 20 client.25925 populate_metadata read hostname 'smithi143' 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7589908740 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6ba3680 0x560ff6c20b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7589908740 4 client.25925 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7570036030 con 0x560ff6ba3680 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f75852cb640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6ba3680 0x560ff6c20b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f7564ff9640 10 client.25925 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.375+0000 7f75852cb640 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6ba3680 0x560ff6c20b50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5190e462b8c1c63c server_cookie=7f8f2b6b229c5aaa in_seq=0 out_seq=0 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7564ff9640 1 -- 172.21.15.143:0/3448496324 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f7570036030 con 0x560ff6ba3680 2024-09-10T10:49:58.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7564ff9640 10 client.25925 handle_command_reply: tid=0 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 client.25925 shutdown 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 msgr2=0x560ff6ba7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 0x560ff6ba7b00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7580075f80 tx=0x7f757400b3c0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 shutdown_connections 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560ff6b98dc0 0x560ff6ba8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ff6b984a0 0x560ff6ba7b00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560ff6ba3680 0x560ff6c20b50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 --2- 172.21.15.143:0/3448496324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560ff6b94a30 0x560ff6ba75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7564ff9640 0 client.25925 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 >> 172.21.15.143:0/3448496324 conn(0x560ff6ba0910 msgr2=0x560ff6b99f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 shutdown_connections 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 1 -- 172.21.15.143:0/3448496324 wait complete. 2024-09-10T10:49:58.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.376+0000 7f7589908740 20 client.25925 trim_cache size 0 max 16384 2024-09-10T10:49:58.382 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:49:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:49:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:49:58.382 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75680a2c00 msgr2=0x7f75680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75680a2c00 0x7f75680a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7578009f80 tx=0x7f7578011810 comp rx=0 tx=0).stop 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 msgr2=0x7f75800744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 0x7f75800744f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f757002ade0 tx=0x7f7570032ff0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 shutdown_connections 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75680a2c00 0x7f75680a50c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580106f10 0x7f7580074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7580102790 0x7f7580074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 --2- 172.21.15.143:0/3603730156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580101de0 0x7f75800744f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 >> 172.21.15.143:0/3603730156 conn(0x7f75800fd8c0 msgr2=0x7f75800fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 shutdown_connections 2024-09-10T10:49:58.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:49:58.377+0000 7f758752f640 1 -- 172.21.15.143:0/3603730156 wait complete. 2024-09-10T10:49:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:58 smithi178 ceph-mon[34813]: pgmap v498: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 509 op/s 2024-09-10T10:49:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:58 smithi144 ceph-mon[34230]: pgmap v498: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 509 op/s 2024-09-10T10:49:58.977 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:58 smithi143 ceph-mon[31057]: pgmap v498: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 509 op/s 2024-09-10T10:49:59.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:49:59.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:49:59.519 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:49:59.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:49:59 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:49:59.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:49:59 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:50:00.033 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:49:59 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:50:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 -- 172.21.15.143:0/3603101679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102570 msgr2=0x7fa9dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3603101679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102570 0x7fa9dc1029b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009900 tx=0x7fa9c402a480 comp rx=0 tx=0).stop 2024-09-10T10:50:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 -- 172.21.15.143:0/3603101679 shutdown_connections 2024-09-10T10:50:00.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3603101679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc102ef0 0x7fa9dc069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3603101679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102570 0x7fa9dc1029b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3603101679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc100e20 0x7fa9dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 -- 172.21.15.143:0/3603101679 >> 172.21.15.143:0/3603101679 conn(0x7fa9dc0fc7d0 msgr2=0x7fa9dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 -- 172.21.15.143:0/3603101679 shutdown_connections 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.291+0000 7fa9e276e640 1 -- 172.21.15.143:0/3603101679 wait complete. 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 Processor -- start 2024-09-10T10:50:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 -- start start 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc100e20 0x7fa9dc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 0x7fa9dc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102ef0 0x7fa9dc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1141e0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc114060 con 0x7fa9dc100e20 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.292+0000 7fa9e276e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc114360 con 0x7fa9dc102570 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 0x7fa9dc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 0x7fa9dc19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35446/0 (socket says 172.21.15.143:35446) 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9d37fe640 1 -- 172.21.15.143:0/3711239064 learned_addr learned my addr 172.21.15.143:0/3711239064 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102ef0 0x7fa9dc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9dbfff640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc100e20 0x7fa9dc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 -- 172.21.15.143:0/3711239064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 msgr2=0x7fa9dc19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 0x7fa9dc19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 -- 172.21.15.143:0/3711239064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc100e20 msgr2=0x7fa9dc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc100e20 0x7fa9dc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 -- 172.21.15.143:0/3711239064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc19f700 con 0x7fa9dc102ef0 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9dbfff640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc100e20 0x7fa9dc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9d37fe640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 0x7fa9dc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.293+0000 7fa9e0d0b640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102ef0 0x7fa9dc19f020 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400e970 tx=0x7fa9d400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.294+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d400ccc0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.294+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9d40035f0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.294+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d400ce60 con 0x7fa9dc102ef0 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.294+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc19f9f0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.294+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc1a72d0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.296+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc1029b0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.297+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa9d40037a0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.298+0000 7fa9d9ffb640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9c80a2b80 0x7fa9c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.298+0000 7fa9dbfff640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9c80a2b80 0x7fa9c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.298+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa9d40cf5a0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.299+0000 7fa9dbfff640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9c80a2b80 0x7fa9c80a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005fd0 tx=0x7fa9cc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.302+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9d409c1a0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.633+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9dc1a74a0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.634+0000 7fa9d9ffb640 1 -- 172.21.15.143:0/3711239064 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7fa9d409baa0 con 0x7fa9dc102ef0 2024-09-10T10:50:00.635 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:50:00.635 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:50:00.636 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:50:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9c80a2b80 msgr2=0x7fa9c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9c80a2b80 0x7fa9c80a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005fd0 tx=0x7fa9cc009500 comp rx=0 tx=0).stop 2024-09-10T10:50:00.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102ef0 msgr2=0x7fa9dc19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102ef0 0x7fa9dc19f020 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400e970 tx=0x7fa9d400ee30 comp rx=0 tx=0).stop 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 shutdown_connections 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9c80a2b80 0x7fa9c80a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9dc102ef0 0x7fa9dc19f020 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9dc102570 0x7fa9dc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 --2- 172.21.15.143:0/3711239064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9dc100e20 0x7fa9dc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 >> 172.21.15.143:0/3711239064 conn(0x7fa9dc0fc7d0 msgr2=0x7fa9dc106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 shutdown_connections 2024-09-10T10:50:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:00.637+0000 7fa9e276e640 1 -- 172.21.15.143:0/3711239064 wait complete. 2024-09-10T10:50:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:00 smithi178 ceph-mon[34813]: pgmap v499: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 901 KiB/s wr, 361 op/s 2024-09-10T10:50:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:00 smithi178 ceph-mon[34813]: overall HEALTH_OK 2024-09-10T10:50:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:50:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:50:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:00 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:00 smithi144 ceph-mon[34230]: pgmap v499: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 901 KiB/s wr, 361 op/s 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:00 smithi144 ceph-mon[34230]: overall HEALTH_OK 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:00 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:50:01.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:00 smithi143 ceph-mon[31057]: pgmap v499: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 901 KiB/s wr, 361 op/s 2024-09-10T10:50:01.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:00 smithi143 ceph-mon[31057]: overall HEALTH_OK 2024-09-10T10:50:01.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:50:01.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:50:01.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:00 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:50:01.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:01 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3711239064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:01.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:01 smithi143 ceph-mon[31057]: pgmap v500: 129 pgs: 129 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 979 KiB/s wr, 262 op/s 2024-09-10T10:50:01.870 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:50:01.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:01 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3711239064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:01.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:01 smithi178 ceph-mon[34813]: pgmap v500: 129 pgs: 129 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 979 KiB/s wr, 262 op/s 2024-09-10T10:50:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:01 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3711239064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:01 smithi144 ceph-mon[34230]: pgmap v500: 129 pgs: 129 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 979 KiB/s wr, 262 op/s 2024-09-10T10:50:02.217 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 -- 172.21.15.143:0/813937292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 msgr2=0x7f666010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 --2- 172.21.15.143:0/813937292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666010d0e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f665800adb0 tx=0x7f665802b7a0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 -- 172.21.15.143:0/813937292 shutdown_connections 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 --2- 172.21.15.143:0/813937292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666010d0e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 --2- 172.21.15.143:0/813937292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6660100de0 0x7f6660104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 --2- 172.21.15.143:0/813937292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f6660100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 -- 172.21.15.143:0/813937292 >> 172.21.15.143:0/813937292 conn(0x7f6660077230 msgr2=0x7f66600fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:02.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 -- 172.21.15.143:0/813937292 shutdown_connections 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.840+0000 7f6667b30640 1 -- 172.21.15.143:0/813937292 wait complete. 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 Processor -- start 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 -- start start 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f666019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6660100de0 0x7f666019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f666010f740 con 0x7f6660105560 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f666010f5c0 con 0x7f6660100de0 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f6667b30640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f666010f8c0 con 0x7f6660100430 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f66658cc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f666019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f66658cc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f666019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35468/0 (socket says 172.21.15.143:35468) 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.841+0000 7f66658cc640 1 -- 172.21.15.143:0/4225743035 learned_addr learned my addr 172.21.15.143:0/4225743035 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:02.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66650cb640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6660100de0 0x7f666019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66660cd640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66658cc640 1 -- 172.21.15.143:0/4225743035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6660100de0 msgr2=0x7f666019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66658cc640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6660100de0 0x7f666019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66658cc640 1 -- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 msgr2=0x7f666019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66658cc640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66658cc640 1 -- 172.21.15.143:0/4225743035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666019f4a0 con 0x7f6660100430 2024-09-10T10:50:02.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66660cd640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f66658cc640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f666019a4f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f665c00ea60 tx=0x7f665c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f6656ffd640 1 -- 172.21.15.143:0/4225743035 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c00cde0 con 0x7f6660100430 2024-09-10T10:50:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f6656ffd640 1 -- 172.21.15.143:0/4225743035 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665c0035f0 con 0x7f6660100430 2024-09-10T10:50:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666019f790 con 0x7f6660100430 2024-09-10T10:50:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f6656ffd640 1 -- 172.21.15.143:0/4225743035 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c010690 con 0x7f6660100430 2024-09-10T10:50:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.842+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601a7070 con 0x7f6660100430 2024-09-10T10:50:02.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.845+0000 7f6656ffd640 1 -- 172.21.15.143:0/4225743035 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f665c0037a0 con 0x7f6660100430 2024-09-10T10:50:02.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 Processor -- start 2024-09-10T10:50:02.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 -- start start 2024-09-10T10:50:02.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f605d0 0x5583c7f60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5583c7ec62f0 con 0x5583c7f6b370 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583c7e6dcf0 con 0x5583c7f605d0 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.846+0000 7f6669f09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5583c7ec22b0 con 0x5583c7f61b40 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f6656ffd640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f664c0a2c00 0x7f664c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66658cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f605d0 0x5583c7f60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f6d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42776/0 (socket says 172.21.15.143:42776) 2024-09-10T10:50:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 -- 172.21.15.143:0/1512514588 learned_addr learned my addr 172.21.15.143:0/1512514588 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 -- 172.21.15.143:0/1512514588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 msgr2=0x5583c7f60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 -- 172.21.15.143:0/1512514588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f605d0 msgr2=0x5583c7f60a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f6656ffd640 1 -- 172.21.15.143:0/4225743035 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f665c0cf6f0 con 0x7f6660100430 2024-09-10T10:50:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f605d0 0x5583c7f60a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 -- 172.21.15.143:0/1512514588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c7e523a0 con 0x5583c7f6b370 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66658cc640 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f605d0 0x5583c7f60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66650cb640 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f6d760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6650009c20 tx=0x7f6650011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaa224d6cbf07ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f66660cd640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f664c0a2c00 0x7f664c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.847+0000 7f664bfff640 1 -- 172.21.15.143:0/1512514588 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650004230 con 0x5583c7f6b370 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.848+0000 7f664bfff640 1 -- 172.21.15.143:0/1512514588 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66500043d0 con 0x5583c7f6b370 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.848+0000 7f664bfff640 1 -- 172.21.15.143:0/1512514588 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500196c0 con 0x5583c7f6b370 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 -- 172.21.15.143:0/1512514588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 msgr2=0x5583c7f6d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f6d760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6650009c20 tx=0x7f6650011810 comp rx=0 tx=0).stop 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f66660cd640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f664c0a2c00 0x7f664c0a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6658005cc0 tx=0x7f6658005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 -- 172.21.15.143:0/1512514588 shutdown_connections 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f6d760 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f605d0 0x5583c7f60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 --2- 172.21.15.143:0/1512514588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 -- 172.21.15.143:0/1512514588 >> 172.21.15.143:0/1512514588 conn(0x5583c7f29310 msgr2=0x5583c7f5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 -- 172.21.15.143:0/1512514588 shutdown_connections 2024-09-10T10:50:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.849+0000 7f6669f09740 1 -- 172.21.15.143:0/1512514588 wait complete. 2024-09-10T10:50:02.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f6669f09740 1 -- start start 2024-09-10T10:50:02.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f6669f09740 1 Processor -- start 2024-09-10T10:50:02.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f6669f09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 0x5583c7f6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f6669f09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f6669f09740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f68530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f66658cc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 0x5583c7f6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f66658cc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 0x5583c7f6ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35492/0 (socket says 172.21.15.143:35492) 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f66658cc640 1 -- 172.21.15.143:0/2269452152 learned_addr learned my addr 172.21.15.143:0/2269452152 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66660cd640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f68530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.850+0000 7f6669f09740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5583c7f76880 con 0x5583c7f6b370 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583c7e81060 con 0x5583c7f61b40 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5583c7f76a00 con 0x5583c7f605d0 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f6669f09740 10 client.0 _ll_register_callbacks cb 0x5583c7bd48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 -- 172.21.15.143:0/2269452152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 msgr2=0x5583c7f6ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 0x5583c7f6ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 -- 172.21.15.143:0/2269452152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 msgr2=0x5583c7f68530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f68530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 -- 172.21.15.143:0/2269452152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c7c373c0 con 0x5583c7f61b40 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66660cd640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f68530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66658cc640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 0x5583c7f6ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66650cb640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f67fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6650005890 tx=0x7f6650003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad53580dc04e4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66497fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.851+0000 7f66497fa640 1 -- 172.21.15.143:0/2269452152 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665001f040 con 0x5583c7f61b40 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66500043d0 con 0x5583c7f61b40 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f66497fa640 1 -- 172.21.15.143:0/2269452152 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6650003a10 con 0x5583c7f61b40 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f66497fa640 1 -- 172.21.15.143:0/2269452152 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6650031050 con 0x5583c7f61b40 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f6669f09740 10 client.25940 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:02.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5583c7f82290 con 0x5583c7f61b40 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f66497fa640 1 -- 172.21.15.143:0/2269452152 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6650009e90 con 0x5583c7f61b40 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f66497fa640 1 -- 172.21.15.143:0/2269452152 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f6650018070 con 0x5583c7f61b40 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f6669f09740 10 client.25940 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.852+0000 7f6669f09740 10 client.25940 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.853+0000 7f6669f09740 20 client.25940 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.853+0000 7f6669f09740 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7f83a10 0x5583c7f85e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.853+0000 7f6669f09740 4 client.25940 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.853+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f66500043d0 con 0x5583c7f83a10 2024-09-10T10:50:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.853+0000 7f66658cc640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7f83a10 0x5583c7f85e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.854+0000 7f66497fa640 10 client.25940 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.854+0000 7f66658cc640 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7f83a10 0x5583c7f85e50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=857769d32fade470 server_cookie=6fae37706ebc52d2 in_seq=0 out_seq=0 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.854+0000 7f66497fa640 1 -- 172.21.15.143:0/2269452152 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f66500043d0 con 0x5583c7f83a10 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.854+0000 7f66497fa640 10 client.25940 handle_command_reply: tid=0 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.854+0000 7f6669f09740 1 client.25940 shutdown 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 msgr2=0x5583c7f67fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f67fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6650005890 tx=0x7f6650003820 comp rx=0 tx=0).stop 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 shutdown_connections 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7f83a10 0x5583c7f85e50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f6b370 0x5583c7f68530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f67fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 --2- 172.21.15.143:0/2269452152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f605d0 0x5583c7f6ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f66497fa640 0 client.25940 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 >> 172.21.15.143:0/2269452152 conn(0x5583c7f29310 msgr2=0x5583c7f62960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 shutdown_connections 2024-09-10T10:50:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 1 -- 172.21.15.143:0/2269452152 wait complete. 2024-09-10T10:50:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.855+0000 7f6669f09740 20 client.25940 trim_cache size 0 max 16384 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 Processor -- start 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 -- start start 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f5ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f7fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f6b370 0x5583c7f61300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66601ac900 con 0x5583c7f605d0 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66601ae470 con 0x5583c7f61b40 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f6669f09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66601acfd0 con 0x5583c7f6b370 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f66658cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f5ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f66658cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f5ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42786/0 (socket says 172.21.15.143:42786) 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.874+0000 7f66658cc640 1 -- 172.21.15.143:0/2542817544 learned_addr learned my addr 172.21.15.143:0/2542817544 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66660cd640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f6b370 0x5583c7f61300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66650cb640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f7fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66658cc640 1 -- 172.21.15.143:0/2542817544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f6b370 msgr2=0x5583c7f61300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66658cc640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f6b370 0x5583c7f61300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66658cc640 1 -- 172.21.15.143:0/2542817544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 msgr2=0x5583c7f7fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66658cc640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f7fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66658cc640 1 -- 172.21.15.143:0/2542817544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c7f66500 con 0x5583c7f605d0 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66660cd640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f6b370 0x5583c7f61300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66650cb640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f7fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f66658cc640 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f5ce10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f665c00efd0 tx=0x7f665c0c6720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ab9d6de12de359 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f664b7fe640 1 -- 172.21.15.143:0/2542817544 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c09bb90 con 0x5583c7f605d0 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f664b7fe640 1 -- 172.21.15.143:0/2542817544 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665c09ff40 con 0x5583c7f605d0 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.875+0000 7f664b7fe640 1 -- 172.21.15.143:0/2542817544 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c09afe0 con 0x5583c7f605d0 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 -- 172.21.15.143:0/2542817544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 msgr2=0x5583c7f5ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f5ce10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f665c00efd0 tx=0x7f665c0c6720 comp rx=0 tx=0).stop 2024-09-10T10:50:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 -- 172.21.15.143:0/2542817544 shutdown_connections 2024-09-10T10:50:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f6b370 0x5583c7f61300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f61b40 0x5583c7f7fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 --2- 172.21.15.143:0/2542817544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f5ce10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.876+0000 7f6669f09740 1 -- 172.21.15.143:0/2542817544 >> 172.21.15.143:0/2542817544 conn(0x5583c7f29310 msgr2=0x5583c7f5c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.877+0000 7f6669f09740 1 -- 172.21.15.143:0/2542817544 shutdown_connections 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.877+0000 7f6669f09740 1 -- 172.21.15.143:0/2542817544 wait complete. 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.877+0000 7f6669f09740 1 -- start start 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.877+0000 7f6669f09740 1 Processor -- start 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f7c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f6b370 0x5583c7f7b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66601add70 con 0x5583c7f605d0 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66601ad6a0 con 0x5583c7f6b370 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66601aa900 con 0x5583c7f61b40 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f6669f09740 10 client.0 _ll_register_callbacks cb 0x7f665c09b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:02.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f66658cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f7c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f66658cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f7c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42788/0 (socket says 172.21.15.143:42788) 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f66658cc640 1 -- 172.21.15.143:0/704516003 learned_addr learned my addr 172.21.15.143:0/704516003 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f66650cb640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.878+0000 7f66660cd640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f6b370 0x5583c7f7b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66658cc640 1 -- 172.21.15.143:0/704516003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 msgr2=0x5583c7f7acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66658cc640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f7acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66658cc640 1 -- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f6b370 msgr2=0x5583c7f7b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66658cc640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f6b370 0x5583c7f7b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66658cc640 1 -- 172.21.15.143:0/704516003 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c7f7b7e0 con 0x5583c7f605d0 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66650cb640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66658cc640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f7c1d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f665c0cfa60 tx=0x7f665c09bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40112c30ce94dfd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66497fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:02.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66497fa640 1 -- 172.21.15.143:0/704516003 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c099f50 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c09ff40 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66497fa640 1 -- 172.21.15.143:0/704516003 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f665c098200 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.879+0000 7f66497fa640 1 -- 172.21.15.143:0/704516003 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f665c0ba300 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 10 client.16578 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5583c7fed570 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f66497fa640 1 -- 172.21.15.143:0/704516003 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f665c09ff40 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f66497fa640 1 -- 172.21.15.143:0/704516003 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f665c0ce040 con 0x5583c7f605d0 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 10 client.16578 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 10 client.16578 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 20 client.16578 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7fedb70 0x5583c7feff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 4 client.16578 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583c7ed3860 con 0x5583c7fedb70 2024-09-10T10:50:02.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.880+0000 7f66650cb640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7fedb70 0x5583c7feff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.881+0000 7f66497fa640 10 client.16578 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.881+0000 7f66650cb640 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7fedb70 0x5583c7feff90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3933f0b21d695c0 server_cookie=e49e4ba216797f78 in_seq=0 out_seq=0 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.881+0000 7f66497fa640 1 -- 172.21.15.143:0/704516003 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583c7ed3860 con 0x5583c7fedb70 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.881+0000 7f66497fa640 10 client.16578 handle_command_reply: tid=0 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.881+0000 7f6669f09740 1 client.16578 shutdown 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 msgr2=0x5583c7f7c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f7c1d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f665c0cfa60 tx=0x7f665c09bb90 comp rx=0 tx=0).stop 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 shutdown_connections 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x5583c7fedb70 0x5583c7feff90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583c7f6b370 0x5583c7f7b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5583c7f61b40 0x5583c7f7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 --2- 172.21.15.143:0/704516003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583c7f605d0 0x5583c7f7c1d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f66497fa640 0 client.16578 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 >> 172.21.15.143:0/704516003 conn(0x5583c7f29310 msgr2=0x5583c7f69810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 shutdown_connections 2024-09-10T10:50:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 1 -- 172.21.15.143:0/704516003 wait complete. 2024-09-10T10:50:02.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.882+0000 7f6669f09740 20 client.16578 trim_cache size 0 max 16384 2024-09-10T10:50:02.887 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f664c0a2c00 msgr2=0x7f664c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f664c0a2c00 0x7f664c0a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6658005cc0 tx=0x7f6658005f80 comp rx=0 tx=0).stop 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 msgr2=0x7f666019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f666019a4f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f665c00ea60 tx=0x7f665c00ef20 comp rx=0 tx=0).stop 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 shutdown_connections 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f664c0a2c00 0x7f664c0a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6660105560 0x7f666019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6660100de0 0x7f666019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 --2- 172.21.15.143:0/4225743035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6660100430 0x7f666019a4f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 >> 172.21.15.143:0/4225743035 conn(0x7f6660077230 msgr2=0x7f6660102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:02.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 shutdown_connections 2024-09-10T10:50:02.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:02.883+0000 7f6667b30640 1 -- 172.21.15.143:0/4225743035 wait complete. 2024-09-10T10:50:03.591 INFO:tasks.workunit.client.0.smithi143.stdout: 76 11608 0 7717 0 4123 0 2024-09-10T10:50:03.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:50:03.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:50:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 -- 172.21.15.143:0/3271315096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 msgr2=0x7f8c7010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/3271315096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 0x7f8c7010ec70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009c10 tx=0x7f8c6802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 -- 172.21.15.143:0/3271315096 shutdown_connections 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/3271315096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 0x7f8c7010ec70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/3271315096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/3271315096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c70102dd0 0x7f8c701031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 -- 172.21.15.143:0/3271315096 >> 172.21.15.143:0/3271315096 conn(0x7f8c700fc7d0 msgr2=0x7f8c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 -- 172.21.15.143:0/3271315096 shutdown_connections 2024-09-10T10:50:04.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.749+0000 7f8c74bb0640 1 -- 172.21.15.143:0/3271315096 wait complete. 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.750+0000 7f8c74bb0640 1 Processor -- start 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.750+0000 7f8c74bb0640 1 -- start start 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c74bb0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c70102dd0 0x7f8c701984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c74bb0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c74bb0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 0x7f8c7019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c74bb0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70111f80 con 0x7f8c70108300 2024-09-10T10:50:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c74bb0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70111e00 con 0x7f8c70102dd0 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c74bb0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70112100 con 0x7f8c70103780 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6ed9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 0x7f8c7019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:35530/0 (socket says 172.21.15.143:35530) 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 -- 172.21.15.143:0/1724209275 learned_addr learned my addr 172.21.15.143:0/1724209275 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6e59c640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c70102dd0 0x7f8c701984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 -- 172.21.15.143:0/1724209275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c70102dd0 msgr2=0x7f8c701984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c70102dd0 0x7f8c701984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 -- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 msgr2=0x7f8c7019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 0x7f8c7019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.751+0000 7f8c6dd9b640 1 -- 172.21.15.143:0/1724209275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7019d470 con 0x7f8c70103780 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.752+0000 7f8c6dd9b640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70198a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400e9c0 tx=0x7f8c6400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.752+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400ccc0 con 0x7f8c70103780 2024-09-10T10:50:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.752+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7019d760 con 0x7f8c70103780 2024-09-10T10:50:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.752+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c640108e0 con 0x7f8c70103780 2024-09-10T10:50:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.752+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640187e0 con 0x7f8c70103780 2024-09-10T10:50:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.752+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c70100960 con 0x7f8c70103780 2024-09-10T10:50:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.753+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c701031b0 con 0x7f8c70103780 2024-09-10T10:50:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:04.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.755+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8c64010a90 con 0x7f8c70103780 2024-09-10T10:50:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.755+0000 7f8c5f7fe640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8c580a2b80 0x7f8c580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.755+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8c640cf630 con 0x7f8c70103780 2024-09-10T10:50:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.755+0000 7f8c6e59c640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8c580a2b80 0x7f8c580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.756+0000 7f8c6e59c640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8c580a2b80 0x7f8c580a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009aa0 tx=0x7f8c60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:04.760+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c6409c230 con 0x7f8c70103780 2024-09-10T10:50:04.887 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:04 smithi143 ceph-mon[31057]: pgmap v501: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-09-10T10:50:04.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:04 smithi178 ceph-mon[34813]: pgmap v501: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-09-10T10:50:04.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:04 smithi144 ceph-mon[34230]: pgmap v501: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-09-10T10:50:05.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.022+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c7010a290 con 0x7f8c70103780 2024-09-10T10:50:05.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.023+0000 7f8c5f7fe640 1 -- 172.21.15.143:0/1724209275 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12006 (secure 0 0 0) 0x7f8c6409bb30 con 0x7f8c70103780 2024-09-10T10:50:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:50:05.025 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"btime":"2024-09-10T10:48:11:574840+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:48:11.253122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:50:05.025 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 32 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.025+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8c580a2b80 msgr2=0x7f8c580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.025+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8c580a2b80 0x7f8c580a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009aa0 tx=0x7f8c60005f90 comp rx=0 tx=0).stop 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.025+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 msgr2=0x7f8c70198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.025+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70198a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400e9c0 tx=0x7f8c6400ee80 comp rx=0 tx=0).stop 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 shutdown_connections 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8c580a2b80 0x7f8c580a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c70108300 0x7f8c7019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c70103780 0x7f8c70198a00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 --2- 172.21.15.143:0/1724209275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c70102dd0 0x7f8c701984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 >> 172.21.15.143:0/1724209275 conn(0x7f8c700fc7d0 msgr2=0x7f8c70105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:05.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 shutdown_connections 2024-09-10T10:50:05.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:05.026+0000 7f8c74bb0640 1 -- 172.21.15.143:0/1724209275 wait complete. 2024-09-10T10:50:05.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 8ac87a67 --release --await' 2024-09-10T10:50:05.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 8ac87a67 --release --await 2024-09-10T10:50:05.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:05 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1724209275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:05.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:05 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1724209275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:05.967 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:05 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1724209275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:06.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1818922605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff4102de0 msgr2=0x7f5ff41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1818922605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff4102de0 0x7f5ff41031c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009900 tx=0x7f5fe802a480 comp rx=0 tx=0).stop 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1818922605 shutdown_connections 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1818922605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410c180 0x7f5ff410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1818922605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 0x7f5ff410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1818922605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff4102de0 0x7f5ff41031c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1818922605 >> 172.21.15.143:0/1818922605 conn(0x7f5ff40fc7a0 msgr2=0x7f5ff40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1818922605 shutdown_connections 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.056+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1818922605 wait complete. 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 Processor -- start 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 -- start start 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4102de0 0x7f5ff41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 0x7f5ff41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 0x7f5ff419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4111de0 con 0x7f5ff410c180 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4111c60 con 0x7f5ff4102de0 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ffb97b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4111f60 con 0x7f5ff4103700 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 0x7f5ff419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 0x7f5ff419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42822/0 (socket says 172.21.15.143:42822) 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 -- 172.21.15.143:0/1475741352 learned_addr learned my addr 172.21.15.143:0/1475741352 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 0x7f5ff41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9717640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4102de0 0x7f5ff41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 -- 172.21.15.143:0/1475741352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 msgr2=0x7f5ff41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 0x7f5ff41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 -- 172.21.15.143:0/1475741352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4102de0 msgr2=0x7f5ff41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4102de0 0x7f5ff41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff9f18640 1 -- 172.21.15.143:0/1475741352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff419d350 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.057+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 0x7f5ff41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.058+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 0x7f5ff419cc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00eaa0 tx=0x7f5fec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.058+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1475741352 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00cdb0 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.058+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff419d640 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.058+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1475741352 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fec005020 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.058+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a4f20 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.058+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1475741352 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec01f060 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.060+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1475741352 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5fec0051d0 con 0x7f5ff410c180 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 Processor -- start 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 -- start start 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5fe67fc640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5fe00a2b80 0x7f5fe00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e182f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 0x5630e182fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e1840b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5630e17952f0 con 0x5630e182f5d0 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5630e173ccf0 con 0x5630e1830b40 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5630e17912b0 con 0x5630e183e780 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5fe67fc640 1 -- 172.21.15.143:0/1475741352 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5fec0cf5d0 con 0x7f5ff410c180 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5fe00a2b80 0x7f5fe00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 0x5630e182fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 0x5630e182fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42826/0 (socket says 172.21.15.143:42826) 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 -- 172.21.15.143:0/574890625 learned_addr learned my addr 172.21.15.143:0/574890625 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff9717640 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e182f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e1840b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 -- 172.21.15.143:0/574890625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 msgr2=0x5630e1840b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e1840b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 -- 172.21.15.143:0/574890625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 msgr2=0x5630e182f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.061+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e182f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.062+0000 7f5ff8f16640 1 -- 172.21.15.143:0/574890625 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e17213a0 con 0x5630e182f5d0 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.062+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5fe00a2b80 0x7f5fe00a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000cfd0 tx=0x7f5ff000cc60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.062+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 0x5630e182fa10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0011ce0 tx=0x7f5ff001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7f6810431756dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.062+0000 7f5fdf7fe640 1 -- 172.21.15.143:0/574890625 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0022070 con 0x5630e182f5d0 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.062+0000 7f5fdf7fe640 1 -- 172.21.15.143:0/574890625 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ff000fc60 con 0x5630e182f5d0 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.062+0000 7f5fdf7fe640 1 -- 172.21.15.143:0/574890625 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0012940 con 0x5630e182f5d0 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 -- 172.21.15.143:0/574890625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 msgr2=0x5630e182fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 0x5630e182fa10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0011ce0 tx=0x7f5ff001b000 comp rx=0 tx=0).stop 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 -- 172.21.15.143:0/574890625 shutdown_connections 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e1840b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e182f5d0 0x5630e182fa10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/574890625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e182f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 -- 172.21.15.143:0/574890625 >> 172.21.15.143:0/574890625 conn(0x5630e17f8310 msgr2=0x5630e182bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 -- 172.21.15.143:0/574890625 shutdown_connections 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 -- 172.21.15.143:0/574890625 wait complete. 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 -- start start 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.063+0000 7f5ffdd54740 1 Processor -- start 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e182f5d0 0x5630e183e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 0x5630e183b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e183bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5630e1841420 con 0x5630e1830b40 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5630e1750060 con 0x5630e182f5d0 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5630e18415a0 con 0x5630e183e780 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 10 client.0 _ll_register_callbacks cb 0x5630e14a38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 0x5630e183b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 0x5630e183b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42834/0 (socket says 172.21.15.143:42834) 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 -- 172.21.15.143:0/3523573506 learned_addr learned my addr 172.21.15.143:0/3523573506 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e183bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff9717640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e182f5d0 0x5630e183e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 -- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 msgr2=0x5630e183bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e183bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 -- 172.21.15.143:0/3523573506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e182f5d0 msgr2=0x5630e183e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e182f5d0 0x5630e183e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 -- 172.21.15.143:0/3523573506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e15063c0 con 0x5630e1830b40 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e183bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 0x5630e183b470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000fc30 tx=0x7f5ff00147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5bd7593dc08011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5fdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5fdcff9640 1 -- 172.21.15.143:0/3523573506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0022070 con 0x5630e1830b40 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5fdcff9640 1 -- 172.21.15.143:0/3523573506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5ff0014270 con 0x5630e1830b40 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff000fc60 con 0x5630e1830b40 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5fdcff9640 1 -- 172.21.15.143:0/3523573506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5ff0038050 con 0x5630e1830b40 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.064+0000 7f5ffdd54740 10 client.16596 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5630e184cf60 con 0x5630e1830b40 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5fdcff9640 1 -- 172.21.15.143:0/3523573506 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ff000fc60 con 0x5630e1830b40 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5fdcff9640 1 -- 172.21.15.143:0/3523573506 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f5ff0004680 con 0x5630e1830b40 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 10 client.16596 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 10 client.16596 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 20 client.16596 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e184e310 0x5630e1850700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 4 client.16596 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e17a2860 con 0x5630e184e310 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.065+0000 7f5ff9717640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e184e310 0x5630e1850700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.066+0000 7f5fdcff9640 10 client.16596 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.066+0000 7f5ff9717640 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e184e310 0x5630e1850700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=127419333e74f7f8 server_cookie=df9dee76f4b14943 in_seq=0 out_seq=0 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.066+0000 7f5fdcff9640 1 -- 172.21.15.143:0/3523573506 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630e17a2860 con 0x5630e184e310 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.066+0000 7f5fdcff9640 10 client.16596 handle_command_reply: tid=0 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.066+0000 7f5ffdd54740 1 client.16596 shutdown 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 msgr2=0x5630e183b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 0x5630e183b470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000fc30 tx=0x7f5ff00147f0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 shutdown_connections 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e184e310 0x5630e1850700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e183e780 0x5630e183bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e1830b40 0x5630e183b470 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/3523573506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e182f5d0 0x5630e183e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5fdcff9640 0 client.16596 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 >> 172.21.15.143:0/3523573506 conn(0x5630e17f8310 msgr2=0x5630e1831960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 shutdown_connections 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 1 -- 172.21.15.143:0/3523573506 wait complete. 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.067+0000 7f5ffdd54740 20 client.16596 trim_cache size 0 max 16384 2024-09-10T10:50:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 Processor -- start 2024-09-10T10:50:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 -- start start 2024-09-10T10:50:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e183b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e1826ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1855ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41ac9a0 con 0x5630e183e780 2024-09-10T10:50:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41ae510 con 0x5630e1830b40 2024-09-10T10:50:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41ad070 con 0x5630e182f5d0 2024-09-10T10:50:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1855ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1855ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42842/0 (socket says 172.21.15.143:42842) 2024-09-10T10:50:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 -- 172.21.15.143:0/4176711847 learned_addr learned my addr 172.21.15.143:0/4176711847 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:06.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e1826ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 -- 172.21.15.143:0/4176711847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 msgr2=0x5630e183b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9717640 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e183b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e183b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 -- 172.21.15.143:0/4176711847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 msgr2=0x5630e1826ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e1826ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.080+0000 7f5ff9f18640 1 -- 172.21.15.143:0/4176711847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e183a420 con 0x5630e183e780 2024-09-10T10:50:06.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1855ee0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00efd0 tx=0x7f5fec09c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32bc2aa0f45f7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5fdeffd640 1 -- 172.21.15.143:0/4176711847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0b9070 con 0x5630e183e780 2024-09-10T10:50:06.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5fdeffd640 1 -- 172.21.15.143:0/4176711847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fec0a4740 con 0x5630e183e780 2024-09-10T10:50:06.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5fdeffd640 1 -- 172.21.15.143:0/4176711847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec098140 con 0x5630e183e780 2024-09-10T10:50:06.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 -- 172.21.15.143:0/4176711847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 msgr2=0x5630e1855ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1855ee0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00efd0 tx=0x7f5fec09c290 comp rx=0 tx=0).stop 2024-09-10T10:50:06.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 -- 172.21.15.143:0/4176711847 shutdown_connections 2024-09-10T10:50:06.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1855ee0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e1826ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/4176711847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e183b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.081+0000 7f5ffdd54740 1 -- 172.21.15.143:0/4176711847 >> 172.21.15.143:0/4176711847 conn(0x5630e17f8310 msgr2=0x5630e183ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 -- 172.21.15.143:0/4176711847 shutdown_connections 2024-09-10T10:50:06.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 -- 172.21.15.143:0/4176711847 wait complete. 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 -- start start 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 Processor -- start 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e1846f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e18380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.082+0000 7f5ffdd54740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1838610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41ade10 con 0x5630e183e780 2024-09-10T10:50:06.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41ad740 con 0x5630e1830b40 2024-09-10T10:50:06.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41aa980 con 0x5630e182f5d0 2024-09-10T10:50:06.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1838610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1838610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42850/0 (socket says 172.21.15.143:42850) 2024-09-10T10:50:06.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 -- 172.21.15.143:0/713886273 learned_addr learned my addr 172.21.15.143:0/713886273 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:06.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 10 client.0 _ll_register_callbacks cb 0x7f5fe8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:06.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e18380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9717640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e1846f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 -- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 msgr2=0x5630e1846f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e1846f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 -- 172.21.15.143:0/713886273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 msgr2=0x5630e18380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e18380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 -- 172.21.15.143:0/713886273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e183bf70 con 0x5630e183e780 2024-09-10T10:50:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff8f16640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e18380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ff9f18640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1838610 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0ccdc0 tx=0x7f5fec099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aea67377ac017d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5fdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5fdcff9640 1 -- 172.21.15.143:0/713886273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec065070 con 0x5630e183e780 2024-09-10T10:50:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5fdcff9640 1 -- 172.21.15.143:0/713886273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5fec0cf310 con 0x5630e183e780 2024-09-10T10:50:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fec0a4740 con 0x5630e183e780 2024-09-10T10:50:06.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5fdcff9640 1 -- 172.21.15.143:0/713886273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5fec0b9070 con 0x5630e183e780 2024-09-10T10:50:06.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 10 client.16602 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:06.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.083+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5630e18b8bf0 con 0x5630e183e780 2024-09-10T10:50:06.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5fdcff9640 1 -- 172.21.15.143:0/713886273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fec0a4740 con 0x5630e183e780 2024-09-10T10:50:06.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5fdcff9640 1 -- 172.21.15.143:0/713886273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f5fec00f040 con 0x5630e183e780 2024-09-10T10:50:06.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ffdd54740 10 client.16602 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ffdd54740 10 client.16602 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:50:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ffdd54740 20 client.16602 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e18b92f0 0x5630e18bb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ffdd54740 4 client.16602 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:50:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630e179cd70 con 0x5630e18b92f0 2024-09-10T10:50:06.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ff9717640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e18b92f0 0x5630e18bb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5fdcff9640 10 client.16602 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:50:06.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.084+0000 7f5ff9717640 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e18b92f0 0x5630e18bb710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=136c5b1e8ff347e4 server_cookie=cb5cbbc33ce6708 in_seq=0 out_seq=0 2024-09-10T10:50:06.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5fdcff9640 1 -- 172.21.15.143:0/713886273 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630e179cd70 con 0x5630e18b92f0 2024-09-10T10:50:06.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5fdcff9640 10 client.16602 handle_command_reply: tid=0 2024-09-10T10:50:06.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 client.16602 shutdown 2024-09-10T10:50:06.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 msgr2=0x5630e1838610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1838610 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0ccdc0 tx=0x7f5fec099be0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 shutdown_connections 2024-09-10T10:50:06.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x5630e18b92f0 0x5630e18bb710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5630e183e780 0x5630e1838610 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630e1830b40 0x5630e18380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 --2- 172.21.15.143:0/713886273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5630e182f5d0 0x5630e1846f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5fdcff9640 0 client.16602 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:50:06.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 >> 172.21.15.143:0/713886273 conn(0x5630e17f8310 msgr2=0x5630e182a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 shutdown_connections 2024-09-10T10:50:06.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 1 -- 172.21.15.143:0/713886273 wait complete. 2024-09-10T10:50:06.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.085+0000 7f5ffdd54740 20 client.16602 trim_cache size 0 max 16384 2024-09-10T10:50:06.107 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:50:06.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5fe00a2b80 msgr2=0x7f5fe00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5fe00a2b80 0x7f5fe00a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000cfd0 tx=0x7f5ff000cc60 comp rx=0 tx=0).stop 2024-09-10T10:50:06.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 msgr2=0x7f5ff419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 0x7f5ff419cc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00eaa0 tx=0x7f5fec00ef60 comp rx=0 tx=0).stop 2024-09-10T10:50:06.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 shutdown_connections 2024-09-10T10:50:06.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f5fe00a2b80 0x7f5fe00a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff410c180 0x7f5ff419cc70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ff4103700 0x7f5ff41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 --2- 172.21.15.143:0/1475741352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4102de0 0x7f5ff41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 >> 172.21.15.143:0/1475741352 conn(0x7f5ff40fc7a0 msgr2=0x7f5ff40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 shutdown_connections 2024-09-10T10:50:06.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:06.086+0000 7f5ffb97b640 1 -- 172.21.15.143:0/1475741352 wait complete. 2024-09-10T10:50:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:06 smithi178 ceph-mon[34813]: pgmap v502: 129 pgs: 129 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-10T10:50:06.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:06 smithi144 ceph-mon[34230]: pgmap v502: 129 pgs: 129 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-10T10:50:07.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:06 smithi143 ceph-mon[31057]: pgmap v502: 129 pgs: 129 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-10T10:50:08.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:08 smithi178 ceph-mon[34813]: pgmap v503: 129 pgs: 129 active+clean; 358 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-09-10T10:50:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:08 smithi144 ceph-mon[34230]: pgmap v503: 129 pgs: 129 active+clean; 358 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-09-10T10:50:09.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:08 smithi143 ceph-mon[31057]: pgmap v503: 129 pgs: 129 active+clean; 358 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-09-10T10:50:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:10 smithi178 ceph-mon[34813]: pgmap v504: 129 pgs: 129 active+clean; 358 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T10:50:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:10 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:50:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:10 smithi144 ceph-mon[34230]: pgmap v504: 129 pgs: 129 active+clean; 358 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T10:50:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:10 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:50:11.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:10 smithi143 ceph-mon[31057]: pgmap v504: 129 pgs: 129 active+clean; 358 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T10:50:11.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:10 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:50:11.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:50:11.834 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:12 smithi178 ceph-mon[34813]: pgmap v505: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-09-10T10:50:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:12 smithi144 ceph-mon[34230]: pgmap v505: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-09-10T10:50:13.047 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:12 smithi143 ceph-mon[31057]: pgmap v505: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-09-10T10:50:13.591 INFO:tasks.workunit.client.0.smithi143.stdout: 76 1702 0 1159 0 554 0 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 -- 172.21.15.143:0/3040303038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478100a80 msgr2=0x7fb478100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 --2- 172.21.15.143:0/3040303038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478100a80 0x7fb478100ee0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb464009930 tx=0x7fb46402a420 comp rx=0 tx=0).stop 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 -- 172.21.15.143:0/3040303038 shutdown_connections 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 --2- 172.21.15.143:0/3040303038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 0x7fb47810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 --2- 172.21.15.143:0/3040303038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478100a80 0x7fb478100ee0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 --2- 172.21.15.143:0/3040303038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478106a80 0x7fb478106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 -- 172.21.15.143:0/3040303038 >> 172.21.15.143:0/3040303038 conn(0x7fb4780fc7a0 msgr2=0x7fb4780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 -- 172.21.15.143:0/3040303038 shutdown_connections 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.681+0000 7fb47ef75640 1 -- 172.21.15.143:0/3040303038 wait complete. 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 Processor -- start 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 -- start start 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478100a80 0x7fb478197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 0x7fb478197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 0x7fb47819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb478110dd0 con 0x7fb478106a80 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb478110c50 con 0x7fb478101420 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47ef75640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb478110f50 con 0x7fb478100a80 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47d512640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 0x7fb47819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47d512640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 0x7fb47819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55758/0 (socket says 172.21.15.143:55758) 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.682+0000 7fb47d512640 1 -- 172.21.15.143:0/986933631 learned_addr learned my addr 172.21.15.143:0/986933631 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb477fff640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 0x7fb478197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47cd11640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478100a80 0x7fb478197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47d512640 1 -- 172.21.15.143:0/986933631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478100a80 msgr2=0x7fb478197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47d512640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478100a80 0x7fb478197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47d512640 1 -- 172.21.15.143:0/986933631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 msgr2=0x7fb478197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47d512640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 0x7fb478197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47d512640 1 -- 172.21.15.143:0/986933631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47819c3e0 con 0x7fb478106a80 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb477fff640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 0x7fb478197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47cd11640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478100a80 0x7fb478197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47d512640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 0x7fb47819bd00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb47000b970 tx=0x7fb47000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb475ffb640 1 -- 172.21.15.143:0/986933631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470019070 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb475ffb640 1 -- 172.21.15.143:0/986933631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb470014440 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47819c6d0 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb475ffb640 1 -- 172.21.15.143:0/986933631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700075d0 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.683+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4781a3fb0 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.685+0000 7fb475ffb640 1 -- 172.21.15.143:0/986933631 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb47001c070 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb475ffb640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb4680a2b80 0x7fb4680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 Processor -- start 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 -- start start 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb47cd11640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb4680a2b80 0x7fb4680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb475ffb640 1 -- 172.21.15.143:0/986933631 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb4700cea20 con 0x7fb478106a80 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559e22710620 con 0x559e227ab980 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e226b7e50 con 0x559e227b6240 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb48134e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559e2270c5e0 con 0x559e227ab060 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb477fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb477fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55764/0 (socket says 172.21.15.143:55764) 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb477fff640 1 -- 172.21.15.143:0/1533116645 learned_addr learned my addr 172.21.15.143:0/1533116645 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb47cd11640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb4680a2b80 0x7fb4680a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb46c006fd0 tx=0x7fb46c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb47cd11640 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb477fff640 1 -- 172.21.15.143:0/1533116645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 msgr2=0x559e227ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb477fff640 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb47d512640 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.686+0000 7fb477fff640 1 -- 172.21.15.143:0/1533116645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 msgr2=0x559e227b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb477fff640 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb477fff640 1 -- 172.21.15.143:0/1533116645 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e226a8d40 con 0x559e227ab980 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb47d512640 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb477fff640 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b5d00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb464000c00 tx=0x7fb464032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9bfa2102179aac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb462ffd640 1 -- 172.21.15.143:0/1533116645 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46403f070 con 0x559e227ab980 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb462ffd640 1 -- 172.21.15.143:0/1533116645 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb464031760 con 0x559e227ab980 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb462ffd640 1 -- 172.21.15.143:0/1533116645 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464035070 con 0x559e227ab980 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb48134e740 1 -- 172.21.15.143:0/1533116645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 msgr2=0x559e227b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.687+0000 7fb48134e740 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b5d00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb464000c00 tx=0x7fb464032e50 comp rx=0 tx=0).stop 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 -- 172.21.15.143:0/1533116645 shutdown_connections 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b5d00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 --2- 172.21.15.143:0/1533116645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 -- 172.21.15.143:0/1533116645 >> 172.21.15.143:0/1533116645 conn(0x559e227a75f0 msgr2=0x559e227a79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 -- 172.21.15.143:0/1533116645 shutdown_connections 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 -- 172.21.15.143:0/1533116645 wait complete. 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 -- start start 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.688+0000 7fb48134e740 1 Processor -- start 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab060 0x559e227b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559e227b8e10 con 0x559e227ab980 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e22698f10 con 0x559e227ab060 2024-09-10T10:50:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559e227b8f90 con 0x559e227b6240 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 10 client.0 _ll_register_callbacks cb 0x559e226bcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55776/0 (socket says 172.21.15.143:55776) 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 -- 172.21.15.143:0/838935823 learned_addr learned my addr 172.21.15.143:0/838935823 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb47d512640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb47cd11640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab060 0x559e227b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 -- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 msgr2=0x559e227b9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227b9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 -- 172.21.15.143:0/838935823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab060 msgr2=0x559e227b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab060 0x559e227b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 -- 172.21.15.143:0/838935823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2241f580 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb47d512640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb47cd11640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab060 0x559e227b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb477fff640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b9760 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb46402a460 tx=0x7fb464005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d993e7c894600920 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb457fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb457fff640 1 -- 172.21.15.143:0/838935823 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46403f070 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb457fff640 1 -- 172.21.15.143:0/838935823 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb46403b460 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb464031760 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb457fff640 1 -- 172.21.15.143:0/838935823 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb464053050 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 10 client.16614 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.689+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e224e9960 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb457fff640 1 -- 172.21.15.143:0/838935823 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb464031760 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb457fff640 1 -- 172.21.15.143:0/838935823 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb464044650 con 0x559e227ab980 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb48134e740 10 client.16614 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb48134e740 10 client.16614 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb48134e740 20 client.16614 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb48134e740 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e227c7df0 0x559e227ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb48134e740 4 client.16614 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e2271db90 con 0x559e227c7df0 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb47cd11640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e227c7df0 0x559e227ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb47cd11640 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e227c7df0 0x559e227ca1e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0173582671a531a server_cookie=3d8e23c087629995 in_seq=0 out_seq=0 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.690+0000 7fb457fff640 10 client.16614 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb457fff640 1 -- 172.21.15.143:0/838935823 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e2271db90 con 0x559e227c7df0 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb457fff640 10 client.16614 handle_command_reply: tid=0 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 client.16614 shutdown 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 msgr2=0x559e227b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b9760 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb46402a460 tx=0x7fb464005740 comp rx=0 tx=0).stop 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 shutdown_connections 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e227c7df0 0x559e227ca1e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227b9760 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb457fff640 0 client.16614 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 --2- 172.21.15.143:0/838935823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab060 0x559e227b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.691+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 >> 172.21.15.143:0/838935823 conn(0x559e227a75f0 msgr2=0x559e227ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.692+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 shutdown_connections 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.692+0000 7fb48134e740 1 -- 172.21.15.143:0/838935823 wait complete. 2024-09-10T10:50:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.692+0000 7fb48134e740 20 client.16614 trim_cache size 0 max 16384 2024-09-10T10:50:13.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.708+0000 7fb48134e740 1 Processor -- start 2024-09-10T10:50:13.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.708+0000 7fb48134e740 1 -- start start 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb48134e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb48134e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb48134e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227be720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb48134e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ac9b0 con 0x559e227ab980 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb48134e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ae4f0 con 0x559e227b6240 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb48134e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ad080 con 0x559e227ab060 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227be1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55778/0 (socket says 172.21.15.143:55778) 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 -- 172.21.15.143:0/130660981 learned_addr learned my addr 172.21.15.143:0/130660981 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb47d512640 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb47cd11640 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 -- 172.21.15.143:0/130660981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 msgr2=0x559e227a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 -- 172.21.15.143:0/130660981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 msgr2=0x559e227be720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227be720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 -- 172.21.15.143:0/130660981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e227a6e70 con 0x559e227ab980 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb47cd11640 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb477fff640 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227be1e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb464044c10 tx=0x7fb464032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8767190f65a2836a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.709+0000 7fb4627fc640 1 -- 172.21.15.143:0/130660981 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46403f040 con 0x559e227ab980 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb4627fc640 1 -- 172.21.15.143:0/130660981 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb464048020 con 0x559e227ab980 2024-09-10T10:50:13.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb4627fc640 1 -- 172.21.15.143:0/130660981 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464035040 con 0x559e227ab980 2024-09-10T10:50:13.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 -- 172.21.15.143:0/130660981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 msgr2=0x559e227be1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227be1e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb464044c10 tx=0x7fb464032fc0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 -- 172.21.15.143:0/130660981 shutdown_connections 2024-09-10T10:50:13.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227b6240 0x559e227be720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab980 0x559e227be1e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 --2- 172.21.15.143:0/130660981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227ab060 0x559e227a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.710+0000 7fb48134e740 1 -- 172.21.15.143:0/130660981 >> 172.21.15.143:0/130660981 conn(0x559e227a75f0 msgr2=0x559e227c3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 -- 172.21.15.143:0/130660981 shutdown_connections 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 -- 172.21.15.143:0/130660981 wait complete. 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 -- start start 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 Processor -- start 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 0x559e227bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab980 0x559e227bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ade20 con 0x559e227ab060 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ad750 con 0x559e227ab980 2024-09-10T10:50:13.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.711+0000 7fb48134e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781aa990 con 0x559e227b6240 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb48134e740 10 client.0 _ll_register_callbacks cb 0x7fb46402aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 0x559e227bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 0x559e227bcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55782/0 (socket says 172.21.15.143:55782) 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 -- 172.21.15.143:0/1196710095 learned_addr learned my addr 172.21.15.143:0/1196710095 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb477fff640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab980 0x559e227bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47d512640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 -- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 msgr2=0x559e227bd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227bd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 -- 172.21.15.143:0/1196710095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab980 msgr2=0x559e227bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab980 0x559e227bd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 -- 172.21.15.143:0/1196710095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e22832800 con 0x559e227ab060 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb477fff640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab980 0x559e227bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:13.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb47cd11640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 0x559e227bcf70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb46c000c00 tx=0x7fb46c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f02493d024610a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb457fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb457fff640 1 -- 172.21.15.143:0/1196710095 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c01c040 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb457fff640 1 -- 172.21.15.143:0/1196710095 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb46c024c50 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.712+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb464048020 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb457fff640 1 -- 172.21.15.143:0/1196710095 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb46c038050 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 10 client.16620 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e22833b80 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb457fff640 1 -- 172.21.15.143:0/1196710095 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb464048020 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb457fff640 1 -- 172.21.15.143:0/1196710095 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb46c029070 con 0x559e227ab060 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 10 client.16620 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 10 client.16620 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 20 client.16620 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e22834330 0x559e22836750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 4 client.16620 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8ac87a67", "release": true, "await": true}] 2024-09-10T10:50:13.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8ac87a67", "release": true, "await": true}) -- 0x559e227180a0 con 0x559e22834330 2024-09-10T10:50:13.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.713+0000 7fb477fff640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e22834330 0x559e22836750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.714+0000 7fb457fff640 10 client.16620 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:13.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:13.714+0000 7fb477fff640 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e22834330 0x559e22836750 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e21a4262cb9e9a9 server_cookie=fb57c0260eb4d0ef in_seq=0 out_seq=0 2024-09-10T10:50:13.765 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 -- 172.21.15.143:0/571698992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 msgr2=0x7fbe48100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 --2- 172.21.15.143:0/571698992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe48100860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009900 tx=0x7fbe4402a410 comp rx=0 tx=0).stop 2024-09-10T10:50:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 -- 172.21.15.143:0/571698992 shutdown_connections 2024-09-10T10:50:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 --2- 172.21.15.143:0/571698992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe481055b0 0x7fbe4810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 --2- 172.21.15.143:0/571698992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe48100e30 0x7fbe48104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 --2- 172.21.15.143:0/571698992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe48100860 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 -- 172.21.15.143:0/571698992 >> 172.21.15.143:0/571698992 conn(0x7fbe48077210 msgr2=0x7fbe480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 -- 172.21.15.143:0/571698992 shutdown_connections 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 -- 172.21.15.143:0/571698992 wait complete. 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.244+0000 7fbe500fc640 1 Processor -- start 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 -- start start 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe481982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe48100e30 0x7fbe48198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe481055b0 0x7fbe4819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810fc40 con 0x7fbe48100480 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810fac0 con 0x7fbe481055b0 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe500fc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810fdc0 con 0x7fbe48100e30 2024-09-10T10:50:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe481982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe481982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55800/0 (socket says 172.21.15.143:55800) 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 -- 172.21.15.143:0/3157361914 learned_addr learned my addr 172.21.15.143:0/3157361914 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4d697640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe48100e30 0x7fbe48198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4e699640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe481055b0 0x7fbe4819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 -- 172.21.15.143:0/3157361914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe48100e30 msgr2=0x7fbe48198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe48100e30 0x7fbe48198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 -- 172.21.15.143:0/3157361914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe481055b0 msgr2=0x7fbe4819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe481055b0 0x7fbe4819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 -- 172.21.15.143:0/3157361914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4819d270 con 0x7fbe48100480 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4e699640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe481055b0 0x7fbe4819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe4de98640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe481982c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbe44005940 tx=0x7fbe440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.245+0000 7fbe3effd640 1 -- 172.21.15.143:0/3157361914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4403f030 con 0x7fbe48100480 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.246+0000 7fbe3effd640 1 -- 172.21.15.143:0/3157361914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe44003cb0 con 0x7fbe48100480 2024-09-10T10:50:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.246+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4819d500 con 0x7fbe48100480 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.246+0000 7fbe3effd640 1 -- 172.21.15.143:0/3157361914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44035070 con 0x7fbe48100480 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.246+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481a4e40 con 0x7fbe48100480 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.248+0000 7fbe3effd640 1 -- 172.21.15.143:0/3157361914 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbe44042030 con 0x7fbe48100480 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.248+0000 7fbe524d5740 1 Processor -- start 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.248+0000 7fbe524d5740 1 -- start start 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.248+0000 7fbe3effd640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbe340a2b80 0x7fbe340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc6363090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 0x555bc6363a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc636e370 0x555bc6370760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe524d5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555bc62c92f0 con 0x555bc6364b40 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe524d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555bc6270cf0 con 0x555bc636e370 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe524d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555bc62c52b0 con 0x555bc63635d0 2024-09-10T10:50:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4e699640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbe340a2b80 0x7fbe340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4de98640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc6363090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 0x555bc6363a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 0x555bc6363a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:50314/0 (socket says 172.21.15.143:50314) 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 -- 172.21.15.143:0/3776543557 learned_addr learned my addr 172.21.15.143:0/3776543557 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4de98640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc6363090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55802/0 (socket says 172.21.15.143:55802) 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe3effd640 1 -- 172.21.15.143:0/3157361914 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe440efbe0 con 0x7fbe48100480 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4e699640 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc636e370 0x555bc6370760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 -- 172.21.15.143:0/3776543557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc636e370 msgr2=0x555bc6370760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc636e370 0x555bc6370760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 -- 172.21.15.143:0/3776543557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 msgr2=0x555bc6363090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc6363090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 -- 172.21.15.143:0/3776543557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bc62553a0 con 0x555bc63635d0 2024-09-10T10:50:14.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4e699640 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc636e370 0x555bc6370760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4e699640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbe340a2b80 0x7fbe340a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000a5e0 tx=0x7fbe40011b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe4d697640 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 0x555bc6363a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800aca0 tx=0x7fbe3800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77160ee235fff75f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.249+0000 7fbe33fff640 1 -- 172.21.15.143:0/3776543557 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801d070 con 0x555bc63635d0 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe33fff640 1 -- 172.21.15.143:0/3776543557 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe38018af0 con 0x555bc63635d0 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe33fff640 1 -- 172.21.15.143:0/3776543557 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38010b10 con 0x555bc63635d0 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 -- 172.21.15.143:0/3776543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 msgr2=0x555bc6363a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 0x555bc6363a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800aca0 tx=0x7fbe3800f790 comp rx=0 tx=0).stop 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 -- 172.21.15.143:0/3776543557 shutdown_connections 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc636e370 0x555bc6370760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc63635d0 0x555bc6363a10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 --2- 172.21.15.143:0/3776543557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc6363090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 -- 172.21.15.143:0/3776543557 >> 172.21.15.143:0/3776543557 conn(0x555bc632c310 msgr2=0x555bc635fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 -- 172.21.15.143:0/3776543557 shutdown_connections 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.250+0000 7fbe524d5740 1 -- 172.21.15.143:0/3776543557 wait complete. 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 -- start start 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 Processor -- start 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc636dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc636b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555bc6371070 con 0x555bc6364b40 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555bc6284060 con 0x555bc63635d0 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555bc63711f0 con 0x555bc636e370 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe524d5740 10 client.0 _ll_register_callbacks cb 0x555bc5fd78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe4d697640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe4d697640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55814/0 (socket says 172.21.15.143:55814) 2024-09-10T10:50:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe4d697640 1 -- 172.21.15.143:0/937183714 learned_addr learned my addr 172.21.15.143:0/937183714 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe4e699640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc636b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.251+0000 7fbe4de98640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc636dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4d697640 1 -- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 msgr2=0x555bc636b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4d697640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc636b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4d697640 1 -- 172.21.15.143:0/937183714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 msgr2=0x555bc636dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4d697640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc636dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4d697640 1 -- 172.21.15.143:0/937183714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bc603a3c0 con 0x555bc6364b40 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4e699640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc636b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe4d697640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636b000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbe3801b040 tx=0x7fbe38018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15bd8b80476346d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe317fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe317fa640 1 -- 172.21.15.143:0/937183714 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801d050 con 0x555bc6364b40 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe317fa640 1 -- 172.21.15.143:0/937183714 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe38010850 con 0x555bc6364b40 2024-09-10T10:50:14.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe38018af0 con 0x555bc6364b40 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe317fa640 1 -- 172.21.15.143:0/937183714 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbe38032050 con 0x555bc6364b40 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe524d5740 10 client.16632 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555bc637caf0 con 0x555bc6364b40 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe317fa640 1 -- 172.21.15.143:0/937183714 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe38018af0 con 0x555bc6364b40 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe317fa640 1 -- 172.21.15.143:0/937183714 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fbe380046c0 con 0x555bc6364b40 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe524d5740 10 client.16632 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe524d5740 10 client.16632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.252+0000 7fbe524d5740 20 client.16632 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.253+0000 7fbe524d5740 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc637de40 0x555bc6380230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.253+0000 7fbe524d5740 4 client.16632 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.253+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bc62d6860 con 0x555bc637de40 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.253+0000 7fbe4de98640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc637de40 0x555bc6380230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.253+0000 7fbe317fa640 10 client.16632 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.253+0000 7fbe4de98640 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc637de40 0x555bc6380230 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da7de2f8138bf44b server_cookie=8eda66790d93e80d in_seq=0 out_seq=0 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe317fa640 1 -- 172.21.15.143:0/937183714 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bc62d6860 con 0x555bc637de40 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe317fa640 10 client.16632 handle_command_reply: tid=0 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 client.16632 shutdown 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 msgr2=0x555bc636b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636b000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbe3801b040 tx=0x7fbe38018440 comp rx=0 tx=0).stop 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 shutdown_connections 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc637de40 0x555bc6380230 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc636b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636b000 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe317fa640 0 client.16632 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:50:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 --2- 172.21.15.143:0/937183714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc636dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 >> 172.21.15.143:0/937183714 conn(0x555bc632c310 msgr2=0x555bc6365960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 shutdown_connections 2024-09-10T10:50:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 1 -- 172.21.15.143:0/937183714 wait complete. 2024-09-10T10:50:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.254+0000 7fbe524d5740 20 client.16632 trim_cache size 0 max 16384 2024-09-10T10:50:14.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 Processor -- start 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 -- start start 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc637a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6385890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481aeaa0 con 0x555bc6364b40 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b05e0 con 0x555bc63635d0 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe524d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481af170 con 0x555bc636e370 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55816/0 (socket says 172.21.15.143:55816) 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 -- 172.21.15.143:0/896150415 learned_addr learned my addr 172.21.15.143:0/896150415 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4e699640 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6385890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4de98640 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc637a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 -- 172.21.15.143:0/896150415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 msgr2=0x555bc6385890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6385890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 -- 172.21.15.143:0/896150415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 msgr2=0x555bc637a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc637a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 -- 172.21.15.143:0/896150415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bc6366120 con 0x555bc6364b40 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4de98640 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc637a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.268+0000 7fbe4d697640 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636cf70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800b010 tx=0x7fbe38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74979333c425e8df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe337fe640 1 -- 172.21.15.143:0/896150415 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3803a070 con 0x555bc6364b40 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe337fe640 1 -- 172.21.15.143:0/896150415 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe38029070 con 0x555bc6364b40 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe337fe640 1 -- 172.21.15.143:0/896150415 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801d040 con 0x555bc6364b40 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 -- 172.21.15.143:0/896150415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 msgr2=0x555bc636cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636cf70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800b010 tx=0x7fbe38005880 comp rx=0 tx=0).stop 2024-09-10T10:50:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 -- 172.21.15.143:0/896150415 shutdown_connections 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6385890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc6364b40 0x555bc636cf70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 --2- 172.21.15.143:0/896150415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc63635d0 0x555bc637a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 -- 172.21.15.143:0/896150415 >> 172.21.15.143:0/896150415 conn(0x555bc632c310 msgr2=0x555bc636c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 -- 172.21.15.143:0/896150415 shutdown_connections 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.269+0000 7fbe524d5740 1 -- 172.21.15.143:0/896150415 wait complete. 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 -- start start 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 Processor -- start 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 0x555bc63767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc6364b40 0x555bc6376ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6371bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481aff10 con 0x555bc63635d0 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481af840 con 0x555bc6364b40 2024-09-10T10:50:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481aca80 con 0x555bc636e370 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe524d5740 10 client.0 _ll_register_callbacks cb 0x7fbe3801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe4de98640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 0x555bc63767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe4e699640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6371bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe4e699640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6371bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:50342/0 (socket says 172.21.15.143:50342) 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe4de98640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 0x555bc63767a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55822/0 (socket says 172.21.15.143:55822) 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.270+0000 7fbe4e699640 1 -- 172.21.15.143:0/2659483598 learned_addr learned my addr 172.21.15.143:0/2659483598 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4d697640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc6364b40 0x555bc6376ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4e699640 1 -- 172.21.15.143:0/2659483598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc6364b40 msgr2=0x555bc6376ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4e699640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc6364b40 0x555bc6376ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4e699640 1 -- 172.21.15.143:0/2659483598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 msgr2=0x555bc63767a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4e699640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 0x555bc63767a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4e699640 1 -- 172.21.15.143:0/2659483598 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bc6372400 con 0x555bc636e370 2024-09-10T10:50:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4de98640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 0x555bc63767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe4e699640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6371bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbe40021690 tx=0x7fbe40004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ab709c062fb342 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe317fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe317fa640 1 -- 172.21.15.143:0/2659483598 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000e9e0 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe38029070 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe317fa640 1 -- 172.21.15.143:0/2659483598 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe4000eb80 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe317fa640 1 -- 172.21.15.143:0/2659483598 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbe4000ed80 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe524d5740 10 client.26041 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.271+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555bc63e8400 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe317fa640 1 -- 172.21.15.143:0/2659483598 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe38029070 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe317fa640 1 -- 172.21.15.143:0/2659483598 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fbe40028070 con 0x555bc636e370 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 10 client.26041 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 10 client.26041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 20 client.26041 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc63e8b50 0x555bc63eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 4 client.26041 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bc62d0d70 con 0x555bc63e8b50 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe4de98640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc63e8b50 0x555bc63eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe317fa640 10 client.26041 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe4de98640 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc63e8b50 0x555bc63eafb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=136a04a95d086b10 server_cookie=661b0d915bc4cfe4 in_seq=0 out_seq=0 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe317fa640 1 -- 172.21.15.143:0/2659483598 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bc62d0d70 con 0x555bc63e8b50 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe317fa640 10 client.26041 handle_command_reply: tid=0 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.272+0000 7fbe524d5740 1 client.26041 shutdown 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 msgr2=0x555bc6371bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6371bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbe40021690 tx=0x7fbe40004ab0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 shutdown_connections 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x555bc63e8b50 0x555bc63eafb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bc636e370 0x555bc6371bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bc6364b40 0x555bc6376ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe317fa640 0 client.26041 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 --2- 172.21.15.143:0/2659483598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555bc63635d0 0x555bc63767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 >> 172.21.15.143:0/2659483598 conn(0x555bc632c310 msgr2=0x555bc636d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 shutdown_connections 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 1 -- 172.21.15.143:0/2659483598 wait complete. 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe524d5740 20 client.26041 trim_cache size 0 max 16384 2024-09-10T10:50:14.286 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbe340a2b80 msgr2=0x7fbe340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe500fc640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbe340a2b80 0x7fbe340a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000a5e0 tx=0x7fbe40011b20 comp rx=0 tx=0).stop 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 msgr2=0x7fbe481982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.273+0000 7fbe500fc640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe481982c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbe44005940 tx=0x7fbe440042f0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 shutdown_connections 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbe340a2b80 0x7fbe340a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe481055b0 0x7fbe4819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe48100e30 0x7fbe48198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 --2- 172.21.15.143:0/3157361914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe48100480 0x7fbe481982c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 >> 172.21.15.143:0/3157361914 conn(0x7fbe48077210 msgr2=0x7fbe48103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 shutdown_connections 2024-09-10T10:50:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.274+0000 7fbe500fc640 1 -- 172.21.15.143:0/3157361914 wait complete. 2024-09-10T10:50:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb457fff640 1 -- 172.21.15.143:0/1196710095 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559e227180a0 con 0x559e22834330 2024-09-10T10:50:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb457fff640 10 client.16620 handle_command_reply: tid=0 2024-09-10T10:50:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 client.16620 shutdown 2024-09-10T10:50:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 msgr2=0x559e227bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 0x559e227bcf70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb46c000c00 tx=0x7fb46c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 shutdown_connections 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x559e22834330 0x559e22836750 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559e227b6240 0x559e227bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb457fff640 0 client.16620 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e227ab980 0x559e227bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 --2- 172.21.15.143:0/1196710095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559e227ab060 0x559e227bcf70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 >> 172.21.15.143:0/1196710095 conn(0x559e227a75f0 msgr2=0x559e227aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 shutdown_connections 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 1 -- 172.21.15.143:0/1196710095 wait complete. 2024-09-10T10:50:14.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.319+0000 7fb48134e740 20 client.16620 trim_cache size 0 max 16384 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 33, 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 16, 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "8ac87a67": { 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 16, 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:50:14.322 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 142.8, 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:14.323 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb4680a2b80 msgr2=0x7fb4680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb4680a2b80 0x7fb4680a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb46c006fd0 tx=0x7fb46c008040 comp rx=0 tx=0).stop 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 msgr2=0x7fb47819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 0x7fb47819bd00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb47000b970 tx=0x7fb47000be30 comp rx=0 tx=0).stop 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 shutdown_connections 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb4680a2b80 0x7fb4680a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb478106a80 0x7fb47819bd00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb478101420 0x7fb478197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 --2- 172.21.15.143:0/986933631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb478100a80 0x7fb478197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 >> 172.21.15.143:0/986933631 conn(0x7fb4780fc7a0 msgr2=0x7fb4780fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 shutdown_connections 2024-09-10T10:50:14.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:14.320+0000 7fb47ef75640 1 -- 172.21.15.143:0/986933631 wait complete. 2024-09-10T10:50:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:14 smithi178 ceph-mon[34813]: pgmap v506: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-09-10T10:50:14.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:14 smithi144 ceph-mon[34230]: pgmap v506: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-09-10T10:50:15.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:14 smithi143 ceph-mon[31057]: pgmap v506: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-09-10T10:50:15.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:50:16.011 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8ac87a67', total seconds elapsed: 35.1 2024-09-10T10:50:16.085 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 -- 172.21.15.143:0/3439482678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5830101df0 msgr2=0x7f58301021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 --2- 172.21.15.143:0/3439482678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5830101df0 0x7f58301021d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f582c013150 tx=0x7f582c032780 comp rx=0 tx=0).stop 2024-09-10T10:50:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 -- 172.21.15.143:0/3439482678 shutdown_connections 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 --2- 172.21.15.143:0/3439482678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5830106f20 0x7f583010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 --2- 172.21.15.143:0/3439482678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58301027a0 0x7f58301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 --2- 172.21.15.143:0/3439482678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5830101df0 0x7f58301021d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 -- 172.21.15.143:0/3439482678 >> 172.21.15.143:0/3439482678 conn(0x7f58300fd8f0 msgr2=0x7f58300ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 -- 172.21.15.143:0/3439482678 shutdown_connections 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.624+0000 7f5837deb640 1 -- 172.21.15.143:0/3439482678 wait complete. 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 Processor -- start 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 -- start start 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830101df0 0x7f58301962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 0x7f58301967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5830106f20 0x7f583019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f583010fce0 con 0x7f5830101df0 2024-09-10T10:50:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f583010fb60 con 0x7f58301027a0 2024-09-10T10:50:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5837deb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f583010fe60 con 0x7f5830106f20 2024-09-10T10:50:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5835b87640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830101df0 0x7f58301962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5835386640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 0x7f58301967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5836388640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5830106f20 0x7f583019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5835b87640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830101df0 0x7f58301962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54756/0 (socket says 172.21.15.143:54756) 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5835386640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 0x7f58301967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46180/0 (socket says 172.21.15.143:46180) 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.625+0000 7f5835b87640 1 -- 172.21.15.143:0/1777278432 learned_addr learned my addr 172.21.15.143:0/1777278432 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.626+0000 7f5835386640 1 -- 172.21.15.143:0/1777278432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5830106f20 msgr2=0x7f583019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.626+0000 7f5835386640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5830106f20 0x7f583019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.626+0000 7f5835386640 1 -- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830101df0 msgr2=0x7f58301962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.626+0000 7f5835386640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830101df0 0x7f58301962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.626+0000 7f5835386640 1 -- 172.21.15.143:0/1777278432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583019b250 con 0x7f58301027a0 2024-09-10T10:50:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.626+0000 7f5835386640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 0x7f58301967e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f582000b3f0 tx=0x7f582000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.629+0000 7f5826ffd640 1 -- 172.21.15.143:0/1777278432 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820015020 con 0x7f58301027a0 2024-09-10T10:50:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.629+0000 7f5826ffd640 1 -- 172.21.15.143:0/1777278432 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5820011ce0 con 0x7f58301027a0 2024-09-10T10:50:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.629+0000 7f5826ffd640 1 -- 172.21.15.143:0/1777278432 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820010de0 con 0x7f58301027a0 2024-09-10T10:50:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.629+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583019b540 con 0x7f58301027a0 2024-09-10T10:50:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.629+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301a2e20 con 0x7f58301027a0 2024-09-10T10:50:16.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.632+0000 7f5826ffd640 1 -- 172.21.15.143:0/1777278432 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5820018020 con 0x7f58301027a0 2024-09-10T10:50:16.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 Processor -- start 2024-09-10T10:50:16.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f5826ffd640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f581c0a2c00 0x7f581c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 -- start start 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51428e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f51428e5d0 0x55f51428ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f514299370 0x55f51429b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f5141f42f0 con 0x55f51428e5d0 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f51419bcf0 con 0x55f51428fb40 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f583a1c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f5141f02b0 con 0x55f514299370 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.633+0000 7f5826ffd640 1 -- 172.21.15.143:0/1777278432 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f58200cead0 con 0x7f58301027a0 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51428e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51428e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46194/0 (socket says 172.21.15.143:46194) 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 -- 172.21.15.143:0/2803235557 learned_addr learned my addr 172.21.15.143:0/2803235557 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5836388640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f581c0a2c00 0x7f581c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 -- 172.21.15.143:0/2803235557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f514299370 msgr2=0x55f51429b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f514299370 0x55f51429b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 -- 172.21.15.143:0/2803235557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f51428e5d0 msgr2=0x55f51428ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f51428e5d0 0x55f51428ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 -- 172.21.15.143:0/2803235557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5141803a0 con 0x55f51428fb40 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.634+0000 7f5835b87640 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51428e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f582c005e90 tx=0x7f582c00fbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c11c669e5abbda3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.635+0000 7f581bfff640 1 -- 172.21.15.143:0/2803235557 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c049030 con 0x55f51428fb40 2024-09-10T10:50:16.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.635+0000 7f581bfff640 1 -- 172.21.15.143:0/2803235557 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c00fdd0 con 0x55f51428fb40 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.635+0000 7f581bfff640 1 -- 172.21.15.143:0/2803235557 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0046c0 con 0x55f51428fb40 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.636+0000 7f583a1c4740 1 -- 172.21.15.143:0/2803235557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 msgr2=0x55f51428e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.636+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51428e090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f582c005e90 tx=0x7f582c00fbe0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f5836388640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f581c0a2c00 0x7f581c0a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55f51428f070 tx=0x7f5828009990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 -- 172.21.15.143:0/2803235557 shutdown_connections 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f514299370 0x55f51429b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f51428e5d0 0x55f51428ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2803235557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51428e090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 -- 172.21.15.143:0/2803235557 >> 172.21.15.143:0/2803235557 conn(0x55f514257310 msgr2=0x55f51428ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 -- 172.21.15.143:0/2803235557 shutdown_connections 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.637+0000 7f583a1c4740 1 -- 172.21.15.143:0/2803235557 wait complete. 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.638+0000 7f583a1c4740 1 -- start start 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.638+0000 7f583a1c4740 1 Processor -- start 2024-09-10T10:50:16.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f514298da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514296050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f5142967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f583a1c4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f51429c010 con 0x55f514299370 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f583a1c4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f5141af060 con 0x55f51428fb40 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f583a1c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f51429c190 con 0x55f51428e5d0 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f5835b87640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f514298da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f5835b87640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f514298da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46314/0 (socket says 172.21.15.143:46314) 2024-09-10T10:50:16.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.639+0000 7f5835b87640 1 -- 172.21.15.143:0/4131543562 learned_addr learned my addr 172.21.15.143:0/4131543562 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835386640 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514296050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835b87640 1 -- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 msgr2=0x55f514296050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835b87640 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514296050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835b87640 1 -- 172.21.15.143:0/4131543562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 msgr2=0x55f5142967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835b87640 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f5142967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835b87640 1 -- 172.21.15.143:0/4131543562 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f513f653c0 con 0x55f51428e5d0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f583a1c4740 10 client.0 _ll_register_callbacks cb 0x55f513f028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f5835b87640 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f514298da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f582c000c00 tx=0x7f582c011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb131902b96bd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c00fdd0 con 0x55f51428e5d0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.640+0000 7f58197fa640 1 -- 172.21.15.143:0/4131543562 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c052020 con 0x55f51428e5d0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f58197fa640 1 -- 172.21.15.143:0/4131543562 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f582c0048c0 con 0x55f51428e5d0 2024-09-10T10:50:16.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f58197fa640 1 -- 172.21.15.143:0/4131543562 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f582c03f040 con 0x55f51428e5d0 2024-09-10T10:50:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 10 client.26047 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:16.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55f5142a7720 con 0x55f51428e5d0 2024-09-10T10:50:16.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f58197fa640 1 -- 172.21.15.143:0/4131543562 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f582c004db0 con 0x55f51428e5d0 2024-09-10T10:50:16.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 10 client.26047 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:16.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 10 client.26047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 20 client.26047 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f5142a8a70 0x55f5142aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 4 client.26047 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.641+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f514201860 con 0x55f5142a8a70 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.642+0000 7f5835386640 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f5142a8a70 0x55f5142aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.642+0000 7f5835386640 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f5142a8a70 0x55f5142aae60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=195ad4eccf372f2b server_cookie=847559d3d6e5850c in_seq=0 out_seq=0 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.642+0000 7f58197fa640 10 client.26047 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f58197fa640 1 -- 172.21.15.143:0/4131543562 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f514201860 con 0x55f5142a8a70 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f58197fa640 10 client.26047 handle_command_reply: tid=0 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 client.26047 shutdown 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 msgr2=0x55f514298da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f514298da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f582c000c00 tx=0x7f582c011a70 comp rx=0 tx=0).stop 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 shutdown_connections 2024-09-10T10:50:16.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f5142967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f5142a8a70 0x55f5142aae60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514296050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 --2- 172.21.15.143:0/4131543562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f514298da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 >> 172.21.15.143:0/4131543562 conn(0x55f514257310 msgr2=0x55f514290930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f58197fa640 0 client.26047 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 shutdown_connections 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 1 -- 172.21.15.143:0/4131543562 wait complete. 2024-09-10T10:50:16.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.643+0000 7f583a1c4740 20 client.26047 trim_cache size 0 max 16384 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.662+0000 7f583a1c4740 1 Processor -- start 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.662+0000 7f583a1c4740 1 -- start start 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f51428ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514289c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f514291120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f583a1c4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b5180 con 0x55f514299370 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f583a1c4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b6cc0 con 0x55f51428fb40 2024-09-10T10:50:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f583a1c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b5850 con 0x55f51428e5d0 2024-09-10T10:50:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5836388640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f514291120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5836388640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f514291120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54774/0 (socket says 172.21.15.143:54774) 2024-09-10T10:50:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514289c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5836388640 1 -- 172.21.15.143:0/511401937 learned_addr learned my addr 172.21.15.143:0/511401937 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:16.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835b87640 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f51428ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 -- 172.21.15.143:0/511401937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 msgr2=0x55f51428ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f51428ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 -- 172.21.15.143:0/511401937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 msgr2=0x55f514291120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f514291120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 -- 172.21.15.143:0/511401937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5142945a0 con 0x55f51428fb40 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5836388640 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f514291120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.663+0000 7f5835386640 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514289c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55f51428bac0 tx=0x7f58200bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4582aeddf8044c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.664+0000 7f581b7fe640 1 -- 172.21.15.143:0/511401937 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200a3c40 con 0x55f51428fb40 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.664+0000 7f581b7fe640 1 -- 172.21.15.143:0/511401937 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58200cc380 con 0x55f51428fb40 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.664+0000 7f581b7fe640 1 -- 172.21.15.143:0/511401937 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820095b40 con 0x55f51428fb40 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 -- 172.21.15.143:0/511401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 msgr2=0x55f514289c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514289c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55f51428bac0 tx=0x7f58200bb040 comp rx=0 tx=0).stop 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 -- 172.21.15.143:0/511401937 shutdown_connections 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f514291120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f514289c10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 --2- 172.21.15.143:0/511401937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f51428ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 -- 172.21.15.143:0/511401937 >> 172.21.15.143:0/511401937 conn(0x55f514257310 msgr2=0x55f514289500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 -- 172.21.15.143:0/511401937 shutdown_connections 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 -- 172.21.15.143:0/511401937 wait complete. 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.665+0000 7f583a1c4740 1 -- start start 2024-09-10T10:50:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 Processor -- start 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f5142a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51429ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f51429d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b65f0 con 0x55f514299370 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58301b5f20 con 0x55f51428fb40 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5830072400 con 0x55f51428e5d0 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f5835b87640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f5142a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f5835b87640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f5142a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46328/0 (socket says 172.21.15.143:46328) 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f5835b87640 1 -- 172.21.15.143:0/2386354958 learned_addr learned my addr 172.21.15.143:0/2386354958 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f583a1c4740 10 client.0 _ll_register_callbacks cb 0x7f582c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f5836388640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f51429d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.666+0000 7f5835386640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51429ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f5835b87640 1 -- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 msgr2=0x55f51429ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f5835b87640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51429ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f5835b87640 1 -- 172.21.15.143:0/2386354958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 msgr2=0x55f51429d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f5835b87640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f51429d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f5835b87640 1 -- 172.21.15.143:0/2386354958 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51429db80 con 0x55f51428e5d0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f5835b87640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f5142a1ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f582c005730 tx=0x7f582c0056c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26512bcc1ce0b792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f58197fa640 1 -- 172.21.15.143:0/2386354958 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c065060 con 0x55f51428e5d0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f58197fa640 1 -- 172.21.15.143:0/2386354958 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f582c0046c0 con 0x55f51428e5d0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58200cc380 con 0x55f51428e5d0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f58197fa640 1 -- 172.21.15.143:0/2386354958 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f582c06c050 con 0x55f51428e5d0 2024-09-10T10:50:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f583a1c4740 10 client.26053 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.667+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5143134b0 con 0x55f51428e5d0 2024-09-10T10:50:16.672 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:16 smithi143 ceph-mon[31057]: pgmap v507: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 846 KiB/s wr, 42 op/s 2024-09-10T10:50:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f58197fa640 1 -- 172.21.15.143:0/2386354958 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c00fdd0 con 0x55f51428e5d0 2024-09-10T10:50:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f58197fa640 1 -- 172.21.15.143:0/2386354958 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f582c060980 con 0x55f51428e5d0 2024-09-10T10:50:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f583a1c4740 10 client.26053 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:16.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f583a1c4740 10 client.26053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f583a1c4740 20 client.26053 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f514313b10 0x55f514315f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f583a1c4740 4 client.26053 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f58200cc380 con 0x55f514313b10 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.668+0000 7f5835386640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f514313b10 0x55f514315f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.669+0000 7f58197fa640 10 client.26053 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.669+0000 7f5835386640 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f514313b10 0x55f514315f00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21698d032babf170 server_cookie=9c9cb140f401ebfa in_seq=0 out_seq=0 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.669+0000 7f58197fa640 1 -- 172.21.15.143:0/2386354958 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f58200cc380 con 0x55f514313b10 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.669+0000 7f58197fa640 10 client.26053 handle_command_reply: tid=0 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.669+0000 7f583a1c4740 1 client.26053 shutdown 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 msgr2=0x55f5142a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f5142a1ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f582c005730 tx=0x7f582c0056c0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 shutdown_connections 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f514299370 0x55f51429d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55f514313b10 0x55f514315f00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f51428fb40 0x55f51429ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 --2- 172.21.15.143:0/2386354958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f51428e5d0 0x55f5142a1ae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 >> 172.21.15.143:0/2386354958 conn(0x55f514257310 msgr2=0x55f514289d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f58197fa640 0 client.26053 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 shutdown_connections 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 1 -- 172.21.15.143:0/2386354958 wait complete. 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.670+0000 7f583a1c4740 20 client.26053 trim_cache size 0 max 16384 2024-09-10T10:50:16.675 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f581c0a2c00 msgr2=0x7f581c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f581c0a2c00 0x7f581c0a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55f51428f070 tx=0x7f5828009990 comp rx=0 tx=0).stop 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 msgr2=0x7f58301967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 0x7f58301967e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f582000b3f0 tx=0x7f582000b8b0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 shutdown_connections 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f581c0a2c00 0x7f581c0a50c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5830106f20 0x7f583019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58301027a0 0x7f58301967e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 --2- 172.21.15.143:0/1777278432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830101df0 0x7f58301962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 >> 172.21.15.143:0/1777278432 conn(0x7f58300fd8f0 msgr2=0x7f58300fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 shutdown_connections 2024-09-10T10:50:16.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:16.671+0000 7f5837deb640 1 -- 172.21.15.143:0/1777278432 wait complete. 2024-09-10T10:50:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:16 smithi178 ceph-mon[34813]: pgmap v507: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 846 KiB/s wr, 42 op/s 2024-09-10T10:50:16.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:16 smithi144 ceph-mon[34230]: pgmap v507: 129 pgs: 129 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 846 KiB/s wr, 42 op/s 2024-09-10T10:50:17.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:50:17.727 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:17.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:17 smithi144 ceph-mon[34230]: pgmap v508: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 670 KiB/s wr, 46 op/s 2024-09-10T10:50:18.008 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:17 smithi143 ceph-mon[31057]: pgmap v508: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 670 KiB/s wr, 46 op/s 2024-09-10T10:50:18.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:17 smithi178 ceph-mon[34813]: pgmap v508: 129 pgs: 129 active+clean; 361 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 670 KiB/s wr, 46 op/s 2024-09-10T10:50:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.360+0000 7f4767fff640 1 -- 172.21.15.143:0/1473545549 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c02fc60 con 0x7f4768102570 2024-09-10T10:50:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 -- 172.21.15.143:0/1473545549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768102570 msgr2=0x7f47681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 --2- 172.21.15.143:0/1473545549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768102570 0x7f47681029b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f475c009cd0 tx=0x7f475c02a980 comp rx=0 tx=0).stop 2024-09-10T10:50:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 -- 172.21.15.143:0/1473545549 shutdown_connections 2024-09-10T10:50:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 --2- 172.21.15.143:0/1473545549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 0x7f476810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 --2- 172.21.15.143:0/1473545549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768102570 0x7f47681029b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 --2- 172.21.15.143:0/1473545549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768100e20 0x7f4768101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 -- 172.21.15.143:0/1473545549 >> 172.21.15.143:0/1473545549 conn(0x7f47680fc7d0 msgr2=0x7f47680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 -- 172.21.15.143:0/1473545549 shutdown_connections 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.361+0000 7f476f7c2640 1 -- 172.21.15.143:0/1473545549 wait complete. 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 Processor -- start 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 -- start start 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768100e20 0x7f476810d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 0x7f4768109910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 0x7f4768109e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4768110e80 con 0x7f4768102ef0 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4768110d00 con 0x7f4768100e20 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.362+0000 7f476f7c2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4768111000 con 0x7f4768102570 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 0x7f4768109910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 0x7f4768109910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46348/0 (socket says 172.21.15.143:46348) 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 -- 172.21.15.143:0/3317865594 learned_addr learned my addr 172.21.15.143:0/3317865594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476dd5f640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 0x7f4768109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 -- 172.21.15.143:0/3317865594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768100e20 msgr2=0x7f476810d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476d55e640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768100e20 0x7f476810d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768100e20 0x7f476810d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 -- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 msgr2=0x7f4768109e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 0x7f4768109e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 -- 172.21.15.143:0/3317865594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476810a710 con 0x7f4768102570 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476dd5f640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 0x7f4768109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476d55e640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768100e20 0x7f476810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.363+0000 7f476cd5d640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 0x7f4768109910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f475c009ca0 tx=0x7f475c003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.364+0000 7f47667fc640 1 -- 172.21.15.143:0/3317865594 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c034040 con 0x7f4768102570 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.364+0000 7f47667fc640 1 -- 172.21.15.143:0/3317865594 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475c031e80 con 0x7f4768102570 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.364+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47681aafe0 con 0x7f4768102570 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.364+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47681ab4e0 con 0x7f4768102570 2024-09-10T10:50:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.364+0000 7f47667fc640 1 -- 172.21.15.143:0/3317865594 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004720 con 0x7f4768102570 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.367+0000 7f47667fc640 1 -- 172.21.15.143:0/3317865594 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f475c037020 con 0x7f4768102570 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 Processor -- start 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 -- start start 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 0x55e0b778b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778b5d0 0x55e0b778ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b7798760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e0b76f12f0 con 0x55e0b7796370 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0b7698cf0 con 0x55e0b778cb40 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f4771b9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0b76ed2b0 con 0x55e0b778b5d0 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f476dd5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 0x55e0b778b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f476dd5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 0x55e0b778b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46246/0 (socket says 172.21.15.143:46246) 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f476dd5f640 1 -- 172.21.15.143:0/1859812886 learned_addr learned my addr 172.21.15.143:0/1859812886 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.368+0000 7f476d55e640 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778b5d0 0x55e0b778ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476cd5d640 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b7798760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476dd5f640 1 -- 172.21.15.143:0/1859812886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778b5d0 msgr2=0x55e0b778ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476dd5f640 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778b5d0 0x55e0b778ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476dd5f640 1 -- 172.21.15.143:0/1859812886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 msgr2=0x55e0b7798760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476dd5f640 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b7798760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476dd5f640 1 -- 172.21.15.143:0/1859812886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b767d3a0 con 0x55e0b778cb40 2024-09-10T10:50:18.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476cd5d640 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b7798760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f476dd5f640 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 0x55e0b778b090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f475800bca0 tx=0x7f475800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ebcf5c314637cca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.369+0000 7f474f7fe640 1 -- 172.21.15.143:0/1859812886 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758015070 con 0x55e0b778cb40 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.370+0000 7f47667fc640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f47500a2c00 0x7f47500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.370+0000 7f474f7fe640 1 -- 172.21.15.143:0/1859812886 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47580035f0 con 0x55e0b778cb40 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.370+0000 7f474f7fe640 1 -- 172.21.15.143:0/1859812886 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47580116a0 con 0x55e0b778cb40 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.370+0000 7f47667fc640 1 -- 172.21.15.143:0/3317865594 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f475c0f0790 con 0x7f4768102570 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.370+0000 7f476d55e640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f47500a2c00 0x7f47500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 -- 172.21.15.143:0/1859812886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 msgr2=0x55e0b778b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f476d55e640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f47500a2c00 0x7f47500a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4760005fd0 tx=0x7f47600094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 0x55e0b778b090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f475800bca0 tx=0x7f475800f730 comp rx=0 tx=0).stop 2024-09-10T10:50:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 -- 172.21.15.143:0/1859812886 shutdown_connections 2024-09-10T10:50:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b7798760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778b5d0 0x55e0b778ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 --2- 172.21.15.143:0/1859812886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778cb40 0x55e0b778b090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 -- 172.21.15.143:0/1859812886 >> 172.21.15.143:0/1859812886 conn(0x55e0b7754310 msgr2=0x55e0b7787c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 -- 172.21.15.143:0/1859812886 shutdown_connections 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.371+0000 7f4771b9b740 1 -- 172.21.15.143:0/1859812886 wait complete. 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.372+0000 7f4771b9b740 1 -- start start 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.372+0000 7f4771b9b740 1 Processor -- start 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.372+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7795cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.372+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b7792f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.372+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b77934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f4771b9b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e0b77990c0 con 0x55e0b7796370 2024-09-10T10:50:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f4771b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0b76ac060 con 0x55e0b778b5d0 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f4771b9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0b7799240 con 0x55e0b778cb40 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f4771b9b740 10 client.0 _ll_register_callbacks cb 0x55e0b73ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b7792f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b7792f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46370/0 (socket says 172.21.15.143:46370) 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 -- 172.21.15.143:0/2561731702 learned_addr learned my addr 172.21.15.143:0/2561731702 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476dd5f640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b77934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 -- 172.21.15.143:0/2561731702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 msgr2=0x55e0b7795cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476d55e640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7795cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7795cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 -- 172.21.15.143:0/2561731702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 msgr2=0x55e0b77934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b77934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 -- 172.21.15.143:0/2561731702 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b74623c0 con 0x55e0b778cb40 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476cd5d640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b7792f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f475c000f80 tx=0x7f475c0bc770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2424336ef8f56f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f476dd5f640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b77934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.373+0000 7f474cff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f474cff9640 1 -- 172.21.15.143:0/2561731702 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bcc30 con 0x55e0b778cb40 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47580035f0 con 0x55e0b778cb40 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f474cff9640 1 -- 172.21.15.143:0/2561731702 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f475c02fd10 con 0x55e0b778cb40 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f474cff9640 1 -- 172.21.15.143:0/2561731702 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f475c0c0fe0 con 0x55e0b778cb40 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f4771b9b740 10 client.26065 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e0b7799170 con 0x55e0b778cb40 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f474cff9640 1 -- 172.21.15.143:0/2561731702 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47580035f0 con 0x55e0b778cb40 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f474cff9640 1 -- 172.21.15.143:0/2561731702 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f475c02f070 con 0x55e0b778cb40 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f4771b9b740 10 client.26065 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f4771b9b740 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.374+0000 7f4771b9b740 20 client.26065 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.375+0000 7f4771b9b740 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b77a6010 0x55e0b77a8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.375+0000 7f4771b9b740 4 client.26065 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.375+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0b76fe860 con 0x55e0b77a6010 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.375+0000 7f476d55e640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b77a6010 0x55e0b77a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.375+0000 7f474cff9640 10 client.26065 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.375+0000 7f476d55e640 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b77a6010 0x55e0b77a8470 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0683d58358bcaa server_cookie=392a6c9bbbd7f708 in_seq=0 out_seq=0 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f474cff9640 1 -- 172.21.15.143:0/2561731702 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0b76fe860 con 0x55e0b77a6010 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f474cff9640 10 client.26065 handle_command_reply: tid=0 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 client.26065 shutdown 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 msgr2=0x55e0b7792f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b7792f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f475c000f80 tx=0x7f475c0bc770 comp rx=0 tx=0).stop 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 shutdown_connections 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b77934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b77a6010 0x55e0b77a8470 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.376+0000 7f4771b9b740 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b7792f80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.377+0000 7f4771b9b740 1 --2- 172.21.15.143:0/2561731702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7795cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.377+0000 7f474cff9640 0 client.26065 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.377+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 >> 172.21.15.143:0/2561731702 conn(0x55e0b7754310 msgr2=0x55e0b778d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.377+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 shutdown_connections 2024-09-10T10:50:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.377+0000 7f4771b9b740 1 -- 172.21.15.143:0/2561731702 wait complete. 2024-09-10T10:50:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.377+0000 7f4771b9b740 20 client.26065 trim_cache size 0 max 16384 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 Processor -- start 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 -- start start 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b77a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 0x55e0b7787d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b7796370 0x55e0b778e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f475400fa50 con 0x55e0b778cb40 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4754011590 con 0x55e0b778b5d0 2024-09-10T10:50:18.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.394+0000 7f4771b9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4754010120 con 0x55e0b7796370 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 0x55e0b7787d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 0x55e0b7787d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54832/0 (socket says 172.21.15.143:54832) 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 -- 172.21.15.143:0/47455673 learned_addr learned my addr 172.21.15.143:0/47455673 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476dd5f640 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b7796370 0x55e0b778e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476d55e640 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b77a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 -- 172.21.15.143:0/47455673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b7796370 msgr2=0x55e0b778e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b7796370 0x55e0b778e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 -- 172.21.15.143:0/47455673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 msgr2=0x55e0b77a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b77a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 -- 172.21.15.143:0/47455673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b7788960 con 0x55e0b778cb40 2024-09-10T10:50:18.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f476cd5d640 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 0x55e0b7787d20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f475c0c5370 tx=0x7f475c0bb140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6ece94c9bf5e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f474effd640 1 -- 172.21.15.143:0/47455673 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bc590 con 0x55e0b778cb40 2024-09-10T10:50:18.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f474effd640 1 -- 172.21.15.143:0/47455673 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475c0bcc30 con 0x55e0b778cb40 2024-09-10T10:50:18.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.395+0000 7f474effd640 1 -- 172.21.15.143:0/47455673 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004160 con 0x55e0b778cb40 2024-09-10T10:50:18.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 -- 172.21.15.143:0/47455673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 msgr2=0x55e0b7787d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 0x55e0b7787d20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f475c0c5370 tx=0x7f475c0bb140 comp rx=0 tx=0).stop 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 -- 172.21.15.143:0/47455673 shutdown_connections 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b7796370 0x55e0b778e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b778cb40 0x55e0b7787d20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 --2- 172.21.15.143:0/47455673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b77a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.396+0000 7f4771b9b740 1 -- 172.21.15.143:0/47455673 >> 172.21.15.143:0/47455673 conn(0x55e0b7754310 msgr2=0x55e0b7787610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.397+0000 7f4771b9b740 1 -- 172.21.15.143:0/47455673 shutdown_connections 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.397+0000 7f4771b9b740 1 -- 172.21.15.143:0/47455673 wait complete. 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.397+0000 7f4771b9b740 1 -- start start 2024-09-10T10:50:18.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.397+0000 7f4771b9b740 1 Processor -- start 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7799ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b779a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b780ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4754010ec0 con 0x55e0b7796370 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47540107f0 con 0x55e0b778b5d0 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f475400da30 con 0x55e0b778cb40 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f4771b9b740 10 client.0 _ll_register_callbacks cb 0x7f475c0048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b780ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476cd5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b779a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b780ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54838/0 (socket says 172.21.15.143:54838) 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 -- 172.21.15.143:0/663831289 learned_addr learned my addr 172.21.15.143:0/663831289 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476d55e640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7799ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 -- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 msgr2=0x55e0b779a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b779a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 -- 172.21.15.143:0/663831289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 msgr2=0x55e0b7799ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7799ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476dd5f640 1 -- 172.21.15.143:0/663831289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b780f340 con 0x55e0b7796370 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.398+0000 7f476d55e640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7799ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f476dd5f640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b780ed00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4758006010 tx=0x7f475800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4999d3eb006130 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f474cff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f474cff9640 1 -- 172.21.15.143:0/663831289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758015050 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f474cff9640 1 -- 172.21.15.143:0/663831289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4758005450 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c0bcc30 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f474cff9640 1 -- 172.21.15.143:0/663831289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f475802f050 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f4771b9b740 10 client.16662 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e0b7810620 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.399+0000 7f474cff9640 1 -- 172.21.15.143:0/663831289 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f475c0bcc30 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f474cff9640 1 -- 172.21.15.143:0/663831289 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4758023070 con 0x55e0b7796370 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f4771b9b740 10 client.16662 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f4771b9b740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:18.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f4771b9b740 20 client.16662 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:18.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f4771b9b740 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b7810a00 0x55e0b7812e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f4771b9b740 4 client.16662 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:50:18.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0b76f8d70 con 0x55e0b7810a00 2024-09-10T10:50:18.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f476d55e640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b7810a00 0x55e0b7812e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f474cff9640 10 client.16662 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f476d55e640 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b7810a00 0x55e0b7812e20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea703703309a327a server_cookie=655c6661151aeb41 in_seq=0 out_seq=0 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.400+0000 7f474cff9640 1 -- 172.21.15.143:0/663831289 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0b76f8d70 con 0x55e0b7810a00 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f474cff9640 10 client.16662 handle_command_reply: tid=0 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 client.16662 shutdown 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 msgr2=0x55e0b780ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b780ed00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4758006010 tx=0x7f475800bfc0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 shutdown_connections 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e0b7796370 0x55e0b780ed00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e0b7810a00 0x55e0b7812e20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b778cb40 0x55e0b779a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 --2- 172.21.15.143:0/663831289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0b778b5d0 0x55e0b7799ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f474cff9640 0 client.16662 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 >> 172.21.15.143:0/663831289 conn(0x55e0b7754310 msgr2=0x55e0b7786500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 shutdown_connections 2024-09-10T10:50:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 1 -- 172.21.15.143:0/663831289 wait complete. 2024-09-10T10:50:18.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.401+0000 7f4771b9b740 20 client.16662 trim_cache size 0 max 16384 2024-09-10T10:50:18.409 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:50:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f47500a2c00 msgr2=0x7f47500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f47500a2c00 0x7f47500a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4760005fd0 tx=0x7f47600094a0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 msgr2=0x7f4768109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 0x7f4768109910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f475c009ca0 tx=0x7f475c003c20 comp rx=0 tx=0).stop 2024-09-10T10:50:18.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 shutdown_connections 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f47500a2c00 0x7f47500a50c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4768102ef0 0x7f4768109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768102570 0x7f4768109910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 --2- 172.21.15.143:0/3317865594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4768100e20 0x7f476810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 >> 172.21.15.143:0/3317865594 conn(0x7f47680fc7d0 msgr2=0x7f476806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.402+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 shutdown_connections 2024-09-10T10:50:18.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:18.403+0000 7f476f7c2640 1 -- 172.21.15.143:0/3317865594 wait complete. 2024-09-10T10:50:20.220 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:50:20.220 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:50:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:20.893 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:20 smithi143 ceph-mon[31057]: pgmap v509: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 524 KiB/s wr, 52 op/s 2024-09-10T10:50:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:20 smithi178 ceph-mon[34813]: pgmap v509: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 524 KiB/s wr, 52 op/s 2024-09-10T10:50:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:20 smithi144 ceph-mon[34230]: pgmap v509: 129 pgs: 129 active+clean; 364 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 524 KiB/s wr, 52 op/s 2024-09-10T10:50:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.150+0000 7f71cb558640 1 -- 172.21.15.143:0/1819106531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 msgr2=0x7f71c4106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.150+0000 7f71cb558640 1 --2- 172.21.15.143:0/1819106531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c4106e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009930 tx=0x7f71c002a420 comp rx=0 tx=0).stop 2024-09-10T10:50:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 -- 172.21.15.143:0/1819106531 shutdown_connections 2024-09-10T10:50:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 --2- 172.21.15.143:0/1819106531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 0x7f71c410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 --2- 172.21.15.143:0/1819106531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 --2- 172.21.15.143:0/1819106531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c4106e70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 -- 172.21.15.143:0/1819106531 >> 172.21.15.143:0/1819106531 conn(0x7f71c40fc7d0 msgr2=0x7f71c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 -- 172.21.15.143:0/1819106531 shutdown_connections 2024-09-10T10:50:21.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.151+0000 7f71cb558640 1 -- 172.21.15.143:0/1819106531 wait complete. 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.152+0000 7f71cb558640 1 Processor -- start 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.152+0000 7f71cb558640 1 -- start start 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.152+0000 7f71cb558640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.152+0000 7f71cb558640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 0x7f71c419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.152+0000 7f71cb558640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c41a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.152+0000 7f71cb558640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71c406b690 con 0x7f71c4106a90 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71cb558640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71c406b510 con 0x7f71c4101430 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71cb558640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71c410fe40 con 0x7f71c4100a90 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c8af3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 0x7f71c419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46422/0 (socket says 172.21.15.143:46422) 2024-09-10T10:50:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c8af3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 0x7f71c419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46286/0 (socket says 172.21.15.143:46286) 2024-09-10T10:50:21.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 -- 172.21.15.143:0/325742473 learned_addr learned my addr 172.21.15.143:0/325742473 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:21.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c9af5640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c41a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 -- 172.21.15.143:0/325742473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 msgr2=0x7f71c419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 0x7f71c419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 -- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 msgr2=0x7f71c41a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c41a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 -- 172.21.15.143:0/325742473 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c41a67e0 con 0x7f71c4100a90 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c9af5640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c41a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.153+0000 7f71c92f4640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c419b340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71c0005940 tx=0x7f71c00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.154+0000 7f71ba7fc640 1 -- 172.21.15.143:0/325742473 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c003f030 con 0x7f71c4100a90 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.154+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c41a6a70 con 0x7f71c4100a90 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.154+0000 7f71ba7fc640 1 -- 172.21.15.143:0/325742473 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c00318a0 con 0x7f71c4100a90 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.154+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c41a6fa0 con 0x7f71c4100a90 2024-09-10T10:50:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.154+0000 7f71ba7fc640 1 -- 172.21.15.143:0/325742473 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0035070 con 0x7f71c4100a90 2024-09-10T10:50:21.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.157+0000 7f71ba7fc640 1 -- 172.21.15.143:0/325742473 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f71c0042030 con 0x7f71c4100a90 2024-09-10T10:50:21.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.158+0000 7f71ba7fc640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71b00a2c00 0x7f71b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.158+0000 7f71cd931740 1 Processor -- start 2024-09-10T10:50:21.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.158+0000 7f71c8af3640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71b00a2c00 0x7f71b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.158+0000 7f71ba7fc640 1 -- 172.21.15.143:0/325742473 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71c00ef9f0 con 0x7f71c4100a90 2024-09-10T10:50:21.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.158+0000 7f71cd931740 1 -- start start 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71cd931740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 0x562c862afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71cd931740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862b03a0 0x562c862c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71cd931740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71cd931740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562c86215280 con 0x562c862afa80 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71cd931740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562c861bcb10 con 0x562c862b03a0 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71cd931740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c86211240 con 0x562c862c3550 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 0x562c862afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 0x562c862afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54868/0 (socket says 172.21.15.143:54868) 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 -- 172.21.15.143:0/2937359151 learned_addr learned my addr 172.21.15.143:0/2937359151 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c8af3640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71b00a2c00 0x7f71b00a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71c410dae0 tx=0x7f71b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c8af3640 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862b03a0 0x562c862c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c9af5640 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 -- 172.21.15.143:0/2937359151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 msgr2=0x562c862c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 -- 172.21.15.143:0/2937359151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862b03a0 msgr2=0x562c862c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.159+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862b03a0 0x562c862c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.160+0000 7f71c92f4640 1 -- 172.21.15.143:0/2937359151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c861c91f0 con 0x562c862afa80 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.160+0000 7f71c9af5640 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.160+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 0x562c862afe60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f71c0031440 tx=0x7f71c00bba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d5dd320c05dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.160+0000 7f71af7fe640 1 -- 172.21.15.143:0/2937359151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0049040 con 0x562c862afa80 2024-09-10T10:50:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.160+0000 7f71af7fe640 1 -- 172.21.15.143:0/2937359151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c00c0240 con 0x562c862afa80 2024-09-10T10:50:21.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.160+0000 7f71af7fe640 1 -- 172.21.15.143:0/2937359151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00c4670 con 0x562c862afa80 2024-09-10T10:50:21.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 -- 172.21.15.143:0/2937359151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 msgr2=0x562c862afe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 0x562c862afe60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f71c0031440 tx=0x7f71c00bba80 comp rx=0 tx=0).stop 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 -- 172.21.15.143:0/2937359151 shutdown_connections 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862b03a0 0x562c862c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 --2- 172.21.15.143:0/2937359151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862afa80 0x562c862afe60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 -- 172.21.15.143:0/2937359151 >> 172.21.15.143:0/2937359151 conn(0x562c862ac0f0 msgr2=0x562c862ac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 -- 172.21.15.143:0/2937359151 shutdown_connections 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.161+0000 7f71cd931740 1 -- 172.21.15.143:0/2937359151 wait complete. 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 -- start start 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 Processor -- start 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562c862c5e80 con 0x562c862b03a0 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562c861cff10 con 0x562c862afa80 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.162+0000 7f71cd931740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c862c6000 con 0x562c862c3550 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46438/0 (socket says 172.21.15.143:46438) 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c8af3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 -- 172.21.15.143:0/2539772250 learned_addr learned my addr 172.21.15.143:0/2539772250 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71cd931740 10 client.0 _ll_register_callbacks cb 0x562c86198a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 -- 172.21.15.143:0/2539772250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 msgr2=0x562c862c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 -- 172.21.15.143:0/2539772250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 msgr2=0x562c862c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862c0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 -- 172.21.15.143:0/2539772250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c85f866a0 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71c9af5640 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c0f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f71bc010ee0 tx=0x7f71bc00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47134fcb6d6ccb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71acff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.163+0000 7f71acff9640 1 -- 172.21.15.143:0/2539772250 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc004d20 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c00c0240 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71acff9640 1 -- 172.21.15.143:0/2539772250 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71bc005020 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71acff9640 1 -- 172.21.15.143:0/2539772250 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f71bc023050 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 10 client.26080 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562c862d1b70 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71acff9640 1 -- 172.21.15.143:0/2539772250 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71bc0056d0 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71acff9640 1 -- 172.21.15.143:0/2539772250 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f71bc01a890 con 0x562c862c3550 2024-09-10T10:50:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 10 client.26080 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 10 client.26080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 20 client.26080 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c862d3450 0x562c862d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 4 client.26080 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.164+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71c00c0240 con 0x562c862d3450 2024-09-10T10:50:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.165+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c862d3450 0x562c862d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.165+0000 7f71acff9640 10 client.26080 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.165+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c862d3450 0x562c862d5870 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ba0edab165da7d server_cookie=4329141a6f2a6900 in_seq=0 out_seq=0 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71acff9640 1 -- 172.21.15.143:0/2539772250 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f71c00c0240 con 0x562c862d3450 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71acff9640 10 client.26080 handle_command_reply: tid=0 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 client.26080 shutdown 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 msgr2=0x562c862c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c0f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f71bc010ee0 tx=0x7f71bc00e720 comp rx=0 tx=0).stop 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 shutdown_connections 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c862d3450 0x562c862d5870 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862c3550 0x562c862c0f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 --2- 172.21.15.143:0/2539772250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 >> 172.21.15.143:0/2539772250 conn(0x562c862ac0f0 msgr2=0x562c862b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71acff9640 0 client.26080 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:21.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 shutdown_connections 2024-09-10T10:50:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 1 -- 172.21.15.143:0/2539772250 wait complete. 2024-09-10T10:50:21.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.166+0000 7f71cd931740 20 client.26080 trim_cache size 0 max 16384 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 Processor -- start 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 -- start start 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862afa80 0x562c862c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862c3550 0x562c862db6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71a800fa90 con 0x562c862b03a0 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71a80115d0 con 0x562c862c3550 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71cd931740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8010160 con 0x562c862afa80 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71c8af3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862afa80 0x562c862c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.193+0000 7f71c8af3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54890/0 (socket says 172.21.15.143:54890) 2024-09-10T10:50:21.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c8af3640 1 -- 172.21.15.143:0/2616519639 learned_addr learned my addr 172.21.15.143:0/2616519639 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:21.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c9af5640 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862c3550 0x562c862db6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c92f4640 1 -- 172.21.15.143:0/2616519639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862c3550 msgr2=0x562c862db6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862c3550 0x562c862db6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c92f4640 1 -- 172.21.15.143:0/2616519639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 msgr2=0x562c862db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c92f4640 1 -- 172.21.15.143:0/2616519639 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c862a7830 con 0x562c862afa80 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71c92f4640 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862afa80 0x562c862c2350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f71c00b8910 tx=0x7f71c00bbbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6037b0c84f93eef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71aeffd640 1 -- 172.21.15.143:0/2616519639 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0049040 con 0x562c862afa80 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71aeffd640 1 -- 172.21.15.143:0/2616519639 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c00d8040 con 0x562c862afa80 2024-09-10T10:50:21.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.194+0000 7f71aeffd640 1 -- 172.21.15.143:0/2616519639 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00bc400 con 0x562c862afa80 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 -- 172.21.15.143:0/2616519639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862afa80 msgr2=0x562c862c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862afa80 0x562c862c2350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f71c00b8910 tx=0x7f71c00bbbb0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 -- 172.21.15.143:0/2616519639 shutdown_connections 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862c3550 0x562c862db6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862b03a0 0x562c862db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 --2- 172.21.15.143:0/2616519639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862afa80 0x562c862c2350 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.195+0000 7f71cd931740 1 -- 172.21.15.143:0/2616519639 >> 172.21.15.143:0/2616519639 conn(0x562c862ac0f0 msgr2=0x562c862c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.196+0000 7f71cd931740 1 -- 172.21.15.143:0/2616519639 shutdown_connections 2024-09-10T10:50:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.196+0000 7f71cd931740 1 -- 172.21.15.143:0/2616519639 wait complete. 2024-09-10T10:50:21.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.196+0000 7f71cd931740 1 -- start start 2024-09-10T10:50:21.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.196+0000 7f71cd931740 1 Processor -- start 2024-09-10T10:50:21.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862b03a0 0x562c862cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862c3550 0x562c862c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46312/0 (socket says 172.21.15.143:46312) 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 -- 172.21.15.143:0/3365084917 learned_addr learned my addr 172.21.15.143:0/3365084917 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8010f00 con 0x562c862c3550 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8010830 con 0x562c862afa80 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71a800da70 con 0x562c862b03a0 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c8af3640 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862b03a0 0x562c862cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71cd931740 10 client.0 _ll_register_callbacks cb 0x7f71bc007c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 -- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862b03a0 msgr2=0x562c862cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862b03a0 0x562c862cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 -- 172.21.15.143:0/3365084917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862c3550 msgr2=0x562c862c6b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.197+0000 7f71c92f4640 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862c3550 0x562c862c6b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.198+0000 7f71c92f4640 1 -- 172.21.15.143:0/3365084917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c862c73b0 con 0x562c862afa80 2024-09-10T10:50:21.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.198+0000 7f71c92f4640 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862cb620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006010 tx=0x7f71c00bc0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7920498088f3c522 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.198+0000 7f71acff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.199+0000 7f71acff9640 1 -- 172.21.15.143:0/3365084917 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0049070 con 0x562c862afa80 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.199+0000 7f71acff9640 1 -- 172.21.15.143:0/3365084917 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71c00bc1a0 con 0x562c862afa80 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.199+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c00d8040 con 0x562c862afa80 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.199+0000 7f71acff9640 1 -- 172.21.15.143:0/3365084917 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f71c00c0240 con 0x562c862afa80 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.199+0000 7f71cd931740 10 client.26027 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.199+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562c8634ee10 con 0x562c862afa80 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71acff9640 1 -- 172.21.15.143:0/3365084917 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c00d8040 con 0x562c862afa80 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71acff9640 1 -- 172.21.15.143:0/3365084917 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f71c00b7c90 con 0x562c862afa80 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71cd931740 10 client.26027 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71cd931740 10 client.26027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71cd931740 20 client.26027 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71cd931740 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c8634f0e0 0x562c863514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71cd931740 4 client.26027 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c862227f0 con 0x562c8634f0e0 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.200+0000 7f71c8af3640 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c8634f0e0 0x562c863514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.201+0000 7f71c8af3640 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c8634f0e0 0x562c863514d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5bbab65910c090 server_cookie=9a180a3858304770 in_seq=0 out_seq=0 2024-09-10T10:50:21.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.201+0000 7f71acff9640 10 client.26027 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71acff9640 1 -- 172.21.15.143:0/3365084917 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c862227f0 con 0x562c8634f0e0 2024-09-10T10:50:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71acff9640 10 client.26027 handle_command_reply: tid=0 2024-09-10T10:50:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 client.26027 shutdown 2024-09-10T10:50:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 msgr2=0x562c862cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862cb620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006010 tx=0x7f71c00bc0c0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 shutdown_connections 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562c862c3550 0x562c862c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562c8634f0e0 0x562c863514d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.202+0000 7f71cd931740 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c862b03a0 0x562c862cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cd931740 1 --2- 172.21.15.143:0/3365084917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c862afa80 0x562c862cb620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71acff9640 0 client.26027 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 >> 172.21.15.143:0/3365084917 conn(0x562c862ac0f0 msgr2=0x562c862be980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 shutdown_connections 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cd931740 1 -- 172.21.15.143:0/3365084917 wait complete. 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cd931740 20 client.26027 trim_cache size 0 max 16384 2024-09-10T10:50:21.205 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "73af3b16-9927-40cb-b078-5f6e5bedec43", 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71b00a2c00 msgr2=0x7f71b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cb558640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71b00a2c00 0x7f71b00a50c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71c410dae0 tx=0x7f71b4005f90 comp rx=0 tx=0).stop 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 msgr2=0x7f71c419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.203+0000 7f71cb558640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c419b340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71c0005940 tx=0x7f71c00042f0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 shutdown_connections 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71b00a2c00 0x7f71b00a50c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c4106a90 0x7f71c41a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71c4101430 0x7f71c419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 --2- 172.21.15.143:0/325742473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c4100a90 0x7f71c419b340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 >> 172.21.15.143:0/325742473 conn(0x7f71c40fc7d0 msgr2=0x7f71c40fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 shutdown_connections 2024-09-10T10:50:21.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:21.204+0000 7f71cb558640 1 -- 172.21.15.143:0/325742473 wait complete. 2024-09-10T10:50:21.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:21 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:50:21.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:21 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:50:22.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:21 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:50:22.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:50:22.421 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:22 smithi143 ceph-mon[31057]: pgmap v510: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 899 KiB/s wr, 62 op/s 2024-09-10T10:50:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:50:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:50:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:50:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:22 smithi178 ceph-mon[34813]: pgmap v510: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 899 KiB/s wr, 62 op/s 2024-09-10T10:50:22.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:50:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:50:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:50:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:22.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:22 smithi144 ceph-mon[34230]: pgmap v510: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 899 KiB/s wr, 62 op/s 2024-09-10T10:50:22.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 -- 172.21.15.143:0/444815191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410c590 msgr2=0x7f1fb410c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/444815191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410c590 0x7f1fb410c9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 -- 172.21.15.143:0/444815191 shutdown_connections 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/444815191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410cf30 0x7f1fb4119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/444815191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410c590 0x7f1fb410c9f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/444815191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb4112640 0x7f1fb4112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 -- 172.21.15.143:0/444815191 >> 172.21.15.143:0/444815191 conn(0x7f1fb406cdc0 msgr2=0x7f1fb406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 -- 172.21.15.143:0/444815191 shutdown_connections 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.959+0000 7f1fb96ef640 1 -- 172.21.15.143:0/444815191 wait complete. 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.960+0000 7f1fb96ef640 1 Processor -- start 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.960+0000 7f1fb96ef640 1 -- start start 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.960+0000 7f1fb96ef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 0x7f1fb41a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.960+0000 7f1fb96ef640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410cf30 0x7f1fb41a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb96ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb4112640 0x7f1fb41a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 0x7f1fb41a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 0x7f1fb41a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46466/0 (socket says 172.21.15.143:46466) 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1959261754 learned_addr learned my addr 172.21.15.143:0/1959261754 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb411c810 con 0x7f1fb410cf30 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb4112640 0x7f1fb41a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410cf30 0x7f1fb41a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb411c690 con 0x7f1fb4112640 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb411c990 con 0x7f1fb410c590 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1959261754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb4112640 msgr2=0x7f1fb41a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb4112640 0x7f1fb41a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410cf30 msgr2=0x7f1fb41a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410cf30 0x7f1fb41a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1959261754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41a7de0 con 0x7f1fb410c590 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.961+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410cf30 0x7f1fb41a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.962+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 0x7f1fb41a2e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e9f0 tx=0x7f1fa400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.962+0000 7f1f9bfff640 1 -- 172.21.15.143:0/1959261754 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400cd80 con 0x7f1fb410c590 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.962+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb41a80d0 con 0x7f1fb410c590 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.962+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41af9b0 con 0x7f1fb410c590 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.962+0000 7f1f9bfff640 1 -- 172.21.15.143:0/1959261754 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa40108e0 con 0x7f1fb410c590 2024-09-10T10:50:22.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.962+0000 7f1f9bfff640 1 -- 172.21.15.143:0/1959261754 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4018920 con 0x7f1fb410c590 2024-09-10T10:50:22.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 Processor -- start 2024-09-10T10:50:22.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1f9bfff640 1 -- 172.21.15.143:0/1959261754 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1fa4018ac0 con 0x7f1fb410c590 2024-09-10T10:50:22.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 -- start start 2024-09-10T10:50:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0059880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0059dc0 0x562ff0064140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0064680 0x562ff0066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562feffbf2e0 con 0x562ff0059dc0 2024-09-10T10:50:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562feff66ce0 con 0x562ff0064680 2024-09-10T10:50:22.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562feffbb2a0 con 0x562ff00594a0 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0059dc0 0x562ff0064140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fb37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0059880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fb27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0064680 0x562ff0066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0059dc0 0x562ff0064140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54922/0 (socket says 172.21.15.143:54922) 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/819458879 learned_addr learned my addr 172.21.15.143:0/819458879 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.965+0000 7f1fb37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0059880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46482/0 (socket says 172.21.15.143:46482) 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1fb27fc640 1 -- 172.21.15.143:0/819458879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 msgr2=0x562ff0059880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0059880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1fb27fc640 1 -- 172.21.15.143:0/819458879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0059dc0 msgr2=0x562ff0064140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0059dc0 0x562ff0064140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1fb27fc640 1 -- 172.21.15.143:0/819458879 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562feff4b440 con 0x562ff0064680 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0064680 0x562ff0066a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8005940 tx=0x7f1fa8032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60124d0d22eb15c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1f9bfff640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1f9c0a2b80 0x7f1f9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.966+0000 7f1f9bfff640 1 -- 172.21.15.143:0/1959261754 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1fa40cf660 con 0x7f1fb410c590 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1f997fa640 1 -- 172.21.15.143:0/819458879 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8035040 con 0x562ff0064680 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1f997fa640 1 -- 172.21.15.143:0/819458879 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa80316b0 con 0x562ff0064680 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1f9c0a2b80 0x7f1f9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1f997fa640 1 -- 172.21.15.143:0/819458879 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8003dc0 con 0x562ff0064680 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1fbbac8740 1 -- 172.21.15.143:0/819458879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0064680 msgr2=0x562ff0066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0064680 0x562ff0066a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8005940 tx=0x7f1fa8032e60 comp rx=0 tx=0).stop 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.967+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1f9c0a2b80 0x7f1f9c0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005fd0 tx=0x7f1fac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 -- 172.21.15.143:0/819458879 shutdown_connections 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0064680 0x562ff0066a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0059dc0 0x562ff0064140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/819458879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0059880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 -- 172.21.15.143:0/819458879 >> 172.21.15.143:0/819458879 conn(0x562ff0055a30 msgr2=0x562ff0055e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 -- 172.21.15.143:0/819458879 shutdown_connections 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 -- 172.21.15.143:0/819458879 wait complete. 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 -- start start 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.968+0000 7f1fbbac8740 1 Processor -- start 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0064010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0059dc0 0x562ff00612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0061840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562ff0067250 con 0x562ff0064680 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562feff7a050 con 0x562ff0059dc0 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ff00673d0 con 0x562ff00594a0 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fbbac8740 10 client.0 _ll_register_callbacks cb 0x562feffba160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0064010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0064010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46486/0 (socket says 172.21.15.143:46486) 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1158355566 learned_addr learned my addr 172.21.15.143:0/1158355566 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0059dc0 0x562ff00612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1158355566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0059dc0 msgr2=0x562ff00612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0059dc0 0x562ff00612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0061840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1158355566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 msgr2=0x562ff0061840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0061840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1158355566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fefd303c0 con 0x562ff00594a0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.969+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0061840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0064010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40a4240 tx=0x7f1fa40ccd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d5bbf64af76c46f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1f96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1f96ff5640 1 -- 172.21.15.143:0/1158355566 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa409b510 con 0x562ff00594a0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1f96ff5640 1 -- 172.21.15.143:0/1158355566 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1fa4098220 con 0x562ff00594a0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa80316b0 con 0x562ff00594a0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1f96ff5640 1 -- 172.21.15.143:0/1158355566 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1fa4010de0 con 0x562ff00594a0 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1fbbac8740 10 client.26098 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562ff0072ca0 con 0x562ff00594a0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1f96ff5640 1 -- 172.21.15.143:0/1158355566 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa80316b0 con 0x562ff00594a0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.970+0000 7f1f96ff5640 1 -- 172.21.15.143:0/1158355566 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f1fa40d1cd0 con 0x562ff00594a0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fbbac8740 10 client.26098 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fbbac8740 10 client.26098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fbbac8740 20 client.26098 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00740b0 0x562ff0076520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fbbac8740 4 client.26098 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562feffcc850 con 0x562ff00740b0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.971+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00740b0 0x562ff0076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00740b0 0x562ff0076520 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e7057b9498e63 server_cookie=8a9422f95cba2250 in_seq=0 out_seq=0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1f96ff5640 10 client.26098 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1f96ff5640 1 -- 172.21.15.143:0/1158355566 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562feffcc850 con 0x562ff00740b0 2024-09-10T10:50:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1f96ff5640 10 client.26098 handle_command_reply: tid=0 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1fbbac8740 1 client.26098 shutdown 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 msgr2=0x562ff0064010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.972+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0064010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40a4240 tx=0x7f1fa40ccd60 comp rx=0 tx=0).stop 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 shutdown_connections 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0061840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff0059dc0 0x562ff00612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00740b0 0x562ff0076520 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1158355566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff00594a0 0x562ff0064010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1f96ff5640 0 client.26098 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 >> 172.21.15.143:0/1158355566 conn(0x562ff0055a30 msgr2=0x562ff005b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 shutdown_connections 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1158355566 wait complete. 2024-09-10T10:50:22.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:22.973+0000 7f1fbbac8740 20 client.26098 trim_cache size 0 max 16384 2024-09-10T10:50:23.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.000+0000 7f1fbbac8740 1 Processor -- start 2024-09-10T10:50:23.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 -- start start 2024-09-10T10:50:23.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff00704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff005c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff007be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b95b0 con 0x562ff0064680 2024-09-10T10:50:23.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41bb0f0 con 0x562ff00594a0 2024-09-10T10:50:23.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fbbac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b9c80 con 0x562ff0059dc0 2024-09-10T10:50:23.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff00704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff00704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46342/0 (socket says 172.21.15.143:46342) 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1805971231 learned_addr learned my addr 172.21.15.143:0/1805971231 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff005c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff007be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1805971231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 msgr2=0x562ff005c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff005c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1805971231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 msgr2=0x562ff007be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff007be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.001+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/1805971231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff0061bb0 con 0x562ff00594a0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.002+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff00704b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4098010 tx=0x7f1fa4009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea7ecb78aae5f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.002+0000 7f1f98ff9640 1 -- 172.21.15.143:0/1805971231 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa409b980 con 0x562ff00594a0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.002+0000 7f1f98ff9640 1 -- 172.21.15.143:0/1805971231 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa409feb0 con 0x562ff00594a0 2024-09-10T10:50:23.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.002+0000 7f1f98ff9640 1 -- 172.21.15.143:0/1805971231 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40979d0 con 0x562ff00594a0 2024-09-10T10:50:23.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1805971231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 msgr2=0x562ff00704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff00704b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4098010 tx=0x7f1fa4009700 comp rx=0 tx=0).stop 2024-09-10T10:50:23.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1805971231 shutdown_connections 2024-09-10T10:50:23.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff007be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff005c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/1805971231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff00704b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1805971231 >> 172.21.15.143:0/1805971231 conn(0x562ff0055a30 msgr2=0x562ff005c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1805971231 shutdown_connections 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 -- 172.21.15.143:0/1805971231 wait complete. 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 -- start start 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.003+0000 7f1fbbac8740 1 Processor -- start 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff0067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff006cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:46344/0 (socket says 172.21.15.143:46344) 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fb27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff0067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fb27fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff0067ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:46506/0 (socket says 172.21.15.143:46506) 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fbbac8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0068400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/780621339 learned_addr learned my addr 172.21.15.143:0/780621339 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41baa20 con 0x562ff0064680 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41ba350 con 0x562ff00594a0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fb37fe640 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.004+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b7590 con 0x562ff0059dc0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fbbac8740 10 client.0 _ll_register_callbacks cb 0x7f1fa40d2d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 msgr2=0x562ff0067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff0067ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/780621339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 msgr2=0x562ff0068400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0068400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fb2ffd640 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff0068a40 con 0x562ff00594a0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fb2ffd640 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff006cb90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fa409bb40 tx=0x7f1fa4098450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11bf64e911f6ba35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1f96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1f96ff5640 1 -- 172.21.15.143:0/780621339 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa409b980 con 0x562ff00594a0 2024-09-10T10:50:23.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1f96ff5640 1 -- 172.21.15.143:0/780621339 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1fa4097b10 con 0x562ff00594a0 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa409feb0 con 0x562ff00594a0 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1f96ff5640 1 -- 172.21.15.143:0/780621339 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1fa408f050 con 0x562ff00594a0 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fbbac8740 10 client.26045 fetch_fsmap learned FSMap version 33 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.005+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562ff0068a40 con 0x562ff00594a0 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1f96ff5640 1 -- 172.21.15.143:0/780621339 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa40ba070 con 0x562ff00594a0 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1f96ff5640 1 -- 172.21.15.143:0/780621339 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f1fa40a1040 con 0x562ff00594a0 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fbbac8740 10 client.26045 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fbbac8740 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fbbac8740 20 client.26045 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00dee20 0x562ff00e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fbbac8740 4 client.26045 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1fa409feb0 con 0x562ff00dee20 2024-09-10T10:50:23.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.006+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00dee20 0x562ff00e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1f96ff5640 10 client.26045 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fb27fc640 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00dee20 0x562ff00e1210 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e031fbee626154e6 server_cookie=caa499994e311193 in_seq=0 out_seq=0 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1f96ff5640 1 -- 172.21.15.143:0/780621339 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f1fa409feb0 con 0x562ff00dee20 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1f96ff5640 10 client.26045 handle_command_reply: tid=0 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 client.26045 shutdown 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 msgr2=0x562ff006cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff006cb90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fa409bb40 tx=0x7f1fa4098450 comp rx=0 tx=0).stop 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 shutdown_connections 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562ff0064680 0x562ff0068400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x562ff00dee20 0x562ff00e1210 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ff0059dc0 0x562ff0067ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1f96ff5640 0 client.26045 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 --2- 172.21.15.143:0/780621339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff00594a0 0x562ff006cb90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 >> 172.21.15.143:0/780621339 conn(0x562ff0055a30 msgr2=0x562ff0054a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.007+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 shutdown_connections 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.008+0000 7f1fbbac8740 1 -- 172.21.15.143:0/780621339 wait complete. 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.008+0000 7f1fbbac8740 20 client.26045 trim_cache size 0 max 16384 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stdout: "73af3b16-9927-40cb-b078-5f6e5bedec43": { 2024-09-10T10:50:23.013 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "73af3b16-9927-40cb-b078-5f6e5bedec43", 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.008+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1f9c0a2b80 msgr2=0x7f1f9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.008+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1f9c0a2b80 0x7f1f9c0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005fd0 tx=0x7f1fac009500 comp rx=0 tx=0).stop 2024-09-10T10:50:23.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.008+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 msgr2=0x7f1fb41a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.008+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 0x7f1fb41a2e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e9f0 tx=0x7f1fa400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 shutdown_connections 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1f9c0a2b80 0x7f1f9c0a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb4112640 0x7f1fb41a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fb410cf30 0x7f1fb41a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 --2- 172.21.15.143:0/1959261754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb410c590 0x7f1fb41a2e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 >> 172.21.15.143:0/1959261754 conn(0x7f1fb406cdc0 msgr2=0x7f1fb410b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 shutdown_connections 2024-09-10T10:50:23.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:23.009+0000 7f1fb96ef640 1 -- 172.21.15.143:0/1959261754 wait complete. 2024-09-10T10:50:23.592 INFO:tasks.workunit.client.0.smithi143.stdout: 76 9199 87 6005 88 2879 165 2024-09-10T10:50:23.886 INFO:tasks.cephfs.filesystem:scrub status for tag:73af3b16-9927-40cb-b078-5f6e5bedec43 - {'path': '/', 'tag': '73af3b16-9927-40cb-b078-5f6e5bedec43', 'options': 'recursive,force'} 2024-09-10T10:50:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:24 smithi178 ceph-mon[34813]: pgmap v511: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 350 op/s 2024-09-10T10:50:24.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:24 smithi144 ceph-mon[34230]: pgmap v511: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 350 op/s 2024-09-10T10:50:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:24 smithi143 ceph-mon[31057]: pgmap v511: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 350 op/s 2024-09-10T10:50:25.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:25 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:25.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:25 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:26.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:25 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:50:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:26 smithi178 ceph-mon[34813]: pgmap v512: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 468 op/s 2024-09-10T10:50:26.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:26 smithi144 ceph-mon[34230]: pgmap v512: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 468 op/s 2024-09-10T10:50:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:26 smithi143 ceph-mon[31057]: pgmap v512: 129 pgs: 129 active+clean; 377 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 468 op/s 2024-09-10T10:50:27.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:27 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:50:27.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:27 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:50:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:27 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:50:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:28 smithi178 ceph-mon[34813]: pgmap v513: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 833 op/s 2024-09-10T10:50:28.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:28 smithi144 ceph-mon[34230]: pgmap v513: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 833 op/s 2024-09-10T10:50:29.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:28 smithi143 ceph-mon[31057]: pgmap v513: 129 pgs: 129 active+clean; 368 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 833 op/s 2024-09-10T10:50:29.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:29 smithi178 ceph-mon[34813]: pgmap v514: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-10T10:50:29.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:29 smithi144 ceph-mon[34230]: pgmap v514: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-10T10:50:30.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:29 smithi143 ceph-mon[31057]: pgmap v514: 129 pgs: 129 active+clean; 369 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.02k op/s 2024-09-10T10:50:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:32 smithi178 ceph-mon[34813]: pgmap v515: 129 pgs: 129 active+clean; 380 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 1.02k op/s 2024-09-10T10:50:32.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:32 smithi144 ceph-mon[34230]: pgmap v515: 129 pgs: 129 active+clean; 380 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 1.02k op/s 2024-09-10T10:50:33.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:32 smithi143 ceph-mon[31057]: pgmap v515: 129 pgs: 129 active+clean; 380 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 1.02k op/s 2024-09-10T10:50:33.592 INFO:tasks.workunit.client.0.smithi143.stdout: 79 10635 113 7694 97 4419 162 2024-09-10T10:50:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:34 smithi178 ceph-mon[34813]: pgmap v516: 129 pgs: 129 active+clean; 371 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 1.04k op/s 2024-09-10T10:50:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:34 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:50:34.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:34 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:50:34.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:34 smithi144 ceph-mon[34230]: pgmap v516: 129 pgs: 129 active+clean; 371 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 1.04k op/s 2024-09-10T10:50:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:34 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:50:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:34 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:50:35.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:34 smithi143 ceph-mon[31057]: pgmap v516: 129 pgs: 129 active+clean; 371 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 1.04k op/s 2024-09-10T10:50:35.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:34 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:50:35.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:34 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:50:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:36 smithi178 ceph-mon[34813]: pgmap v517: 129 pgs: 129 active+clean; 371 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 762 op/s 2024-09-10T10:50:36.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:36 smithi144 ceph-mon[34230]: pgmap v517: 129 pgs: 129 active+clean; 371 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 762 op/s 2024-09-10T10:50:37.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:36 smithi143 ceph-mon[31057]: pgmap v517: 129 pgs: 129 active+clean; 371 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 762 op/s 2024-09-10T10:50:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:38 smithi178 ceph-mon[34813]: pgmap v518: 129 pgs: 129 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 675 op/s 2024-09-10T10:50:38.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:38 smithi144 ceph-mon[34230]: pgmap v518: 129 pgs: 129 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 675 op/s 2024-09-10T10:50:39.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:38 smithi143 ceph-mon[31057]: pgmap v518: 129 pgs: 129 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 675 op/s 2024-09-10T10:50:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:40 smithi178 ceph-mon[34813]: pgmap v519: 129 pgs: 129 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 345 op/s 2024-09-10T10:50:40.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:40 smithi144 ceph-mon[34230]: pgmap v519: 129 pgs: 129 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 345 op/s 2024-09-10T10:50:41.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:40 smithi143 ceph-mon[31057]: pgmap v519: 129 pgs: 129 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 345 op/s 2024-09-10T10:50:42.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:42 smithi143 ceph-mon[31057]: pgmap v520: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 164 op/s 2024-09-10T10:50:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:42 smithi178 ceph-mon[34813]: pgmap v520: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 164 op/s 2024-09-10T10:50:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:42 smithi144 ceph-mon[34230]: pgmap v520: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 164 op/s 2024-09-10T10:50:43.592 INFO:tasks.workunit.client.0.smithi143.stdout: 80 11250 149 7482 97 3389 247 2024-09-10T10:50:44.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:44 smithi178 ceph-mon[34813]: pgmap v521: 129 pgs: 129 active+clean; 375 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-09-10T10:50:44.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:44 smithi144 ceph-mon[34230]: pgmap v521: 129 pgs: 129 active+clean; 375 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-09-10T10:50:45.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:44 smithi143 ceph-mon[31057]: pgmap v521: 129 pgs: 129 active+clean; 375 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-09-10T10:50:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:46 smithi178 ceph-mon[34813]: pgmap v522: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 150 op/s 2024-09-10T10:50:46.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:46 smithi144 ceph-mon[34230]: pgmap v522: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 150 op/s 2024-09-10T10:50:47.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:46 smithi143 ceph-mon[31057]: pgmap v522: 129 pgs: 129 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 150 op/s 2024-09-10T10:50:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:48 smithi178 ceph-mon[34813]: pgmap v523: 129 pgs: 129 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 156 op/s 2024-09-10T10:50:48.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:48 smithi144 ceph-mon[34230]: pgmap v523: 129 pgs: 129 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 156 op/s 2024-09-10T10:50:49.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:48 smithi143 ceph-mon[31057]: pgmap v523: 129 pgs: 129 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 156 op/s 2024-09-10T10:50:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:50 smithi178 ceph-mon[34813]: pgmap v524: 129 pgs: 129 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-10T10:50:50.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:50 smithi144 ceph-mon[34230]: pgmap v524: 129 pgs: 129 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-10T10:50:51.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:50 smithi143 ceph-mon[31057]: pgmap v524: 129 pgs: 129 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 138 op/s 2024-09-10T10:50:52.908 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:52 smithi143 ceph-mon[31057]: pgmap v525: 129 pgs: 129 active+clean; 377 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 109 op/s 2024-09-10T10:50:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:52 smithi178 ceph-mon[34813]: pgmap v525: 129 pgs: 129 active+clean; 377 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 109 op/s 2024-09-10T10:50:52.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:52 smithi144 ceph-mon[34230]: pgmap v525: 129 pgs: 129 active+clean; 377 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 109 op/s 2024-09-10T10:50:53.592 INFO:tasks.workunit.client.0.smithi143.stdout: 83 11125 150 7914 150 4318 172 2024-09-10T10:50:53.887 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:50:53.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:53 smithi144 ceph-mon[34230]: pgmap v526: 129 pgs: 129 active+clean; 382 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 117 op/s 2024-09-10T10:50:54.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:53 smithi143 ceph-mon[31057]: pgmap v526: 129 pgs: 129 active+clean; 382 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 117 op/s 2024-09-10T10:50:54.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:53 smithi178 ceph-mon[34813]: pgmap v526: 129 pgs: 129 active+clean; 382 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 117 op/s 2024-09-10T10:50:54.247 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:54.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 -- 172.21.15.143:0/98990965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102ef0 msgr2=0x7f248c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 --2- 172.21.15.143:0/98990965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102ef0 0x7f248c10da40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2484009ca0 tx=0x7f248402a950 comp rx=0 tx=0).stop 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 -- 172.21.15.143:0/98990965 shutdown_connections 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 --2- 172.21.15.143:0/98990965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102ef0 0x7f248c10da40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 --2- 172.21.15.143:0/98990965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c102570 0x7f248c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 --2- 172.21.15.143:0/98990965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 -- 172.21.15.143:0/98990965 >> 172.21.15.143:0/98990965 conn(0x7f248c0fc7d0 msgr2=0x7f248c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 -- 172.21.15.143:0/98990965 shutdown_connections 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.776+0000 7f24909da640 1 -- 172.21.15.143:0/98990965 wait complete. 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 Processor -- start 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 -- start start 2024-09-10T10:50:54.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102570 0x7f248c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c102ef0 0x7f248c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f248c110e30 con 0x7f248c102570 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f248c110cb0 con 0x7f248c102ef0 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f24909da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f248c110fb0 con 0x7f248c100e20 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f2483fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102570 0x7f248c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:50666/0 (socket says 172.21.15.143:50666) 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f248a59c640 1 -- 172.21.15.143:0/3047381307 learned_addr learned my addr 172.21.15.143:0/3047381307 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f248ad9d640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c102ef0 0x7f248c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f248a59c640 1 -- 172.21.15.143:0/3047381307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c102ef0 msgr2=0x7f248c19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.777+0000 7f248a59c640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c102ef0 0x7f248c19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f248a59c640 1 -- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102570 msgr2=0x7f248c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f248a59c640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102570 0x7f248c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f248a59c640 1 -- 172.21.15.143:0/3047381307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c19f510 con 0x7f248c100e20 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f2483fff640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102570 0x7f248c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f248a59c640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c19a560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9f0 tx=0x7f247c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f24837fe640 1 -- 172.21.15.143:0/3047381307 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00cd80 con 0x7f248c100e20 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f24837fe640 1 -- 172.21.15.143:0/3047381307 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f247c0054d0 con 0x7f248c100e20 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c19f800 con 0x7f248c100e20 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f24837fe640 1 -- 172.21.15.143:0/3047381307 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c010490 con 0x7f248c100e20 2024-09-10T10:50:54.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.778+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1a70e0 con 0x7f248c100e20 2024-09-10T10:50:54.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.781+0000 7f24837fe640 1 -- 172.21.15.143:0/3047381307 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f247c005020 con 0x7f248c100e20 2024-09-10T10:50:54.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.781+0000 7f2492db3740 1 Processor -- start 2024-09-10T10:50:54.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 -- start start 2024-09-10T10:50:54.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f24837fe640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f24780a2c00 0x7f24780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6065d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60668140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e60668680 0x557e6066aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e605c32e0 con 0x557e60668680 2024-09-10T10:50:54.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557e6056ace0 con 0x557e6065ddc0 2024-09-10T10:50:54.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f2492db3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e605bf2a0 con 0x557e6065d4a0 2024-09-10T10:50:54.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f248ad9d640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f24780a2c00 0x7f24780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6065d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6065d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:50672/0 (socket says 172.21.15.143:50672) 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.782+0000 7f248a59c640 1 -- 172.21.15.143:0/508213544 learned_addr learned my addr 172.21.15.143:0/508213544 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f2483fff640 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60668140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248a59c640 1 -- 172.21.15.143:0/508213544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 msgr2=0x557e60668140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248a59c640 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60668140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248a59c640 1 -- 172.21.15.143:0/508213544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e60668680 msgr2=0x557e6066aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248a59c640 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e60668680 0x557e6066aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248a59c640 1 -- 172.21.15.143:0/508213544 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6054f440 con 0x557e6065d4a0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f24837fe640 1 -- 172.21.15.143:0/3047381307 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f247c0cf6d0 con 0x7f248c100e20 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248a59c640 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6065d880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f247c06a560 tx=0x7f247c05be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3011b0ea72a5d7e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f248ad9d640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f24780a2c00 0x7f24780a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2484005e30 tx=0x7f2484005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f2480ff9640 1 -- 172.21.15.143:0/508213544 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c097e70 con 0x557e6065d4a0 2024-09-10T10:50:54.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f2480ff9640 1 -- 172.21.15.143:0/508213544 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f247c005310 con 0x557e6065d4a0 2024-09-10T10:50:54.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.783+0000 7f2480ff9640 1 -- 172.21.15.143:0/508213544 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c09bbd0 con 0x557e6065d4a0 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 -- 172.21.15.143:0/508213544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 msgr2=0x557e6065d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6065d880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f247c06a560 tx=0x7f247c05be30 comp rx=0 tx=0).stop 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 -- 172.21.15.143:0/508213544 shutdown_connections 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e60668680 0x557e6066aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60668140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 --2- 172.21.15.143:0/508213544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6065d880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 -- 172.21.15.143:0/508213544 >> 172.21.15.143:0/508213544 conn(0x557e60659a30 msgr2=0x557e60659e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 -- 172.21.15.143:0/508213544 shutdown_connections 2024-09-10T10:50:54.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.784+0000 7f2492db3740 1 -- 172.21.15.143:0/508213544 wait complete. 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.785+0000 7f2492db3740 1 -- start start 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.785+0000 7f2492db3740 1 Processor -- start 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.785+0000 7f2492db3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 0x557e60667fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.785+0000 7f2492db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60665290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2492db3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e60668680 0x557e606657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2492db3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557e6066b250 con 0x557e6065d4a0 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2492db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557e6057e050 con 0x557e6065ddc0 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2492db3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e6066b3d0 con 0x557e60668680 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2492db3740 10 client.0 _ll_register_callbacks cb 0x557e605be160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 0x557e60667fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 0x557e60667fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48348/0 (socket says 172.21.15.143:48348) 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 -- 172.21.15.143:0/4085360815 learned_addr learned my addr 172.21.15.143:0/4085360815 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:54.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2483fff640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60665290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248ad9d640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e60668680 0x557e606657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 -- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e60668680 msgr2=0x557e606657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e60668680 0x557e606657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 -- 172.21.15.143:0/4085360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 msgr2=0x557e60665290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60665290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 -- 172.21.15.143:0/4085360815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e603343c0 con 0x557e6065d4a0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f248a59c640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 0x557e60667fe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f247c0a4310 tx=0x7f247c09b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508634aa73bfd1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.786+0000 7f2471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2471ffb640 1 -- 172.21.15.143:0/4085360815 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c097e70 con 0x557e6065d4a0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2471ffb640 1 -- 172.21.15.143:0/4085360815 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f247c099ed0 con 0x557e6065d4a0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e605d0850 con 0x557e6065d4a0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2471ffb640 1 -- 172.21.15.143:0/4085360815 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f247c09ff80 con 0x557e6065d4a0 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2492db3740 10 client.16701 fetch_fsmap learned FSMap version 34 2024-09-10T10:50:54.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e60676c20 con 0x557e6065d4a0 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2471ffb640 1 -- 172.21.15.143:0/4085360815 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557e605d0850 con 0x557e6065d4a0 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.787+0000 7f2471ffb640 1 -- 172.21.15.143:0/4085360815 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f247c086200 con 0x557e6065d4a0 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2492db3740 10 client.16701 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2492db3740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2492db3740 20 client.16701 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2492db3740 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e60677f70 0x557e6067a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2492db3740 4 client.16701 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e605cad60 con 0x557e60677f70 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2483fff640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e60677f70 0x557e6067a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2483fff640 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e60677f70 0x557e6067a360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234a555c3d2b6817 server_cookie=5b5c515aa654039 in_seq=0 out_seq=0 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.788+0000 7f2471ffb640 10 client.16701 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2471ffb640 1 -- 172.21.15.143:0/4085360815 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e605cad60 con 0x557e60677f70 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2471ffb640 10 client.16701 handle_command_reply: tid=0 2024-09-10T10:50:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 client.16701 shutdown 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 msgr2=0x557e60667fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 0x557e60667fe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f247c0a4310 tx=0x7f247c09b810 comp rx=0 tx=0).stop 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 shutdown_connections 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e60677f70 0x557e6067a360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e60668680 0x557e606657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e6065ddc0 0x557e60665290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 --2- 172.21.15.143:0/4085360815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065d4a0 0x557e60667fe0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 >> 172.21.15.143:0/4085360815 conn(0x557e60659a30 msgr2=0x557e6065f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2471ffb640 0 client.16701 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 shutdown_connections 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.789+0000 7f2492db3740 1 -- 172.21.15.143:0/4085360815 wait complete. 2024-09-10T10:50:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.790+0000 7f2492db3740 20 client.16701 trim_cache size 0 max 16384 2024-09-10T10:50:54.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 Processor -- start 2024-09-10T10:50:54.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 -- start start 2024-09-10T10:50:54.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e60659e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e60658a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e6067cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1ac860 con 0x557e6065ddc0 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1ae3a0 con 0x557e60668680 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2492db3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1acf30 con 0x557e6065d4a0 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e60659e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e60659e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:50690/0 (socket says 172.21.15.143:50690) 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f248a59c640 1 -- 172.21.15.143:0/2712311213 learned_addr learned my addr 172.21.15.143:0/2712311213 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.813+0000 7f2483fff640 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e60658a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248ad9d640 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e6067cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248a59c640 1 -- 172.21.15.143:0/2712311213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 msgr2=0x557e6067cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248a59c640 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e6067cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248a59c640 1 -- 172.21.15.143:0/2712311213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 msgr2=0x557e60658a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248a59c640 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e60658a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248a59c640 1 -- 172.21.15.143:0/2712311213 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e606637e0 con 0x557e6065d4a0 2024-09-10T10:50:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248ad9d640 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e6067cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f248a59c640 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e60659e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f247c09b8d0 tx=0x7f247c00cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d60b731fbb5f76d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f2473fff640 1 -- 172.21.15.143:0/2712311213 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0b9070 con 0x557e6065d4a0 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f2473fff640 1 -- 172.21.15.143:0/2712311213 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f247c09c2d0 con 0x557e6065d4a0 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.814+0000 7f2473fff640 1 -- 172.21.15.143:0/2712311213 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c09bbd0 con 0x557e6065d4a0 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 -- 172.21.15.143:0/2712311213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 msgr2=0x557e60659e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e60659e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f247c09b8d0 tx=0x7f247c00cf20 comp rx=0 tx=0).stop 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 -- 172.21.15.143:0/2712311213 shutdown_connections 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e6067cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e60658a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 --2- 172.21.15.143:0/2712311213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e60659e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 -- 172.21.15.143:0/2712311213 >> 172.21.15.143:0/2712311213 conn(0x557e60659a30 msgr2=0x557e60658370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 -- 172.21.15.143:0/2712311213 shutdown_connections 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.815+0000 7f2492db3740 1 -- 172.21.15.143:0/2712311213 wait complete. 2024-09-10T10:50:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 -- start start 2024-09-10T10:50:54.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 Processor -- start 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6067c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e6067cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e606da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1adcd0 con 0x557e6065ddc0 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1ad600 con 0x557e60668680 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1aa880 con 0x557e6065d4a0 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2492db3740 10 client.0 _ll_register_callbacks cb 0x7f247c09b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2483fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e6067cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f248a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6067c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2483fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e6067cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48372/0 (socket says 172.21.15.143:48372) 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.816+0000 7f2483fff640 1 -- 172.21.15.143:0/1413015395 learned_addr learned my addr 172.21.15.143:0/1413015395 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f248ad9d640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e606da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2483fff640 1 -- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 msgr2=0x557e6067c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2483fff640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6067c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2483fff640 1 -- 172.21.15.143:0/1413015395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 msgr2=0x557e606da8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2483fff640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e606da8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2483fff640 1 -- 172.21.15.143:0/1413015395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e606daf90 con 0x557e6065ddc0 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f248a59c640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6067c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f248ad9d640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e606da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2483fff640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e6067cbe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x557e6065a890 tx=0x7f2474012380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6067d3a7014a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2471ffb640 1 -- 172.21.15.143:0/1413015395 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247402d060 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2471ffb640 1 -- 172.21.15.143:0/1413015395 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2474004050 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.817+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f247c09c2d0 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2471ffb640 1 -- 172.21.15.143:0/1413015395 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2474004250 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2492db3740 10 client.16707 fetch_fsmap learned FSMap version 34 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e606e2390 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2471ffb640 1 -- 172.21.15.143:0/1413015395 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f247c09c2d0 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2471ffb640 1 -- 172.21.15.143:0/1413015395 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2474010530 con 0x557e6065ddc0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2492db3740 10 client.16707 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.818+0000 7f2492db3740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2492db3740 20 client.16707 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2492db3740 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e606e2b40 0x557e606e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2492db3740 4 client.16707 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e60547210 con 0x557e606e2b40 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f248a59c640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e606e2b40 0x557e606e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f248a59c640 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e606e2b40 0x557e606e4f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74946603a3526b7 server_cookie=de895d051e5228d9 in_seq=0 out_seq=0 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2471ffb640 10 client.16707 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2471ffb640 1 -- 172.21.15.143:0/1413015395 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e60547210 con 0x557e606e2b40 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2471ffb640 10 client.16707 handle_command_reply: tid=0 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.819+0000 7f2492db3740 1 client.16707 shutdown 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 msgr2=0x557e6067cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e6067cbe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x557e6065a890 tx=0x7f2474012380 comp rx=0 tx=0).stop 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 shutdown_connections 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e60668680 0x557e606da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557e6065ddc0 0x557e6067cbe0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557e606e2b40 0x557e606e4f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 --2- 172.21.15.143:0/1413015395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e6065d4a0 0x557e6067c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2471ffb640 0 client.16707 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 >> 172.21.15.143:0/1413015395 conn(0x557e60659a30 msgr2=0x557e60658b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 shutdown_connections 2024-09-10T10:50:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 1 -- 172.21.15.143:0/1413015395 wait complete. 2024-09-10T10:50:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.820+0000 7f2492db3740 20 client.16707 trim_cache size 0 max 16384 2024-09-10T10:50:54.826 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:50:54.826 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:50:54.826 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:50:54.826 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f24780a2c00 msgr2=0x7f24780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f24780a2c00 0x7f24780a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2484005e30 tx=0x7f2484005ef0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 msgr2=0x7f248c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c19a560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9f0 tx=0x7f247c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 shutdown_connections 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f24780a2c00 0x7f24780a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c102ef0 0x7f248c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f248c102570 0x7f248c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 --2- 172.21.15.143:0/3047381307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c100e20 0x7f248c19a560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 >> 172.21.15.143:0/3047381307 conn(0x7f248c0fc7d0 msgr2=0x7f248c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 shutdown_connections 2024-09-10T10:50:54.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:54.821+0000 7f24909da640 1 -- 172.21.15.143:0/3047381307 wait complete. 2024-09-10T10:50:55.729 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:50:55.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:50:56.122 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 -- 172.21.15.143:0/4004946741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40481082a0 msgr2=0x7f404810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 --2- 172.21.15.143:0/4004946741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40481082a0 0x7f404810ec40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4040009be0 tx=0x7f404002a6c0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 -- 172.21.15.143:0/4004946741 shutdown_connections 2024-09-10T10:50:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 --2- 172.21.15.143:0/4004946741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40481082a0 0x7f404810ec40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 --2- 172.21.15.143:0/4004946741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 0x7f4048107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 --2- 172.21.15.143:0/4004946741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4048102d30 0x7f4048103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.678+0000 7f404d380640 1 -- 172.21.15.143:0/4004946741 >> 172.21.15.143:0/4004946741 conn(0x7f40480fc7d0 msgr2=0x7f40480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.679+0000 7f404d380640 1 -- 172.21.15.143:0/4004946741 shutdown_connections 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.679+0000 7f404d380640 1 -- 172.21.15.143:0/4004946741 wait complete. 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.679+0000 7f404d380640 1 Processor -- start 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 -- start start 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4048102d30 0x7f40481a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 0x7f40481a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 0x7f404810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4048111f50 con 0x7f40481036e0 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4048111dd0 con 0x7f40481082a0 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.680+0000 7f404d380640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40481120d0 con 0x7f4048102d30 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.681+0000 7f40477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 0x7f404810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.681+0000 7f40477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 0x7f404810de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:35350/0 (socket says 172.21.15.143:35350) 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.681+0000 7f40477fe640 1 -- 172.21.15.143:0/731083594 learned_addr learned my addr 172.21.15.143:0/731083594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.681+0000 7f40467fc640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 0x7f40481a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.681+0000 7f40477fe640 1 -- 172.21.15.143:0/731083594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4048102d30 msgr2=0x7f40481a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f4046ffd640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4048102d30 0x7f40481a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f40477fe640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4048102d30 0x7f40481a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f40477fe640 1 -- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 msgr2=0x7f40481a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f40477fe640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 0x7f40481a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f40477fe640 1 -- 172.21.15.143:0/731083594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404810e590 con 0x7f40481082a0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f40467fc640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 0x7f40481a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f40477fe640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 0x7f404810de40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4040004af0 tx=0x7f4040003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040041020 con 0x7f40481082a0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404810e820 con 0x7f40481082a0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4040037040 con 0x7f40481082a0 2024-09-10T10:50:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040036070 con 0x7f40481082a0 2024-09-10T10:50:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.682+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481ab610 con 0x7f40481082a0 2024-09-10T10:50:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.684+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4048103110 con 0x7f40481082a0 2024-09-10T10:50:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.685+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4040044020 con 0x7f40481082a0 2024-09-10T10:50:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.686+0000 7f402ffff640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f40300a2c00 0x7f40300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.687+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f40400efbf0 con 0x7f40481082a0 2024-09-10T10:50:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.687+0000 7f4046ffd640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f40300a2c00 0x7f40300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.688+0000 7f4046ffd640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f40300a2c00 0x7f40300a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4038009aa0 tx=0x7f4038005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:56.691+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f404004b030 con 0x7f40481082a0 2024-09-10T10:50:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:56 smithi178 ceph-mon[34813]: pgmap v527: 129 pgs: 129 active+clean; 390 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 83 op/s 2024-09-10T10:50:56.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:56 smithi144 ceph-mon[34230]: pgmap v527: 129 pgs: 129 active+clean; 390 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 83 op/s 2024-09-10T10:50:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.029+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f404802d7c0 con 0x7f40481082a0 2024-09-10T10:50:57.031 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:56 smithi143 ceph-mon[31057]: pgmap v527: 129 pgs: 129 active+clean; 390 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 83 op/s 2024-09-10T10:50:57.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.030+0000 7f402ffff640 1 -- 172.21.15.143:0/731083594 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11999 (secure 0 0 0) 0x7f4040031850 con 0x7f40481082a0 2024-09-10T10:50:57.031 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:50:57.032 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":34,"btime":"2024-09-10T10:50:26:619214+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:50:26.304736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:50:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 34 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f40300a2c00 msgr2=0x7f40300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f40300a2c00 0x7f40300a50c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4038009aa0 tx=0x7f4038005f90 comp rx=0 tx=0).stop 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 msgr2=0x7f404810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 0x7f404810de40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4040004af0 tx=0x7f4040003d60 comp rx=0 tx=0).stop 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 shutdown_connections 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f40300a2c00 0x7f40300a50c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40481082a0 0x7f404810de40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40481036e0 0x7f40481a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 --2- 172.21.15.143:0/731083594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4048102d30 0x7f40481a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 >> 172.21.15.143:0/731083594 conn(0x7f40480fc7d0 msgr2=0x7f4048103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:57.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 shutdown_connections 2024-09-10T10:50:57.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:57.034+0000 7f404d380640 1 -- 172.21.15.143:0/731083594 wait complete. 2024-09-10T10:50:57.756 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:57 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/731083594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:57.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:50:57.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:57 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/731083594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:57.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:57 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/731083594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:50:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:50:58.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 -- 172.21.15.143:0/1803619590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 msgr2=0x7feb3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 --2- 172.21.15.143:0/1803619590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 0x7feb3410ea90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009ca0 tx=0x7feb2c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 -- 172.21.15.143:0/1803619590 shutdown_connections 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 --2- 172.21.15.143:0/1803619590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 0x7feb3410ea90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 --2- 172.21.15.143:0/1803619590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb34102790 0x7feb341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 --2- 172.21.15.143:0/1803619590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34101de0 0x7feb341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 -- 172.21.15.143:0/1803619590 >> 172.21.15.143:0/1803619590 conn(0x7feb340fd8c0 msgr2=0x7feb340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.686+0000 7feb393c8640 1 -- 172.21.15.143:0/1803619590 shutdown_connections 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.687+0000 7feb393c8640 1 -- 172.21.15.143:0/1803619590 wait complete. 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.687+0000 7feb393c8640 1 Processor -- start 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 -- start start 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb34101de0 0x7feb340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 0x7feb34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 0x7feb34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb327fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 0x7feb34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb327fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 0x7feb34074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:32854/0 (socket says 172.21.15.143:32854) 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb327fc640 1 -- 172.21.15.143:0/4097426901 learned_addr learned my addr 172.21.15.143:0/4097426901 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feb34111e20 con 0x7feb34106f10 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb34111ca0 con 0x7feb34101de0 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb34111fa0 con 0x7feb34102790 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb32ffd640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb34101de0 0x7feb340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.688+0000 7feb337fe640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 0x7feb34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb327fc640 1 -- 172.21.15.143:0/4097426901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb34101de0 msgr2=0x7feb340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb327fc640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb34101de0 0x7feb340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb327fc640 1 -- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 msgr2=0x7feb34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb327fc640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 0x7feb34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb327fc640 1 -- 172.21.15.143:0/4097426901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb341a6800 con 0x7feb34102790 2024-09-10T10:50:58.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb327fc640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 0x7feb34074a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e9f0 tx=0x7feb2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb1bfff640 1 -- 172.21.15.143:0/4097426901 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2800cd80 con 0x7feb34102790 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341a6af0 con 0x7feb34102790 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb1bfff640 1 -- 172.21.15.143:0/4097426901 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb280108e0 con 0x7feb34102790 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.689+0000 7feb1bfff640 1 -- 172.21.15.143:0/4097426901 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28018850 con 0x7feb34102790 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.690+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a6fc0 con 0x7feb34102790 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.692+0000 7feb1bfff640 1 -- 172.21.15.143:0/4097426901 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7feb28010a90 con 0x7feb34102790 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 Processor -- start 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 -- start start 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d762d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d762da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d7638370 0x55b0d763a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b0d75932f0 con 0x55b0d7638370 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0d753acf0 con 0x55b0d762eb40 2024-09-10T10:50:58.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0d758f2b0 con 0x55b0d762d5d0 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d762da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d762da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:32862/0 (socket says 172.21.15.143:32862) 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb32ffd640 1 -- 172.21.15.143:0/1728717430 learned_addr learned my addr 172.21.15.143:0/1728717430 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.693+0000 7feb327fc640 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d7638370 0x55b0d763a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb1bfff640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb337fe640 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d762d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb32ffd640 1 -- 172.21.15.143:0/1728717430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 msgr2=0x55b0d762d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb32ffd640 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d762d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb32ffd640 1 -- 172.21.15.143:0/1728717430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d7638370 msgr2=0x55b0d763a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb32ffd640 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d7638370 0x55b0d763a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb32ffd640 1 -- 172.21.15.143:0/1728717430 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d751f3a0 con 0x55b0d762d5d0 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb327fc640 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d7638370 0x55b0d763a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:58.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb1bfff640 1 -- 172.21.15.143:0/4097426901 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feb280cf6a0 con 0x7feb34102790 2024-09-10T10:50:58.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb337fe640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb32ffd640 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d762da10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb24009cf0 tx=0x7feb24011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6df2afb1f810987a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.694+0000 7feb197fa640 1 -- 172.21.15.143:0/1728717430 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24004230 con 0x55b0d762d5d0 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.695+0000 7feb197fa640 1 -- 172.21.15.143:0/1728717430 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb240043d0 con 0x55b0d762d5d0 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.695+0000 7feb197fa640 1 -- 172.21.15.143:0/1728717430 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24019850 con 0x55b0d762d5d0 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.695+0000 7feb337fe640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009a60 tx=0x7feb2c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.695+0000 7feb3b7a1740 1 -- 172.21.15.143:0/1728717430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 msgr2=0x55b0d762da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.695+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d762da10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb24009cf0 tx=0x7feb24011810 comp rx=0 tx=0).stop 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 -- 172.21.15.143:0/1728717430 shutdown_connections 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d7638370 0x55b0d763a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d762da10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/1728717430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d762d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 -- 172.21.15.143:0/1728717430 >> 172.21.15.143:0/1728717430 conn(0x55b0d75f6310 msgr2=0x55b0d7629c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 -- 172.21.15.143:0/1728717430 shutdown_connections 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 -- 172.21.15.143:0/1728717430 wait complete. 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 -- start start 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.696+0000 7feb3b7a1740 1 Processor -- start 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7637ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d7634f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d7638370 0x55b0d76354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b0d763b070 con 0x55b0d762d5d0 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0d754e060 con 0x55b0d762eb40 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0d763b1f0 con 0x55b0d7638370 2024-09-10T10:50:58.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb3b7a1740 10 client.0 _ll_register_callbacks cb 0x55b0d72a18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7637ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d7638370 0x55b0d76354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7637ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52692/0 (socket says 172.21.15.143:52692) 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 -- 172.21.15.143:0/3126427954 learned_addr learned my addr 172.21.15.143:0/3126427954 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb327fc640 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d7634f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 -- 172.21.15.143:0/3126427954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d7638370 msgr2=0x55b0d76354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d7638370 0x55b0d76354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 -- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 msgr2=0x55b0d7634f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d7634f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.697+0000 7feb32ffd640 1 -- 172.21.15.143:0/3126427954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d73043c0 con 0x55b0d762d5d0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb32ffd640 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7637ce0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feb24009250 tx=0x7feb24019ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa42d4676a0e954 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:58.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb16ff5640 1 -- 172.21.15.143:0/3126427954 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24019850 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb16ff5640 1 -- 172.21.15.143:0/3126427954 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feb24003800 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb240043d0 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb16ff5640 1 -- 172.21.15.143:0/3126427954 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feb24033050 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb3b7a1740 10 client.16725 fetch_fsmap learned FSMap version 34 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b0d7646a80 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.698+0000 7feb16ff5640 1 -- 172.21.15.143:0/3126427954 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb240043d0 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb16ff5640 1 -- 172.21.15.143:0/3126427954 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7feb240046c0 con 0x55b0d762d5d0 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb3b7a1740 10 client.16725 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb3b7a1740 10 client.16725 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb3b7a1740 20 client.16725 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d7647ee0 0x55b0d764a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb3b7a1740 4 client.16725 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:58.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0d75a0860 con 0x55b0d7647ee0 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb327fc640 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d7647ee0 0x55b0d764a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb16ff5640 10 client.16725 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.699+0000 7feb327fc640 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d7647ee0 0x55b0d764a2d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da907493996cdf77 server_cookie=f67df2bb33a1ebad in_seq=0 out_seq=0 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb16ff5640 1 -- 172.21.15.143:0/3126427954 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0d75a0860 con 0x55b0d7647ee0 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb16ff5640 10 client.16725 handle_command_reply: tid=0 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 client.16725 shutdown 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 msgr2=0x55b0d7637ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7637ce0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feb24009250 tx=0x7feb24019ca0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 shutdown_connections 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d7647ee0 0x55b0d764a2d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d7638370 0x55b0d76354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.700+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d762eb40 0x55b0d7634f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.701+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3126427954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7637ce0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.701+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 >> 172.21.15.143:0/3126427954 conn(0x55b0d75f6310 msgr2=0x55b0d762f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:58.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.701+0000 7feb16ff5640 0 client.16725 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:58.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.701+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 shutdown_connections 2024-09-10T10:50:58.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.701+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3126427954 wait complete. 2024-09-10T10:50:58.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.701+0000 7feb3b7a1740 20 client.16725 trim_cache size 0 max 16384 2024-09-10T10:50:58.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 Processor -- start 2024-09-10T10:50:58.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 -- start start 2024-09-10T10:50:58.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7644310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762eb40 0x55b0d7630120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d7630660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b0e20 con 0x55b0d762d5d0 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b2960 con 0x55b0d7638370 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b14f0 con 0x55b0d762eb40 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7644310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d7630660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7644310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52706/0 (socket says 172.21.15.143:52706) 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb327fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762eb40 0x55b0d7630120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.716+0000 7feb32ffd640 1 -- 172.21.15.143:0/3176788107 learned_addr learned my addr 172.21.15.143:0/3176788107 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb327fc640 1 -- 172.21.15.143:0/3176788107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 msgr2=0x55b0d7630660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb327fc640 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d7630660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb327fc640 1 -- 172.21.15.143:0/3176788107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 msgr2=0x55b0d7644310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb327fc640 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7644310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb327fc640 1 -- 172.21.15.143:0/3176788107 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d7630ba0 con 0x55b0d762eb40 2024-09-10T10:50:58.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb337fe640 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d7630660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:58.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb327fc640 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762eb40 0x55b0d7630120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb28098510 tx=0x7feb280d5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb96af5bb54e46bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:58.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb18ff9640 1 -- 172.21.15.143:0/3176788107 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2809bb40 con 0x55b0d762eb40 2024-09-10T10:50:58.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb18ff9640 1 -- 172.21.15.143:0/3176788107 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2809fef0 con 0x55b0d762eb40 2024-09-10T10:50:58.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.717+0000 7feb18ff9640 1 -- 172.21.15.143:0/3176788107 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2809af90 con 0x55b0d762eb40 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3176788107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762eb40 msgr2=0x55b0d7630120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762eb40 0x55b0d7630120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb28098510 tx=0x7feb280d5530 comp rx=0 tx=0).stop 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3176788107 shutdown_connections 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d7630660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762eb40 0x55b0d7630120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/3176788107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762d5d0 0x55b0d7644310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3176788107 >> 172.21.15.143:0/3176788107 conn(0x55b0d75f6310 msgr2=0x55b0d7634080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3176788107 shutdown_connections 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.718+0000 7feb3b7a1740 1 -- 172.21.15.143:0/3176788107 wait complete. 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 -- start start 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 Processor -- start 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d7640710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 0x55b0d7640c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d76318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b2290 con 0x55b0d762eb40 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b1bc0 con 0x55b0d7638370 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb341aee00 con 0x55b0d762d5d0 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb3b7a1740 10 client.0 _ll_register_callbacks cb 0x7feb280d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb327fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 0x55b0d7640c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb327fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 0x55b0d7640c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52716/0 (socket says 172.21.15.143:52716) 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb327fc640 1 -- 172.21.15.143:0/2244978413 learned_addr learned my addr 172.21.15.143:0/2244978413 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.719+0000 7feb32ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d7640710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb337fe640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d76318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb327fc640 1 -- 172.21.15.143:0/2244978413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 msgr2=0x55b0d7640710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb327fc640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d7640710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb327fc640 1 -- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 msgr2=0x55b0d76318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:58.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb327fc640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d76318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb327fc640 1 -- 172.21.15.143:0/2244978413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d76320d0 con 0x55b0d762eb40 2024-09-10T10:50:58.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb337fe640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d76318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:58.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.720+0000 7feb327fc640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 0x55b0d7640c50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feb28000c00 tx=0x7feb2809c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3603708f5fc3860 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:58.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.721+0000 7feb16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.722+0000 7feb16ff5640 1 -- 172.21.15.143:0/2244978413 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2809bb40 con 0x55b0d762eb40 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.723+0000 7feb16ff5640 1 -- 172.21.15.143:0/2244978413 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feb280981b0 con 0x55b0d762eb40 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.723+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2809fef0 con 0x55b0d762eb40 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.723+0000 7feb16ff5640 1 -- 172.21.15.143:0/2244978413 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feb28083050 con 0x55b0d762eb40 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.723+0000 7feb3b7a1740 10 client.16731 fetch_fsmap learned FSMap version 34 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.723+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b0d76b2540 con 0x55b0d762eb40 2024-09-10T10:50:58.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb16ff5640 1 -- 172.21.15.143:0/2244978413 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2809fef0 con 0x55b0d762eb40 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb16ff5640 1 -- 172.21.15.143:0/2244978413 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7feb280d1c10 con 0x55b0d762eb40 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb3b7a1740 10 client.16731 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb3b7a1740 10 client.16731 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb3b7a1740 20 client.16731 populate_metadata read hostname 'smithi143' 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d76b2c10 0x55b0d76b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb3b7a1740 4 client.16731 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0d759ad70 con 0x55b0d76b2c10 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.724+0000 7feb32ffd640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d76b2c10 0x55b0d76b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.725+0000 7feb32ffd640 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d76b2c10 0x55b0d76b5030 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4fc31f2be783e05 server_cookie=b2c0e38d6035f3f3 in_seq=0 out_seq=0 2024-09-10T10:50:58.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.725+0000 7feb16ff5640 10 client.16731 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:58.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.726+0000 7feb16ff5640 1 -- 172.21.15.143:0/2244978413 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0d759ad70 con 0x55b0d76b2c10 2024-09-10T10:50:58.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.726+0000 7feb16ff5640 10 client.16731 handle_command_reply: tid=0 2024-09-10T10:50:58.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.726+0000 7feb3b7a1740 1 client.16731 shutdown 2024-09-10T10:50:58.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 msgr2=0x55b0d7640c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 0x55b0d7640c50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feb28000c00 tx=0x7feb2809c370 comp rx=0 tx=0).stop 2024-09-10T10:50:58.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 shutdown_connections 2024-09-10T10:50:58.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b0d76b2c10 0x55b0d76b5030 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0d7638370 0x55b0d76318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0d762eb40 0x55b0d7640c50 secure :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feb28000c00 tx=0x7feb2809c370 comp rx=0 tx=0).stop 2024-09-10T10:50:58.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 --2- 172.21.15.143:0/2244978413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0d762d5d0 0x55b0d7640710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 >> 172.21.15.143:0/2244978413 conn(0x55b0d75f6310 msgr2=0x55b0d76284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:58.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.728+0000 7feb16ff5640 0 client.16731 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:50:58.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.729+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 shutdown_connections 2024-09-10T10:50:58.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.730+0000 7feb3b7a1740 1 -- 172.21.15.143:0/2244978413 wait complete. 2024-09-10T10:50:58.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.730+0000 7feb3b7a1740 20 client.16731 trim_cache size 0 max 16384 2024-09-10T10:50:58.732 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.732+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb1c0a2c00 msgr2=0x7feb1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.732+0000 7feb393c8640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009a60 tx=0x7feb2c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.732+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 msgr2=0x7feb34074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.732+0000 7feb393c8640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 0x7feb34074a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e9f0 tx=0x7feb2800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.734+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 shutdown_connections 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.734+0000 7feb393c8640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.734+0000 7feb393c8640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb34106f10 0x7feb34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.734+0000 7feb393c8640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb34102790 0x7feb34074a30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.734+0000 7feb393c8640 1 --2- 172.21.15.143:0/4097426901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb34101de0 0x7feb340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:58.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.734+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 >> 172.21.15.143:0/4097426901 conn(0x7feb340fd8c0 msgr2=0x7feb340fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:58.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.735+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 shutdown_connections 2024-09-10T10:50:58.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:50:58.736+0000 7feb393c8640 1 -- 172.21.15.143:0/4097426901 wait complete. 2024-09-10T10:50:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:50:58 smithi178 ceph-mon[34813]: pgmap v528: 129 pgs: 129 active+clean; 391 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 97 op/s 2024-09-10T10:50:58.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:50:58 smithi144 ceph-mon[34230]: pgmap v528: 129 pgs: 129 active+clean; 391 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 97 op/s 2024-09-10T10:50:59.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:50:58 smithi143 ceph-mon[31057]: pgmap v528: 129 pgs: 129 active+clean; 391 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 97 op/s 2024-09-10T10:50:59.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:50:59.885 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:00.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.449+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/3584305909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc100a70 msgr2=0x7fe8bc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.449+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/3584305909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc100a70 0x7fe8bc100ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000b9f0 tx=0x7fe8b002c0a0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/3584305909 shutdown_connections 2024-09-10T10:51:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/3584305909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc101410 0x7fe8bc10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/3584305909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc100a70 0x7fe8bc100ed0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/3584305909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8bc106a70 0x7fe8bc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/3584305909 >> 172.21.15.143:0/3584305909 conn(0x7fe8bc0fc790 msgr2=0x7fe8bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/3584305909 shutdown_connections 2024-09-10T10:51:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.450+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/3584305909 wait complete. 2024-09-10T10:51:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.451+0000 7fe8c1ff6640 1 Processor -- start 2024-09-10T10:51:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.451+0000 7fe8c1ff6640 1 -- start start 2024-09-10T10:51:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.451+0000 7fe8c1ff6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc100a70 0x7fe8bc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.451+0000 7fe8c1ff6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc101410 0x7fe8bc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8c1ff6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8bc106a70 0x7fe8bc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8c1ff6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc10fc10 con 0x7fe8bc100a70 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8c1ff6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc10fa90 con 0x7fe8bc101410 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc100a70 0x7fe8bc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc100a70 0x7fe8bc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52742/0 (socket says 172.21.15.143:52742) 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/270199172 learned_addr learned my addr 172.21.15.143:0/270199172 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc101410 0x7fe8bc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc10fd90 con 0x7fe8bc106a70 2024-09-10T10:51:00.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8bc106a70 0x7fe8bc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 -- 172.21.15.143:0/270199172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8bc106a70 msgr2=0x7fe8bc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8bc106a70 0x7fe8bc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 -- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc100a70 msgr2=0x7fe8bc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc100a70 0x7fe8bc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 -- 172.21.15.143:0/270199172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc19f540 con 0x7fe8bc101410 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8baffd640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc101410 0x7fe8bc19aad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0037040 tx=0x7fe8b0009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8b8ff9640 1 -- 172.21.15.143:0/270199172 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0039070 con 0x7fe8bc101410 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8b8ff9640 1 -- 172.21.15.143:0/270199172 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b00042c0 con 0x7fe8bc101410 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.452+0000 7fe8b8ff9640 1 -- 172.21.15.143:0/270199172 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0034830 con 0x7fe8bc101410 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.453+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc19f7d0 con 0x7fe8bc101410 2024-09-10T10:51:00.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.453+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc19fc00 con 0x7fe8bc101410 2024-09-10T10:51:00.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.455+0000 7fe8b8ff9640 1 -- 172.21.15.143:0/270199172 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe8b0044060 con 0x7fe8bc101410 2024-09-10T10:51:00.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 Processor -- start 2024-09-10T10:51:00.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 -- start start 2024-09-10T10:51:00.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0405b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0405ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04066370 0x55da04068760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55da03fc12f0 con 0x55da04066370 2024-09-10T10:51:00.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da03f68cf0 con 0x55da0405cb40 2024-09-10T10:51:00.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da03fbd2b0 con 0x55da0405b5d0 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.456+0000 7fe8b8ff9640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe8a40a2c00 0x7fe8a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0405ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8baffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04066370 0x55da04068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0405ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:32926/0 (socket says 172.21.15.143:32926) 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2631666141 learned_addr learned my addr 172.21.15.143:0/2631666141 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0405b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe8a40a2c00 0x7fe8a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8b8ff9640 1 -- 172.21.15.143:0/270199172 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8b00ee2a0 con 0x7fe8bc101410 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2631666141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 msgr2=0x55da0405b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0405b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2631666141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04066370 msgr2=0x55da04068760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04066370 0x55da04068760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2631666141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da03f4d3a0 con 0x55da0405b5d0 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8baffd640 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04066370 0x55da04068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.457+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe8a40a2c00 0x7fe8a40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009b80 tx=0x7fe8b4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.458+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0405ba10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009d00 tx=0x7fe8ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cde882c18250fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.458+0000 7fe8a1ffb640 1 -- 172.21.15.143:0/2631666141 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac004230 con 0x55da0405b5d0 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.458+0000 7fe8a1ffb640 1 -- 172.21.15.143:0/2631666141 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac0043d0 con 0x55da0405b5d0 2024-09-10T10:51:00.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.458+0000 7fe8a1ffb640 1 -- 172.21.15.143:0/2631666141 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac019810 con 0x55da0405b5d0 2024-09-10T10:51:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2631666141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 msgr2=0x55da0405ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0405ba10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009d00 tx=0x7fe8ac011810 comp rx=0 tx=0).stop 2024-09-10T10:51:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2631666141 shutdown_connections 2024-09-10T10:51:00.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04066370 0x55da04068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0405ba10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2631666141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0405b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2631666141 >> 172.21.15.143:0/2631666141 conn(0x55da04024310 msgr2=0x55da04057c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2631666141 shutdown_connections 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2631666141 wait complete. 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.459+0000 7fe8c43cf740 1 -- start start 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 Processor -- start 2024-09-10T10:51:00.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 0x55da04065c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405cb40 0x55da04062f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da04066370 0x55da04063480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55da04069070 con 0x55da0405b5d0 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da03f7c060 con 0x55da04066370 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da040691f0 con 0x55da0405cb40 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 0x55da04065c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8c43cf740 10 client.0 _ll_register_callbacks cb 0x55da03ccf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 0x55da04065c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52758/0 (socket says 172.21.15.143:52758) 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2246235491 learned_addr learned my addr 172.21.15.143:0/2246235491 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8baffd640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405cb40 0x55da04062f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da04066370 0x55da04063480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405cb40 msgr2=0x55da04062f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405cb40 0x55da04062f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2246235491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da04066370 msgr2=0x55da04063480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da04066370 0x55da04063480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/2246235491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da03d323c0 con 0x55da0405b5d0 2024-09-10T10:51:00.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8baffd640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405cb40 0x55da04062f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.460+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da04066370 0x55da04063480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 0x55da04065c90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac005c80 tx=0x7fe8ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58aa2becb5d3d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe89affd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe89affd640 1 -- 172.21.15.143:0/2246235491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac029020 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe89affd640 1 -- 172.21.15.143:0/2246235491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8ac005240 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac0043d0 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe89affd640 1 -- 172.21.15.143:0/2246235491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8ac034050 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe8c43cf740 10 client.16743 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.461+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da04074a30 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe89affd640 1 -- 172.21.15.143:0/2246235491 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac009920 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe89affd640 1 -- 172.21.15.143:0/2246235491 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fe8ac004c90 con 0x55da0405b5d0 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8c43cf740 10 client.16743 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8c43cf740 10 client.16743 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:51:00.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8c43cf740 20 client.16743 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04075e90 0x55da04078280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8c43cf740 4 client.16743 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe8ac0043d0 con 0x55da04075e90 2024-09-10T10:51:00.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.462+0000 7fe8baffd640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04075e90 0x55da04078280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.463+0000 7fe89affd640 10 client.16743 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.463+0000 7fe8baffd640 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04075e90 0x55da04078280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e1150b48d4f5a69 server_cookie=8566ea8a39c8069 in_seq=0 out_seq=0 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.463+0000 7fe89affd640 1 -- 172.21.15.143:0/2246235491 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe8ac0043d0 con 0x55da04075e90 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.463+0000 7fe89affd640 10 client.16743 handle_command_reply: tid=0 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.463+0000 7fe8c43cf740 1 client.16743 shutdown 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 msgr2=0x55da04065c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 0x55da04065c90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac005c80 tx=0x7fe8ac0038a0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 shutdown_connections 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04075e90 0x55da04078280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da04066370 0x55da04063480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405cb40 0x55da04062f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/2246235491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405b5d0 0x55da04065c90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe89affd640 0 client.16743 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 >> 172.21.15.143:0/2246235491 conn(0x55da04024310 msgr2=0x55da0405d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 shutdown_connections 2024-09-10T10:51:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 1 -- 172.21.15.143:0/2246235491 wait complete. 2024-09-10T10:51:00.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.464+0000 7fe8c43cf740 20 client.16743 trim_cache size 0 max 16384 2024-09-10T10:51:00.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.485+0000 7fe8c43cf740 1 Processor -- start 2024-09-10T10:51:00.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 -- start start 2024-09-10T10:51:00.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da04024310 0x55da04062820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405b5d0 0x55da04067760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405cb40 0x55da04067ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b0d90 con 0x55da0405cb40 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b28d0 con 0x55da0405b5d0 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b1460 con 0x55da04024310 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405cb40 0x55da04067ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da04024310 0x55da04062820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da04024310 0x55da04062820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:32956/0 (socket says 172.21.15.143:32956) 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8bb7fe640 1 -- 172.21.15.143:0/1366138251 learned_addr learned my addr 172.21.15.143:0/1366138251 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8baffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405b5d0 0x55da04067760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405cb40 0x55da04067ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52774/0 (socket says 172.21.15.143:52774) 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8baffd640 1 -- 172.21.15.143:0/1366138251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da04024310 msgr2=0x55da04062820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8baffd640 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da04024310 0x55da04062820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8baffd640 1 -- 172.21.15.143:0/1366138251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405cb40 msgr2=0x55da04067ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8baffd640 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405cb40 0x55da04067ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.486+0000 7fe8baffd640 1 -- 172.21.15.143:0/1366138251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da04058960 con 0x55da0405b5d0 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.487+0000 7fe8baffd640 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405b5d0 0x55da04067760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ebdb0 tx=0x7fe8b002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f925b830d22c22dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.487+0000 7fe8a17fa640 1 -- 172.21.15.143:0/1366138251 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00ba190 con 0x55da0405b5d0 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.487+0000 7fe8a17fa640 1 -- 172.21.15.143:0/1366138251 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b00baf30 con 0x55da0405b5d0 2024-09-10T10:51:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.487+0000 7fe8a17fa640 1 -- 172.21.15.143:0/1366138251 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00b88d0 con 0x55da0405b5d0 2024-09-10T10:51:00.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 -- 172.21.15.143:0/1366138251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405b5d0 msgr2=0x55da04067760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405b5d0 0x55da04067760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ebdb0 tx=0x7fe8b002e040 comp rx=0 tx=0).stop 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 -- 172.21.15.143:0/1366138251 shutdown_connections 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da0405cb40 0x55da04067ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405b5d0 0x55da04067760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/1366138251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da04024310 0x55da04062820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 -- 172.21.15.143:0/1366138251 >> 172.21.15.143:0/1366138251 conn(0x55da040635e0 msgr2=0x55da04056310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 -- 172.21.15.143:0/1366138251 shutdown_connections 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 -- 172.21.15.143:0/1366138251 wait complete. 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 -- start start 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.488+0000 7fe8c43cf740 1 Processor -- start 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04024310 0x55da0407ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0407cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0407d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b2200 con 0x55da04024310 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b1b30 con 0x55da0405cb40 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1aed70 con 0x55da0405b5d0 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8c43cf740 10 client.0 _ll_register_callbacks cb 0x7fe8b0005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04024310 0x55da0407ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0407d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0407d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:35442/0 (socket says 172.21.15.143:35442) 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8bbfff640 1 -- 172.21.15.143:0/3766172977 learned_addr learned my addr 172.21.15.143:0/3766172977 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.489+0000 7fe8baffd640 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0407cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8bbfff640 1 -- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 msgr2=0x55da0407cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0407cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8bbfff640 1 -- 172.21.15.143:0/3766172977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04024310 msgr2=0x55da0407ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04024310 0x55da0407ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8bbfff640 1 -- 172.21.15.143:0/3766172977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0406d6f0 con 0x55da0405cb40 2024-09-10T10:51:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8bbfff640 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0407d4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4012f90 tx=0x7fe8b4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14becb582135265e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe89affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe89affd640 1 -- 172.21.15.143:0/3766172977 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4030060 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe89affd640 1 -- 172.21.15.143:0/3766172977 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe8b4004440 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.490+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b00baf30 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe89affd640 1 -- 172.21.15.143:0/3766172977 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8b4037050 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 10 client.26099 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0406ea50 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe89affd640 1 -- 172.21.15.143:0/3766172977 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b00baf30 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe89affd640 1 -- 172.21.15.143:0/3766172977 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fe8b401d040 con 0x55da0405cb40 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 10 client.26099 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 10 client.26099 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 20 client.26099 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04066370 0x55da040e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:00.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 4 client.26099 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.491+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da03fce860 con 0x55da04066370 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04066370 0x55da040e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe8bb7fe640 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04066370 0x55da040e2a20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a7a5cdd84e4efcf server_cookie=24b923135a778575 in_seq=0 out_seq=0 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe89affd640 10 client.26099 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe89affd640 1 -- 172.21.15.143:0/3766172977 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da03fce860 con 0x55da04066370 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe89affd640 10 client.26099 handle_command_reply: tid=0 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe8c43cf740 1 client.26099 shutdown 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 msgr2=0x55da0407d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.492+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0407d4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4012f90 tx=0x7fe8b4004ab0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 shutdown_connections 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55da04066370 0x55da040e2a20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe89affd640 0 client.26099 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da0405cb40 0x55da0407d4b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da0405b5d0 0x55da0407cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 --2- 172.21.15.143:0/3766172977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55da04024310 0x55da0407ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 >> 172.21.15.143:0/3766172977 conn(0x55da040635e0 msgr2=0x55da04056b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 shutdown_connections 2024-09-10T10:51:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 1 -- 172.21.15.143:0/3766172977 wait complete. 2024-09-10T10:51:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.493+0000 7fe8c43cf740 20 client.26099 trim_cache size 0 max 16384 2024-09-10T10:51:00.497 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe8a40a2c00 msgr2=0x7fe8a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe8a40a2c00 0x7fe8a40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009b80 tx=0x7fe8b4009500 comp rx=0 tx=0).stop 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc101410 msgr2=0x7fe8bc19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc101410 0x7fe8bc19aad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0037040 tx=0x7fe8b0009a60 comp rx=0 tx=0).stop 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 shutdown_connections 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe8a40a2c00 0x7fe8a40a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8bc106a70 0x7fe8bc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8bc101410 0x7fe8bc19aad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 --2- 172.21.15.143:0/270199172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8bc100a70 0x7fe8bc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 >> 172.21.15.143:0/270199172 conn(0x7fe8bc0fc790 msgr2=0x7fe8bc0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 shutdown_connections 2024-09-10T10:51:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:00.494+0000 7fe8c1ff6640 1 -- 172.21.15.143:0/270199172 wait complete. 2024-09-10T10:51:00.881 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:00 smithi143 ceph-mon[31057]: pgmap v529: 129 pgs: 129 active+clean; 391 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 102 op/s 2024-09-10T10:51:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:00 smithi178 ceph-mon[34813]: pgmap v529: 129 pgs: 129 active+clean; 391 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 102 op/s 2024-09-10T10:51:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:00 smithi144 ceph-mon[34230]: pgmap v529: 129 pgs: 129 active+clean; 391 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 102 op/s 2024-09-10T10:51:01.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:51:01.633 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:01 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:51:01.724 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:01.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:01 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:51:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:01 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:51:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 -- 172.21.15.143:0/2509283238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560100a80 msgr2=0x7f9560100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2509283238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560100a80 0x7f9560100ee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9554009930 tx=0x7f955402a420 comp rx=0 tx=0).stop 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 -- 172.21.15.143:0/2509283238 shutdown_connections 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2509283238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560101420 0x7f956010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2509283238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560100a80 0x7f9560100ee0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2509283238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560106a80 0x7f9560106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 -- 172.21.15.143:0/2509283238 >> 172.21.15.143:0/2509283238 conn(0x7f95600fc7c0 msgr2=0x7f95600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 -- 172.21.15.143:0/2509283238 shutdown_connections 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.251+0000 7f9565ee6640 1 -- 172.21.15.143:0/2509283238 wait complete. 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.252+0000 7f9565ee6640 1 Processor -- start 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.252+0000 7f9565ee6640 1 -- start start 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f9565ee6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560100a80 0x7f95601983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f9565ee6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 0x7f95601988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f9565ee6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560106a80 0x7f956019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 0x7f95601988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f9565ee6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110d80 con 0x7f9560101420 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 0x7f95601988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52802/0 (socket says 172.21.15.143:52802) 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 -- 172.21.15.143:0/2182649993 learned_addr learned my addr 172.21.15.143:0/2182649993 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110c00 con 0x7f9560100a80 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110f00 con 0x7f9560106a80 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955ffff640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560106a80 0x7f956019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560100a80 0x7f95601983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 -- 172.21.15.143:0/2182649993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560106a80 msgr2=0x7f956019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560106a80 0x7f956019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 -- 172.21.15.143:0/2182649993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560100a80 msgr2=0x7f95601983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560100a80 0x7f95601983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 -- 172.21.15.143:0/2182649993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956019d350 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560100a80 0x7f95601983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955ffff640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560106a80 0x7f956019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.253+0000 7f955effd640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 0x7f95601988e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9554005ab0 tx=0x7f9554003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.254+0000 7f955cff9640 1 -- 172.21.15.143:0/2182649993 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955403f070 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.254+0000 7f955cff9640 1 -- 172.21.15.143:0/2182649993 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9554031b50 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.254+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956019d5e0 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.254+0000 7f955cff9640 1 -- 172.21.15.143:0/2182649993 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554035030 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.254+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95601a4f20 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.256+0000 7f955cff9640 1 -- 172.21.15.143:0/2182649993 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9554042070 con 0x7f9560101420 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.257+0000 7f955cff9640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95480a2c00 0x7f95480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.257+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95480a2c00 0x7f95480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.258+0000 7f955cff9640 1 -- 172.21.15.143:0/2182649993 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f95540efa50 con 0x7f9560101420 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.258+0000 7f95682bf740 1 Processor -- start 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.258+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95480a2c00 0x7f95480a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9550005e00 tx=0x7f9550005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.258+0000 7f95682bf740 1 -- start start 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.258+0000 7f95682bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x56176170a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f95682bf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x56176170aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f95682bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761717760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x56176170aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x56176170a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x56176170aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52804/0 (socket says 172.21.15.143:52804) 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 -- 172.21.15.143:0/2472805525 learned_addr learned my addr 172.21.15.143:0/2472805525 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f95682bf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5617616702f0 con 0x56176170a5d0 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561761617cf0 con 0x56176170bb40 2024-09-10T10:51:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56176166c2b0 con 0x561761715370 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955ffff640 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761717760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 -- 172.21.15.143:0/2472805525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 msgr2=0x561761717760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761717760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 -- 172.21.15.143:0/2472805525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 msgr2=0x56176170a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x56176170a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 -- 172.21.15.143:0/2472805525 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617615fc3a0 con 0x56176170a5d0 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x56176170a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f955effd640 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x56176170aa10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f95540b8920 tx=0x7f9554009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5306ac1769ff210e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.259+0000 7f9545ffb640 1 -- 172.21.15.143:0/2472805525 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554049050 con 0x56176170a5d0 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.260+0000 7f9545ffb640 1 -- 172.21.15.143:0/2472805525 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95540c0300 con 0x56176170a5d0 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.260+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 msgr2=0x56176170aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.260+0000 7f95682bf740 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x56176170aa10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f95540b8920 tx=0x7f9554009260 comp rx=0 tx=0).stop 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 shutdown_connections 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761717760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x56176170aa10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 --2- 172.21.15.143:0/2472805525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x56176170a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 >> 172.21.15.143:0/2472805525 conn(0x5617616d3310 msgr2=0x561761706c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 shutdown_connections 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 -- 172.21.15.143:0/2472805525 wait complete. 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.261+0000 7f95682bf740 1 -- start start 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 Processor -- start 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x561761714c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x561761711f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761712480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f955f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x561761714c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f955f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x561761714c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52816/0 (socket says 172.21.15.143:52816) 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f955f7fe640 1 -- 172.21.15.143:0/2478294993 learned_addr learned my addr 172.21.15.143:0/2478294993 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561761717e30 con 0x56176170a5d0 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56176162b060 con 0x56176170bb40 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f955effd640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x561761711f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.262+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561761717fb0 con 0x561761715370 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955ffff640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761712480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f95682bf740 10 client.0 _ll_register_callbacks cb 0x56176137e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955f7fe640 1 -- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 msgr2=0x561761712480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761712480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955f7fe640 1 -- 172.21.15.143:0/2478294993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 msgr2=0x561761711f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x561761711f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955f7fe640 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617613e13c0 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f955f7fe640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x561761714c90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f95500116c0 tx=0x7f9550004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d21ca23d6358c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f953effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f953effd640 1 -- 172.21.15.143:0/2478294993 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955001f900 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f953effd640 1 -- 172.21.15.143:0/2478294993 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f955001faa0 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.263+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95540c0300 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f953effd640 1 -- 172.21.15.143:0/2478294993 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9550031050 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 10 client.16761 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561761723890 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f953effd640 1 -- 172.21.15.143:0/2478294993 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95540c0300 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f953effd640 1 -- 172.21.15.143:0/2478294993 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f95500103d0 con 0x56176170a5d0 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 10 client.16761 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 10 client.16761 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 20 client.16761 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:02.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761724d50 0x561761727160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 4 client.16761 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.264+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176167d860 con 0x561761724d50 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.265+0000 7f955effd640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761724d50 0x561761727160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.265+0000 7f955effd640 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761724d50 0x561761727160 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=708b12c7b2d13fda server_cookie=8ba4eac141cef992 in_seq=0 out_seq=0 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.265+0000 7f953effd640 10 client.16761 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f953effd640 1 -- 172.21.15.143:0/2478294993 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56176167d860 con 0x561761724d50 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f953effd640 10 client.16761 handle_command_reply: tid=0 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 client.16761 shutdown 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 msgr2=0x561761714c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x561761714c90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f95500116c0 tx=0x7f9550004730 comp rx=0 tx=0).stop 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 shutdown_connections 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761724d50 0x561761727160 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561761715370 0x561761712480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f953effd640 0 client.16761 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170bb40 0x561761711f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 --2- 172.21.15.143:0/2478294993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170a5d0 0x561761714c90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 >> 172.21.15.143:0/2478294993 conn(0x5617616d3310 msgr2=0x56176170c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 shutdown_connections 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 1 -- 172.21.15.143:0/2478294993 wait complete. 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.266+0000 7f95682bf740 20 client.16761 trim_cache size 0 max 16384 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 Processor -- start 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 -- start start 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170a5d0 0x561761721130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 0x56176170c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561761715370 0x56176172ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95601aeb60 con 0x561761715370 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95601b06a0 con 0x56176170a5d0 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f95682bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95601af230 con 0x56176170bb40 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f955ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561761715370 0x56176172ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f955effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 0x56176170c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f955effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 0x56176170c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33018/0 (socket says 172.21.15.143:33018) 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f955effd640 1 -- 172.21.15.143:0/1819964020 learned_addr learned my addr 172.21.15.143:0/1819964020 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.286+0000 7f955f7fe640 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170a5d0 0x561761721130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955effd640 1 -- 172.21.15.143:0/1819964020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170a5d0 msgr2=0x561761721130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955effd640 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170a5d0 0x561761721130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955effd640 1 -- 172.21.15.143:0/1819964020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561761715370 msgr2=0x56176172ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955effd640 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561761715370 0x56176172ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955effd640 1 -- 172.21.15.143:0/1819964020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561761706ed0 con 0x56176170bb40 2024-09-10T10:51:02.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955ffff640 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561761715370 0x56176172ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f955effd640 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 0x56176170c880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95540bbee0 tx=0x7f955402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c0724ca81e60980 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f95457fa640 1 -- 172.21.15.143:0/1819964020 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554049040 con 0x56176170bb40 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f95457fa640 1 -- 172.21.15.143:0/1819964020 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95540c0300 con 0x56176170bb40 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.287+0000 7f95457fa640 1 -- 172.21.15.143:0/1819964020 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540c4950 con 0x56176170bb40 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 -- 172.21.15.143:0/1819964020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 msgr2=0x56176170c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 0x56176170c880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95540bbee0 tx=0x7f955402a420 comp rx=0 tx=0).stop 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 -- 172.21.15.143:0/1819964020 shutdown_connections 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561761715370 0x56176172ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170bb40 0x56176170c880 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 --2- 172.21.15.143:0/1819964020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56176170a5d0 0x561761721130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 -- 172.21.15.143:0/1819964020 >> 172.21.15.143:0/1819964020 conn(0x5617616d3310 msgr2=0x56176170c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 -- 172.21.15.143:0/1819964020 shutdown_connections 2024-09-10T10:51:02.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.288+0000 7f95682bf740 1 -- 172.21.15.143:0/1819964020 wait complete. 2024-09-10T10:51:02.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.289+0000 7f95682bf740 1 -- start start 2024-09-10T10:51:02.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.289+0000 7f95682bf740 1 Processor -- start 2024-09-10T10:51:02.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.289+0000 7f95682bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170a5d0 0x56176171db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.289+0000 7f95682bf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170bb40 0x561761718e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.289+0000 7f95682bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561761715370 0x561761719610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.289+0000 7f95682bf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95601affd0 con 0x56176170bb40 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f95682bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95601af900 con 0x561761715370 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f95682bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95601acb40 con 0x56176170a5d0 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f95682bf740 10 client.0 _ll_register_callbacks cb 0x7f955000fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170bb40 0x561761718e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561761715370 0x561761719610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170bb40 0x561761718e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52820/0 (socket says 172.21.15.143:52820) 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955effd640 1 -- 172.21.15.143:0/819024630 learned_addr learned my addr 172.21.15.143:0/819024630 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955f7fe640 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170a5d0 0x56176171db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 -- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170a5d0 msgr2=0x56176171db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170a5d0 0x56176171db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 -- 172.21.15.143:0/819024630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170bb40 msgr2=0x561761718e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170bb40 0x561761718e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 -- 172.21.15.143:0/819024630 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561761797210 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f955ffff640 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561761715370 0x561761719610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x56176170b070 tx=0x7f95580128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd7ab903324f5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f953effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f953effd640 1 -- 172.21.15.143:0/819024630 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558022020 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95540c0300 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.290+0000 7f953effd640 1 -- 172.21.15.143:0/819024630 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9558004d20 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f953effd640 1 -- 172.21.15.143:0/819024630 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f955802a050 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 10 client.26117 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561761798420 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f953effd640 1 -- 172.21.15.143:0/819024630 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95540c0300 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f953effd640 1 -- 172.21.15.143:0/819024630 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f95580104c0 con 0x561761715370 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 10 client.26117 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:02.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 10 client.26117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 20 client.26117 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761798b70 0x56176179af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 4 client.26117 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561761677d70 con 0x561761798b70 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.291+0000 7f955f7fe640 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761798b70 0x56176179af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f953effd640 10 client.26117 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f955f7fe640 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761798b70 0x56176179af90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afc994cbf76c2879 server_cookie=d88298f7ea0a8e43 in_seq=0 out_seq=0 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f953effd640 1 -- 172.21.15.143:0/819024630 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561761677d70 con 0x561761798b70 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f953effd640 10 client.26117 handle_command_reply: tid=0 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f95682bf740 1 client.26117 shutdown 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561761715370 msgr2=0x561761719610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f95682bf740 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561761715370 0x561761719610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x56176170b070 tx=0x7f95580128e0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 shutdown_connections 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f95682bf740 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x561761798b70 0x56176179af90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.292+0000 7f95682bf740 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561761715370 0x561761719610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f953effd640 0 client.26117 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f95682bf740 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56176170bb40 0x561761718e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f95682bf740 1 --2- 172.21.15.143:0/819024630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56176170a5d0 0x56176171db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 >> 172.21.15.143:0/819024630 conn(0x5617616d3310 msgr2=0x56176170c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 shutdown_connections 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f95682bf740 1 -- 172.21.15.143:0/819024630 wait complete. 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f95682bf740 20 client.26117 trim_cache size 0 max 16384 2024-09-10T10:51:02.308 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95480a2c00 msgr2=0x7f95480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95480a2c00 0x7f95480a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9550005e00 tx=0x7f9550005c70 comp rx=0 tx=0).stop 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 msgr2=0x7f95601988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.293+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 0x7f95601988e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9554005ab0 tx=0x7f9554003b90 comp rx=0 tx=0).stop 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 shutdown_connections 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f95480a2c00 0x7f95480a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9560106a80 0x7f956019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9560101420 0x7f95601988e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 --2- 172.21.15.143:0/2182649993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560100a80 0x7f95601983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 >> 172.21.15.143:0/2182649993 conn(0x7f95600fc7c0 msgr2=0x7f956006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 shutdown_connections 2024-09-10T10:51:02.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:02.294+0000 7f9565ee6640 1 -- 172.21.15.143:0/2182649993 wait complete. 2024-09-10T10:51:02.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:02 smithi178 ceph-mon[34813]: pgmap v530: 129 pgs: 129 active+clean; 392 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 96 op/s 2024-09-10T10:51:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:51:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:51:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:51:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:02 smithi144 ceph-mon[34230]: pgmap v530: 129 pgs: 129 active+clean; 392 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 96 op/s 2024-09-10T10:51:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:51:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:51:02.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:51:03.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:02 smithi143 ceph-mon[31057]: pgmap v530: 129 pgs: 129 active+clean; 392 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 96 op/s 2024-09-10T10:51:03.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:51:03.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:51:03.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:51:03.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:51:03.445 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:03.592 INFO:tasks.workunit.client.0.smithi143.stdout: 86 10663 109 7298 178 3870 152 2024-09-10T10:51:03.919 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:03 smithi143 ceph-mon[31057]: pgmap v531: 129 pgs: 129 active+clean; 394 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 123 op/s 2024-09-10T10:51:03.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:03 smithi178 ceph-mon[34813]: pgmap v531: 129 pgs: 129 active+clean; 394 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 123 op/s 2024-09-10T10:51:03.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:03 smithi144 ceph-mon[34230]: pgmap v531: 129 pgs: 129 active+clean; 394 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 123 op/s 2024-09-10T10:51:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 -- 172.21.15.143:0/4278988438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 msgr2=0x7f2d14106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/4278988438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d14106e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009890 tx=0x7f2d0c02a360 comp rx=0 tx=0).stop 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 -- 172.21.15.143:0/4278988438 shutdown_connections 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/4278988438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 0x7f2d1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/4278988438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 0x7f2d14100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/4278988438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d14106e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 -- 172.21.15.143:0/4278988438 >> 172.21.15.143:0/4278988438 conn(0x7f2d140fc7a0 msgr2=0x7f2d140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 -- 172.21.15.143:0/4278988438 shutdown_connections 2024-09-10T10:51:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.035+0000 7f2d1aec6640 1 -- 172.21.15.143:0/4278988438 wait complete. 2024-09-10T10:51:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 Processor -- start 2024-09-10T10:51:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 -- start start 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 0x7f2d1419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 0x7f2d1419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d1419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14111e30 con 0x7f2d14101420 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14111cb0 con 0x7f2d14100a80 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d1aec6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14111fb0 con 0x7f2d14106a80 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d19463640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d1419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d19463640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d1419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33044/0 (socket says 172.21.15.143:33044) 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d19463640 1 -- 172.21.15.143:0/2307631613 learned_addr learned my addr 172.21.15.143:0/2307631613 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d18c62640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 0x7f2d1419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.036+0000 7f2d13fff640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 0x7f2d1419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d19463640 1 -- 172.21.15.143:0/2307631613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 msgr2=0x7f2d1419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d19463640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 0x7f2d1419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d19463640 1 -- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 msgr2=0x7f2d1419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d19463640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 0x7f2d1419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d19463640 1 -- 172.21.15.143:0/2307631613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1419f560 con 0x7f2d14106a80 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d18c62640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 0x7f2d1419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d13fff640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 0x7f2d1419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d19463640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d1419ee80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400efc0 tx=0x7f2d0400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d11ffb640 1 -- 172.21.15.143:0/2307631613 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04003970 con 0x7f2d14106a80 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d11ffb640 1 -- 172.21.15.143:0/2307631613 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d04004020 con 0x7f2d14106a80 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d11ffb640 1 -- 172.21.15.143:0/2307631613 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400b940 con 0x7f2d14106a80 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1419f850 con 0x7f2d14106a80 2024-09-10T10:51:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.037+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a7130 con 0x7f2d14106a80 2024-09-10T10:51:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.040+0000 7f2d11ffb640 1 -- 172.21.15.143:0/2307631613 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2d0401d020 con 0x7f2d14106a80 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 Processor -- start 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 -- start start 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d11ffb640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cfc0a2c00 0x7f2cfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d370090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d370a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55573d2d62f0 con 0x55573d37b370 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55573d27dcf0 con 0x55573d3705d0 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55573d2d22b0 con 0x55573d371b40 2024-09-10T10:51:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d19463640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d370090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d370090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33058/0 (socket says 172.21.15.143:33058) 2024-09-10T10:51:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d11ffb640 1 -- 172.21.15.143:0/2307631613 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2d040cf1e0 con 0x7f2d14106a80 2024-09-10T10:51:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.041+0000 7f2d18c62640 1 -- 172.21.15.143:0/3856898139 learned_addr learned my addr 172.21.15.143:0/3856898139 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:04.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d18c62640 1 -- 172.21.15.143:0/3856898139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 msgr2=0x55573d370a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d18c62640 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d370a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d18c62640 1 -- 172.21.15.143:0/3856898139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 msgr2=0x55573d37d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d13fff640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cfc0a2c00 0x7f2cfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d18c62640 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d18c62640 1 -- 172.21.15.143:0/3856898139 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55573d2623a0 con 0x55573d371b40 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.042+0000 7f2d19463640 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.043+0000 7f2d18c62640 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d370090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c035040 tx=0x7f2d0c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74a481c5e62eb79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.043+0000 7f2cfaffd640 1 -- 172.21.15.143:0/3856898139 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037040 con 0x55573d371b40 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.043+0000 7f2cfaffd640 1 -- 172.21.15.143:0/3856898139 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d0c046070 con 0x55573d371b40 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.043+0000 7f2cfaffd640 1 -- 172.21.15.143:0/3856898139 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c030340 con 0x55573d371b40 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3856898139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 msgr2=0x55573d370090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d13fff640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cfc0a2c00 0x7f2cfc0a50c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55573d36dac0 tx=0x7f2d08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d370090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c035040 tx=0x7f2d0c009510 comp rx=0 tx=0).stop 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3856898139 shutdown_connections 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d370a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3856898139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d370090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3856898139 >> 172.21.15.143:0/3856898139 conn(0x55573d339310 msgr2=0x55573d36cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3856898139 shutdown_connections 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3856898139 wait complete. 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.044+0000 7f2d1d29f740 1 -- start start 2024-09-10T10:51:04.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.045+0000 7f2d1d29f740 1 Processor -- start 2024-09-10T10:51:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.045+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d37ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.045+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d378030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.045+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d378640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.045+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55573d37e010 con 0x55573d3705d0 2024-09-10T10:51:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d37ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d37ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52848/0 (socket says 172.21.15.143:52848) 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d18c62640 1 -- 172.21.15.143:0/1558435511 learned_addr learned my addr 172.21.15.143:0/1558435511 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d378030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55573d291060 con 0x55573d37b370 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d19463640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d378640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55573d37e190 con 0x55573d371b40 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d1d29f740 10 client.0 _ll_register_callbacks cb 0x55573cfe48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 -- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 msgr2=0x55573d378640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d378640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 -- 172.21.15.143:0/1558435511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 msgr2=0x55573d37ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d37ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 -- 172.21.15.143:0/1558435511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55573d0473c0 con 0x55573d371b40 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d378030 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d080127a0 tx=0x7f2d08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f579fbefe132714f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.046+0000 7f2cf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2cf3fff640 1 -- 172.21.15.143:0/1558435511 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800fb50 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2cf3fff640 1 -- 172.21.15.143:0/1558435511 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2d0800fcf0 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c046070 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2cf3fff640 1 -- 172.21.15.143:0/1558435511 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2d0801f420 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 10 client.26191 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55573d389b20 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2cf3fff640 1 -- 172.21.15.143:0/1558435511 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d0c046070 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2cf3fff640 1 -- 172.21.15.143:0/1558435511 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2d080276f0 con 0x55573d371b40 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 10 client.26191 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 10 client.26191 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 20 client.26191 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d38afe0 0x55573d38d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 4 client.26191 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.047+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55573d2e3860 con 0x55573d38afe0 2024-09-10T10:51:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.048+0000 7f2d18c62640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d38afe0 0x55573d38d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.048+0000 7f2cf3fff640 10 client.26191 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.048+0000 7f2d18c62640 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d38afe0 0x55573d38d450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c3804241251a7b7 server_cookie=fdd68ef12b2ebcd2 in_seq=0 out_seq=0 2024-09-10T10:51:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2cf3fff640 1 -- 172.21.15.143:0/1558435511 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55573d2e3860 con 0x55573d38afe0 2024-09-10T10:51:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2cf3fff640 10 client.26191 handle_command_reply: tid=0 2024-09-10T10:51:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 client.26191 shutdown 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 msgr2=0x55573d378030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d378030 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d080127a0 tx=0x7f2d08004730 comp rx=0 tx=0).stop 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 shutdown_connections 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d38afe0 0x55573d38d450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2cf3fff640 0 client.26191 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d378640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d378030 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1558435511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d37ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.049+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 >> 172.21.15.143:0/1558435511 conn(0x55573d339310 msgr2=0x55573d372960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.050+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 shutdown_connections 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.050+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1558435511 wait complete. 2024-09-10T10:51:04.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.050+0000 7f2d1d29f740 20 client.26191 trim_cache size 0 max 16384 2024-09-10T10:51:04.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 Processor -- start 2024-09-10T10:51:04.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 -- start start 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d373120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d373660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d392d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ac930 con 0x55573d3705d0 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ae470 con 0x55573d37b370 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.069+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ad000 con 0x55573d371b40 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d373660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d373120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d373660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33072/0 (socket says 172.21.15.143:33072) 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 -- 172.21.15.143:0/1307577787 learned_addr learned my addr 172.21.15.143:0/1307577787 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:04.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d19463640 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d392d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 -- 172.21.15.143:0/1307577787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 msgr2=0x55573d392d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d392d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 -- 172.21.15.143:0/1307577787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 msgr2=0x55573d373120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d373120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 -- 172.21.15.143:0/1307577787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55573d378970 con 0x55573d371b40 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d18c62640 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d373120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.070+0000 7f2d13fff640 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d373660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009a20 tx=0x7f2d08028f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5cbb32049a5c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.071+0000 7f2cfa7fc640 1 -- 172.21.15.143:0/1307577787 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08030030 con 0x55573d371b40 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.071+0000 7f2cfa7fc640 1 -- 172.21.15.143:0/1307577787 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d0802c040 con 0x55573d371b40 2024-09-10T10:51:04.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.071+0000 7f2cfa7fc640 1 -- 172.21.15.143:0/1307577787 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800fa00 con 0x55573d371b40 2024-09-10T10:51:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1307577787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 msgr2=0x55573d373660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d373660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009a20 tx=0x7f2d08028f00 comp rx=0 tx=0).stop 2024-09-10T10:51:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1307577787 shutdown_connections 2024-09-10T10:51:04.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d37b370 0x55573d392d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d373660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/1307577787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d3705d0 0x55573d373120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1307577787 >> 172.21.15.143:0/1307577787 conn(0x55573d339310 msgr2=0x55573d37a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1307577787 shutdown_connections 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 -- 172.21.15.143:0/1307577787 wait complete. 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 -- start start 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.072+0000 7f2d1d29f740 1 Processor -- start 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d3ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d3ef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141adda0 con 0x55573d37b370 2024-09-10T10:51:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ad6d0 con 0x55573d3705d0 2024-09-10T10:51:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141aa910 con 0x55573d371b40 2024-09-10T10:51:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d1d29f740 10 client.0 _ll_register_callbacks cb 0x7f2d08009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d3ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d3ef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:35520/0 (socket says 172.21.15.143:35520) 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 -- 172.21.15.143:0/3914361293 learned_addr learned my addr 172.21.15.143:0/3914361293 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d13fff640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d3ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d19463640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 -- 172.21.15.143:0/3914361293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 msgr2=0x55573d3ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d3ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 -- 172.21.15.143:0/3914361293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 msgr2=0x55573d37f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d18c62640 1 -- 172.21.15.143:0/3914361293 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55573d380100 con 0x55573d3705d0 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.073+0000 7f2d13fff640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d3ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2d18c62640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d3ef050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005b60 tx=0x7f2d0c005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad2e2304ab6ae49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2cf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2cf3fff640 1 -- 172.21.15.143:0/3914361293 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037040 con 0x55573d3705d0 2024-09-10T10:51:04.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0802c040 con 0x55573d3705d0 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2cf3fff640 1 -- 172.21.15.143:0/3914361293 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2d0c030d20 con 0x55573d3705d0 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2cf3fff640 1 -- 172.21.15.143:0/3914361293 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c042ae0 con 0x55573d3705d0 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2d1d29f740 10 client.26132 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55573d3f6850 con 0x55573d3705d0 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.074+0000 7f2cf3fff640 1 -- 172.21.15.143:0/3914361293 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d0802c040 con 0x55573d3705d0 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2cf3fff640 1 -- 172.21.15.143:0/3914361293 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2d0c05e050 con 0x55573d3705d0 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d1d29f740 10 client.26132 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d1d29f740 10 client.26132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d1d29f740 20 client.26132 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d3f6b90 0x55573d3f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d1d29f740 4 client.26132 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55573d2ddd70 con 0x55573d3f6b90 2024-09-10T10:51:04.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.075+0000 7f2d13fff640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d3f6b90 0x55573d3f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2cf3fff640 10 client.26132 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d13fff640 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d3f6b90 0x55573d3f8f80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edf95a47e34ab70d server_cookie=e54b4e19d0bf5d34 in_seq=0 out_seq=0 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2cf3fff640 1 -- 172.21.15.143:0/3914361293 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55573d2ddd70 con 0x55573d3f6b90 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2cf3fff640 10 client.26132 handle_command_reply: tid=0 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 client.26132 shutdown 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 msgr2=0x55573d3ef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d3ef050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005b60 tx=0x7f2d0c005d20 comp rx=0 tx=0).stop 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 shutdown_connections 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55573d37b370 0x55573d37f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55573d371b40 0x55573d3ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55573d3f6b90 0x55573d3f8f80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2cf3fff640 0 client.26132 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 --2- 172.21.15.143:0/3914361293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55573d3705d0 0x55573d3ef050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 >> 172.21.15.143:0/3914361293 conn(0x55573d339310 msgr2=0x55573d379890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 shutdown_connections 2024-09-10T10:51:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 1 -- 172.21.15.143:0/3914361293 wait complete. 2024-09-10T10:51:04.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.076+0000 7f2d1d29f740 20 client.26132 trim_cache size 0 max 16384 2024-09-10T10:51:04.083 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cfc0a2c00 msgr2=0x7f2cfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cfc0a2c00 0x7f2cfc0a50c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55573d36dac0 tx=0x7f2d08005f90 comp rx=0 tx=0).stop 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 msgr2=0x7f2d1419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d1419ee80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400efc0 tx=0x7f2d0400c800 comp rx=0 tx=0).stop 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 shutdown_connections 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2cfc0a2c00 0x7f2cfc0a50c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106a80 0x7f2d1419ee80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d14101420 0x7f2d1419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 --2- 172.21.15.143:0/2307631613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d14100a80 0x7f2d1419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.077+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 >> 172.21.15.143:0/2307631613 conn(0x7f2d140fc7a0 msgr2=0x7f2d1410c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.078+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 shutdown_connections 2024-09-10T10:51:04.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:04.078+0000 7f2d1aec6640 1 -- 172.21.15.143:0/2307631613 wait complete. 2024-09-10T10:51:05.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:51:05.499 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:06.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 -- 172.21.15.143:0/682975607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c106a80 msgr2=0x7fa61c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 --2- 172.21.15.143:0/682975607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c106a80 0x7fa61c106e60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009930 tx=0x7fa60c02a420 comp rx=0 tx=0).stop 2024-09-10T10:51:06.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 -- 172.21.15.143:0/682975607 shutdown_connections 2024-09-10T10:51:06.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 --2- 172.21.15.143:0/682975607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c101420 0x7fa61c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 --2- 172.21.15.143:0/682975607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c100a80 0x7fa61c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 --2- 172.21.15.143:0/682975607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c106a80 0x7fa61c106e60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 -- 172.21.15.143:0/682975607 >> 172.21.15.143:0/682975607 conn(0x7fa61c0fc7a0 msgr2=0x7fa61c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 -- 172.21.15.143:0/682975607 shutdown_connections 2024-09-10T10:51:06.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.042+0000 7fa621564640 1 -- 172.21.15.143:0/682975607 wait complete. 2024-09-10T10:51:06.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.043+0000 7fa621564640 1 Processor -- start 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.043+0000 7fa621564640 1 -- start start 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 0x7fa61c196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c101420 0x7fa61c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c106a80 0x7fa61c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10fc20 con 0x7fa61c100a80 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10faa0 con 0x7fa61c101420 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10fda0 con 0x7fa61c106a80 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 0x7fa61c196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 0x7fa61c196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52886/0 (socket says 172.21.15.143:52886) 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61affd640 1 -- 172.21.15.143:0/144681885 learned_addr learned my addr 172.21.15.143:0/144681885 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c101420 0x7fa61c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c106a80 0x7fa61c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 -- 172.21.15.143:0/144681885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c106a80 msgr2=0x7fa61c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c106a80 0x7fa61c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 -- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 msgr2=0x7fa61c196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 0x7fa61c196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 -- 172.21.15.143:0/144681885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c19b0e0 con 0x7fa61c101420 2024-09-10T10:51:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c106a80 0x7fa61c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61affd640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 0x7fa61c196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c101420 0x7fa61c196670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa61000f770 tx=0x7fa61000fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa603fff640 1 -- 172.21.15.143:0/144681885 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61000eab0 con 0x7fa61c101420 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.044+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c19b3d0 con 0x7fa61c101420 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.045+0000 7fa603fff640 1 -- 172.21.15.143:0/144681885 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa610019070 con 0x7fa61c101420 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.045+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a2cb0 con 0x7fa61c101420 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.045+0000 7fa603fff640 1 -- 172.21.15.143:0/144681885 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610009630 con 0x7fa61c101420 2024-09-10T10:51:06.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.047+0000 7fa603fff640 1 -- 172.21.15.143:0/144681885 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa61000ec50 con 0x7fa61c101420 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.048+0000 7fa603fff640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6040a2c00 0x7fa6040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.048+0000 7fa603fff640 1 -- 172.21.15.143:0/144681885 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6100cf370 con 0x7fa61c101420 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.048+0000 7fa61affd640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6040a2c00 0x7fa6040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.048+0000 7fa62393d740 1 Processor -- start 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.048+0000 7fa62393d740 1 -- start start 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa62393d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e936c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa62393d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936c5d0 0x5626e936ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa62393d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e9381f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa62393d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5626e92d22f0 con 0x5626e936db40 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa62393d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626e9279cf0 con 0x5626e936c5d0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa62393d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5626e92ce2b0 con 0x5626e937fb80 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61affd640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6040a2c00 0x7fa6040a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa60c031440 tx=0x7fa60c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e936c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e936c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52896/0 (socket says 172.21.15.143:52896) 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e9381f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61affd640 1 -- 172.21.15.143:0/3418486997 learned_addr learned my addr 172.21.15.143:0/3418486997 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936c5d0 0x5626e936ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 -- 172.21.15.143:0/3418486997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936c5d0 msgr2=0x5626e936ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936c5d0 0x5626e936ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 -- 172.21.15.143:0/3418486997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 msgr2=0x5626e936c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e936c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 -- 172.21.15.143:0/3418486997 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e925e3a0 con 0x5626e937fb80 2024-09-10T10:51:06.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936c5d0 0x5626e936ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:06.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.049+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e9381f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa614009f10 tx=0x7fa6140107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93ebe9cc7b08da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.050+0000 7fa6017fa640 1 -- 172.21.15.143:0/3418486997 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61401c680 con 0x5626e937fb80 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.050+0000 7fa6017fa640 1 -- 172.21.15.143:0/3418486997 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61401ccd0 con 0x5626e937fb80 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.050+0000 7fa6017fa640 1 -- 172.21.15.143:0/3418486997 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614011dd0 con 0x5626e937fb80 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 -- 172.21.15.143:0/3418486997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 msgr2=0x5626e9381f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e9381f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa614009f10 tx=0x7fa6140107a0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 -- 172.21.15.143:0/3418486997 shutdown_connections 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e9381f70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936c5d0 0x5626e936ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 --2- 172.21.15.143:0/3418486997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e936c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 -- 172.21.15.143:0/3418486997 >> 172.21.15.143:0/3418486997 conn(0x5626e9335310 msgr2=0x5626e9368c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 -- 172.21.15.143:0/3418486997 shutdown_connections 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.051+0000 7fa62393d740 1 -- 172.21.15.143:0/3418486997 wait complete. 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 -- start start 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 Processor -- start 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e937f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e937c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e937cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5626e9382880 con 0x5626e936db40 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626e928d060 con 0x5626e937fb80 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5626e9382a00 con 0x5626e936c5d0 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa61affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e937f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa61affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e937f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:33122/0 (socket says 172.21.15.143:33122) 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e937c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa61affd640 1 -- 172.21.15.143:0/1022326872 learned_addr learned my addr 172.21.15.143:0/1022326872 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.052+0000 7fa62393d740 10 client.0 _ll_register_callbacks cb 0x5626e8fe08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e937cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61a7fc640 1 -- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 msgr2=0x5626e937f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e937f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61a7fc640 1 -- 172.21.15.143:0/1022326872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 msgr2=0x5626e937cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e937cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61a7fc640 1 -- 172.21.15.143:0/1022326872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e90433c0 con 0x5626e936db40 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61affd640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e937f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e937c7d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa610014c30 tx=0x7fa61009bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dd64d17c98adea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa5feff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa5feff5640 1 -- 172.21.15.143:0/1022326872 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6100a4480 con 0x5626e936db40 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa5feff5640 1 -- 172.21.15.143:0/1022326872 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa6100860c0 con 0x5626e936db40 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61401ccd0 con 0x5626e936db40 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.053+0000 7fa5feff5640 1 -- 172.21.15.143:0/1022326872 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa61000b020 con 0x5626e936db40 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 10 client.16797 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5626e938e370 con 0x5626e936db40 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa5feff5640 1 -- 172.21.15.143:0/1022326872 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61401ccd0 con 0x5626e936db40 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa5feff5640 1 -- 172.21.15.143:0/1022326872 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fa61009adf0 con 0x5626e936db40 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 10 client.16797 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 20 client.16797 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e938fd10 0x5626e9392190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 4 client.16797 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e92df860 con 0x5626e938fd10 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.054+0000 7fa61affd640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e938fd10 0x5626e9392190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.055+0000 7fa5feff5640 10 client.16797 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.055+0000 7fa61affd640 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e938fd10 0x5626e9392190 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f818f387fbd8cdef server_cookie=c79ac68de65b6c48 in_seq=0 out_seq=0 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.055+0000 7fa5feff5640 1 -- 172.21.15.143:0/1022326872 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626e92df860 con 0x5626e938fd10 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa5feff5640 10 client.16797 handle_command_reply: tid=0 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 client.16797 shutdown 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 msgr2=0x5626e937c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e937c7d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa610014c30 tx=0x7fa61009bfd0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 shutdown_connections 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e937cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e937c7d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e938fd10 0x5626e9392190 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 --2- 172.21.15.143:0/1022326872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e937f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa5feff5640 0 client.16797 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 >> 172.21.15.143:0/1022326872 conn(0x5626e9335310 msgr2=0x5626e936e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 shutdown_connections 2024-09-10T10:51:06.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 1 -- 172.21.15.143:0/1022326872 wait complete. 2024-09-10T10:51:06.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.056+0000 7fa62393d740 20 client.16797 trim_cache size 0 max 16384 2024-09-10T10:51:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 Processor -- start 2024-09-10T10:51:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 -- start start 2024-09-10T10:51:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936c5d0 0x5626e938bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936db40 0x5626e937b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e937b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa60800faa0 con 0x5626e936c5d0 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936c5d0 0x5626e938bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936c5d0 0x5626e938bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52910/0 (socket says 172.21.15.143:52910) 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61affd640 1 -- 172.21.15.143:0/4139760035 learned_addr learned my addr 172.21.15.143:0/4139760035 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080115e0 con 0x5626e936db40 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa608010170 con 0x5626e937fb80 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e937b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936db40 0x5626e937b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4139760035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 msgr2=0x5626e937b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e937b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4139760035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936c5d0 msgr2=0x5626e938bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936c5d0 0x5626e938bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.076+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4139760035 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e93822e0 con 0x5626e936db40 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.077+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936db40 0x5626e937b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa610099ac0 tx=0x7fa610099d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a321c1a0edd9ce6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.077+0000 7fa600ff9640 1 -- 172.21.15.143:0/4139760035 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6100a4480 con 0x5626e936db40 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.077+0000 7fa600ff9640 1 -- 172.21.15.143:0/4139760035 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61000b1c0 con 0x5626e936db40 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.077+0000 7fa600ff9640 1 -- 172.21.15.143:0/4139760035 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610097ab0 con 0x5626e936db40 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936db40 msgr2=0x5626e937b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936db40 0x5626e937b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa610099ac0 tx=0x7fa610099d20 comp rx=0 tx=0).stop 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 shutdown_connections 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e937fb80 0x5626e937b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e936db40 0x5626e937b0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 --2- 172.21.15.143:0/4139760035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936c5d0 0x5626e938bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 >> 172.21.15.143:0/4139760035 conn(0x5626e9335310 msgr2=0x5626e9363a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 shutdown_connections 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 -- 172.21.15.143:0/4139760035 wait complete. 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 -- start start 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.078+0000 7fa62393d740 1 Processor -- start 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e9388060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e93885a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e93793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e93885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa608010f10 con 0x5626e936db40 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e93885a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52916/0 (socket says 172.21.15.143:52916) 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa608010840 con 0x5626e937fb80 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4271416206 learned_addr learned my addr 172.21.15.143:0/4271416206 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa60800da80 con 0x5626e936c5d0 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa62393d740 10 client.0 _ll_register_callbacks cb 0x7fa61000ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61affd640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e9388060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61b7fe640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e93793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 msgr2=0x5626e9388060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e9388060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4271416206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 msgr2=0x5626e93793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e93793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 -- 172.21.15.143:0/4271416206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e937f890 con 0x5626e936db40 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa61a7fc640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e93885a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa61009b7a0 tx=0x7fa6100cdf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3531fef175ea002c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.079+0000 7fa5feff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa5feff5640 1 -- 172.21.15.143:0/4271416206 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61009b350 con 0x5626e936db40 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa5feff5640 1 -- 172.21.15.143:0/4271416206 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa61009ac60 con 0x5626e936db40 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61000b1c0 con 0x5626e936db40 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa5feff5640 1 -- 172.21.15.143:0/4271416206 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa610097ab0 con 0x5626e936db40 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa62393d740 10 client.16803 fetch_fsmap learned FSMap version 34 2024-09-10T10:51:06.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5626e937f890 con 0x5626e936db40 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa5feff5640 1 -- 172.21.15.143:0/4271416206 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61000b1c0 con 0x5626e936db40 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.080+0000 7fa5feff5640 1 -- 172.21.15.143:0/4271416206 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fa6100a0d50 con 0x5626e936db40 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa62393d740 10 client.16803 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa62393d740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa62393d740 20 client.16803 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa62393d740 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e93fa7a0 0x5626e93fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa62393d740 4 client.16803 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626e92d9d70 con 0x5626e93fa7a0 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa61affd640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e93fa7a0 0x5626e93fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa5feff5640 10 client.16803 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.081+0000 7fa61affd640 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e93fa7a0 0x5626e93fcb90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641757236057c807 server_cookie=1984985bba4b7a6c in_seq=0 out_seq=0 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa5feff5640 1 -- 172.21.15.143:0/4271416206 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626e92d9d70 con 0x5626e93fa7a0 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa5feff5640 10 client.16803 handle_command_reply: tid=0 2024-09-10T10:51:06.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 client.16803 shutdown 2024-09-10T10:51:06.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 msgr2=0x5626e93885a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e93885a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa61009b7a0 tx=0x7fa6100cdf80 comp rx=0 tx=0).stop 2024-09-10T10:51:06.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 shutdown_connections 2024-09-10T10:51:06.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e937fb80 0x5626e93793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5626e936db40 0x5626e93885a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5626e93fa7a0 0x5626e93fcb90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 --2- 172.21.15.143:0/4271416206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626e936c5d0 0x5626e9388060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 >> 172.21.15.143:0/4271416206 conn(0x5626e9335310 msgr2=0x5626e937af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa5feff5640 0 client.16803 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 shutdown_connections 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.082+0000 7fa62393d740 1 -- 172.21.15.143:0/4271416206 wait complete. 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.083+0000 7fa62393d740 20 client.16803 trim_cache size 0 max 16384 2024-09-10T10:51:06.088 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6040a2c00 msgr2=0x7fa6040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6040a2c00 0x7fa6040a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa60c031440 tx=0x7fa60c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c101420 msgr2=0x7fa61c196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c101420 0x7fa61c196670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa61000f770 tx=0x7fa61000fc30 comp rx=0 tx=0).stop 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 shutdown_connections 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa6040a2c00 0x7fa6040a50c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa61c106a80 0x7fa61c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c101420 0x7fa61c196670 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 --2- 172.21.15.143:0/144681885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa61c100a80 0x7fa61c196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 >> 172.21.15.143:0/144681885 conn(0x7fa61c0fc7a0 msgr2=0x7fa61c0fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 shutdown_connections 2024-09-10T10:51:06.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:06.084+0000 7fa621564640 1 -- 172.21.15.143:0/144681885 wait complete. 2024-09-10T10:51:06.923 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:06 smithi143 ceph-mon[31057]: pgmap v532: 129 pgs: 129 active+clean; 396 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 106 op/s 2024-09-10T10:51:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:06 smithi178 ceph-mon[34813]: pgmap v532: 129 pgs: 129 active+clean; 396 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 106 op/s 2024-09-10T10:51:06.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:06 smithi144 ceph-mon[34230]: pgmap v532: 129 pgs: 129 active+clean; 396 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 106 op/s 2024-09-10T10:51:07.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:07 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:51:07.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:07 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:51:07.962 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:51:07.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:51:07.991 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:07 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:51:08.336 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:08.771 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:08 smithi143 ceph-mon[31057]: pgmap v533: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 132 op/s 2024-09-10T10:51:08.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:08 smithi178 ceph-mon[34813]: pgmap v533: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 132 op/s 2024-09-10T10:51:08.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:08 smithi144 ceph-mon[34230]: pgmap v533: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 132 op/s 2024-09-10T10:51:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.042+0000 7f5731109640 1 -- 172.21.15.143:0/1129550644 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728031880 con 0x7f572c101de0 2024-09-10T10:51:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 -- 172.21.15.143:0/1129550644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c101de0 msgr2=0x7f572c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 --2- 172.21.15.143:0/1129550644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c101de0 0x7f572c1021c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5728009900 tx=0x7f572802a410 comp rx=0 tx=0).stop 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 -- 172.21.15.143:0/1129550644 shutdown_connections 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 --2- 172.21.15.143:0/1129550644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c106f10 0x7f572c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 --2- 172.21.15.143:0/1129550644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f572c102790 0x7f572c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 --2- 172.21.15.143:0/1129550644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c101de0 0x7f572c1021c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 -- 172.21.15.143:0/1129550644 >> 172.21.15.143:0/1129550644 conn(0x7f572c0fd8e0 msgr2=0x7f572c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 -- 172.21.15.143:0/1129550644 shutdown_connections 2024-09-10T10:51:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.043+0000 7f573436f640 1 -- 172.21.15.143:0/1129550644 wait complete. 2024-09-10T10:51:09.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.044+0000 7f573436f640 1 Processor -- start 2024-09-10T10:51:09.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.044+0000 7f573436f640 1 -- start start 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573436f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 0x7f572c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573436f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 0x7f572c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573436f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f572c106f10 0x7f572c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573436f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f572c111e30 con 0x7f572c102790 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573436f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f572c111cb0 con 0x7f572c101de0 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573436f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f572c111fb0 con 0x7f572c106f10 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 0x7f572c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573210b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 0x7f572c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 0x7f572c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52430/0 (socket says 172.21.15.143:52430) 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 -- 172.21.15.143:0/1062530632 learned_addr learned my addr 172.21.15.143:0/1062530632 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573210b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 0x7f572c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:47820/0 (socket says 172.21.15.143:47820) 2024-09-10T10:51:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573290c640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f572c106f10 0x7f572c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 -- 172.21.15.143:0/1062530632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f572c106f10 msgr2=0x7f572c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f572c106f10 0x7f572c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 -- 172.21.15.143:0/1062530632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 msgr2=0x7f572c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 0x7f572c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573190a640 1 -- 172.21.15.143:0/1062530632 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572c19d3d0 con 0x7f572c102790 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.045+0000 7f573210b640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 0x7f572c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.046+0000 7f573190a640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 0x7f572c198960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f571c00e9f0 tx=0x7f571c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.046+0000 7f57237fe640 1 -- 172.21.15.143:0/1062530632 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c00cd80 con 0x7f572c102790 2024-09-10T10:51:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.046+0000 7f57237fe640 1 -- 172.21.15.143:0/1062530632 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f571c0108e0 con 0x7f572c102790 2024-09-10T10:51:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.046+0000 7f57237fe640 1 -- 172.21.15.143:0/1062530632 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c018920 con 0x7f572c102790 2024-09-10T10:51:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.046+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572c19d6c0 con 0x7f572c102790 2024-09-10T10:51:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.046+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572c19daf0 con 0x7f572c102790 2024-09-10T10:51:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.049+0000 7f57237fe640 1 -- 172.21.15.143:0/1062530632 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f571c010a90 con 0x7f572c102790 2024-09-10T10:51:09.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.050+0000 7f57237fe640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f57180a7090 0x7f57180a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.050+0000 7f573210b640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f57180a7090 0x7f57180a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.050+0000 7f5736748740 1 Processor -- start 2024-09-10T10:51:09.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.050+0000 7f57237fe640 1 -- 172.21.15.143:0/1062530632 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f571c0cf6e0 con 0x7f572c102790 2024-09-10T10:51:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 -- start start 2024-09-10T10:51:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573210b640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f57180a7090 0x7f57180a9550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5728031440 tx=0x7f5728005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec5ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 0x5599cec70940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5599cebc0280 con 0x5599cec5aa80 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599ceb67b10 con 0x5599cec6e550 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f5736748740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599cebbc240 con 0x5599cec5b3a0 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 0x5599cec70940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573210b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec5ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 0x5599cec70940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:47836/0 (socket says 172.21.15.143:47836) 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 -- 172.21.15.143:0/1034809541 learned_addr learned my addr 172.21.15.143:0/1034809541 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573190a640 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 -- 172.21.15.143:0/1034809541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 msgr2=0x5599cec6e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 -- 172.21.15.143:0/1034809541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 msgr2=0x5599cec5ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec5ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573290c640 1 -- 172.21.15.143:0/1034809541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ceb741f0 con 0x5599cec6e550 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.051+0000 7f573210b640 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec5ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.052+0000 7f573290c640 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 0x5599cec70940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5724009e60 tx=0x7f5724010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e482d64c017e9280 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.052+0000 7f5720dea640 1 -- 172.21.15.143:0/1034809541 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724016070 con 0x5599cec6e550 2024-09-10T10:51:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.052+0000 7f5720dea640 1 -- 172.21.15.143:0/1034809541 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5724004d20 con 0x5599cec6e550 2024-09-10T10:51:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.052+0000 7f5720dea640 1 -- 172.21.15.143:0/1034809541 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240126b0 con 0x5599cec6e550 2024-09-10T10:51:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 -- 172.21.15.143:0/1034809541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 msgr2=0x5599cec70940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 0x5599cec70940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5724009e60 tx=0x7f5724010740 comp rx=0 tx=0).stop 2024-09-10T10:51:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 -- 172.21.15.143:0/1034809541 shutdown_connections 2024-09-10T10:51:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec6e550 0x5599cec70940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 --2- 172.21.15.143:0/1034809541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec5ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 -- 172.21.15.143:0/1034809541 >> 172.21.15.143:0/1034809541 conn(0x5599cec570f0 msgr2=0x5599cec574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 -- 172.21.15.143:0/1034809541 shutdown_connections 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 -- 172.21.15.143:0/1034809541 wait complete. 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 -- start start 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.053+0000 7f5736748740 1 Processor -- start 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5aa80 0x5599cec6b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec6e550 0x5599cec6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:56450/0 (socket says 172.21.15.143:56450) 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 -- 172.21.15.143:0/2034920148 learned_addr learned my addr 172.21.15.143:0/2034920148 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5599cec710c0 con 0x5599cec6e550 2024-09-10T10:51:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599ceb7af10 con 0x5599cec5aa80 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599cec71240 con 0x5599cec5b3a0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573210b640 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5aa80 0x5599cec6b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573290c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec6e550 0x5599cec6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f5736748740 10 client.0 _ll_register_callbacks cb 0x5599ceb43a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 -- 172.21.15.143:0/2034920148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5aa80 msgr2=0x5599cec6b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5aa80 0x5599cec6b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 -- 172.21.15.143:0/2034920148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec6e550 msgr2=0x5599cec6bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec6e550 0x5599cec6bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.054+0000 7f573190a640 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ce9316a0 con 0x5599cec5b3a0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f573190a640 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6b9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f571c0cce20 tx=0x7f571c010ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c17a90c1a5a5cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5715ffb640 1 -- 172.21.15.143:0/2034920148 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c09ff30 con 0x5599cec5b3a0 2024-09-10T10:51:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5715ffb640 1 -- 172.21.15.143:0/2034920148 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f571c0981f0 con 0x5599cec5b3a0 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5724004d20 con 0x5599cec5b3a0 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5715ffb640 1 -- 172.21.15.143:0/2034920148 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f571c0a47f0 con 0x5599cec5b3a0 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5736748740 10 client.26227 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599cec7cd20 con 0x5599cec5b3a0 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.055+0000 7f5715ffb640 1 -- 172.21.15.143:0/2034920148 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f571c09afd0 con 0x5599cec5b3a0 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5715ffb640 1 -- 172.21.15.143:0/2034920148 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f571c018ac0 con 0x5599cec5b3a0 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5736748740 10 client.26227 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5736748740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5736748740 20 client.26227 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5736748740 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cec805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5736748740 4 client.26227 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5724004d20 con 0x5599cec7e130 2024-09-10T10:51:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.056+0000 7f573210b640 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cec805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.057+0000 7f573210b640 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cec805a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f2ae77c76e9153 server_cookie=878c58bd7ffcf5dc in_seq=0 out_seq=0 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.057+0000 7f5715ffb640 10 client.26227 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5715ffb640 1 -- 172.21.15.143:0/2034920148 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5724004d20 con 0x5599cec7e130 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5715ffb640 10 client.26227 handle_command_reply: tid=0 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 client.26227 shutdown 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 msgr2=0x5599cec6b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6b9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f571c0cce20 tx=0x7f571c010ea0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 shutdown_connections 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec6e550 0x5599cec6bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cec805a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec5b3a0 0x5599cec6b9f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5715ffb640 0 client.26227 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 --2- 172.21.15.143:0/2034920148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5aa80 0x5599cec6b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 >> 172.21.15.143:0/2034920148 conn(0x5599cec570f0 msgr2=0x5599cec56270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 shutdown_connections 2024-09-10T10:51:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 1 -- 172.21.15.143:0/2034920148 wait complete. 2024-09-10T10:51:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.058+0000 7f5736748740 20 client.26227 trim_cache size 0 max 16384 2024-09-10T10:51:09.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 Processor -- start 2024-09-10T10:51:09.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 -- start start 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec85e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec68730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ac9b0 con 0x5599cec5aa80 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ae4f0 con 0x5599cec5b3a0 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.080+0000 7f5736748740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ad080 con 0x5599cec6e550 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec68730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573190a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573210b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec85e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec68730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:56454/0 (socket says 172.21.15.143:56454) 2024-09-10T10:51:09.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 -- 172.21.15.143:0/1201504341 learned_addr learned my addr 172.21.15.143:0/1201504341 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:09.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 -- 172.21.15.143:0/1201504341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 msgr2=0x5599cec863b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec863b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 -- 172.21.15.143:0/1201504341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 msgr2=0x5599cec85e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec85e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 -- 172.21.15.143:0/1201504341 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cec51680 con 0x5599cec6e550 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573190a640 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573210b640 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec85e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.081+0000 7f573290c640 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec68730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5724009bf0 tx=0x7f5724009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39131155c162c2b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.082+0000 7f5717fff640 1 -- 172.21.15.143:0/1201504341 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724027070 con 0x5599cec6e550 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.082+0000 7f5717fff640 1 -- 172.21.15.143:0/1201504341 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5724005bd0 con 0x5599cec6e550 2024-09-10T10:51:09.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.082+0000 7f5717fff640 1 -- 172.21.15.143:0/1201504341 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724016040 con 0x5599cec6e550 2024-09-10T10:51:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 -- 172.21.15.143:0/1201504341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 msgr2=0x5599cec68730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec68730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5724009bf0 tx=0x7f5724009fb0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 -- 172.21.15.143:0/1201504341 shutdown_connections 2024-09-10T10:51:09.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec68730 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 --2- 172.21.15.143:0/1201504341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec85e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 -- 172.21.15.143:0/1201504341 >> 172.21.15.143:0/1201504341 conn(0x5599cec570f0 msgr2=0x5599cec6d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 -- 172.21.15.143:0/1201504341 shutdown_connections 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 -- 172.21.15.143:0/1201504341 wait complete. 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.083+0000 7f5736748740 1 -- start start 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 Processor -- start 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec76a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ade20 con 0x5599cec5aa80 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1ad750 con 0x5599cec5b3a0 2024-09-10T10:51:09.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1aa990 con 0x5599cec6e550 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f5736748740 10 client.0 _ll_register_callbacks cb 0x7f571c09b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f573210b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f573190a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec76a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f573210b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52470/0 (socket says 172.21.15.143:52470) 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f573290c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.084+0000 7f573210b640 1 -- 172.21.15.143:0/882183754 learned_addr learned my addr 172.21.15.143:0/882183754 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573190a640 1 -- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 msgr2=0x5599cec734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573190a640 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573190a640 1 -- 172.21.15.143:0/882183754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 msgr2=0x5599cec764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573190a640 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573190a640 1 -- 172.21.15.143:0/882183754 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cec73ae0 con 0x5599cec5b3a0 2024-09-10T10:51:09.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573290c640 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f573190a640 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec76a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f571c00c6a0 tx=0x7f571c097eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901ba5c5cdd49c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5715ffb640 1 -- 172.21.15.143:0/882183754 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0ba4c0 con 0x5599cec5b3a0 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5715ffb640 1 -- 172.21.15.143:0/882183754 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f571c010430 con 0x5599cec5b3a0 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5724005bd0 con 0x5599cec5b3a0 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5715ffb640 1 -- 172.21.15.143:0/882183754 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f571c018cd0 con 0x5599cec5b3a0 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5736748740 10 client.26168 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:09.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599cec73ae0 con 0x5599cec5b3a0 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.085+0000 7f5715ffb640 1 -- 172.21.15.143:0/882183754 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5724005bd0 con 0x5599cec5b3a0 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5715ffb640 1 -- 172.21.15.143:0/882183754 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f571c0978d0 con 0x5599cec5b3a0 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5736748740 10 client.26168 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5736748740 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5736748740 20 client.26168 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5736748740 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cecebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5736748740 4 client.26168 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5599cebcd7f0 con 0x5599cec7e130 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f573210b640 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cecebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f5715ffb640 10 client.26168 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.086+0000 7f573210b640 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cecebbf0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c51ec711bd8644 server_cookie=31df3e36275e5410 in_seq=0 out_seq=0 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5715ffb640 1 -- 172.21.15.143:0/882183754 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5599cebcd7f0 con 0x5599cec7e130 2024-09-10T10:51:09.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5715ffb640 10 client.26168 handle_command_reply: tid=0 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 client.26168 shutdown 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 msgr2=0x5599cec76a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec76a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f571c00c6a0 tx=0x7f571c097eb0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 shutdown_connections 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5599cec7e130 0x5599cecebbf0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599cec6e550 0x5599cec734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599cec5b3a0 0x5599cec76a10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 --2- 172.21.15.143:0/882183754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5599cec5aa80 0x5599cec764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 >> 172.21.15.143:0/882183754 conn(0x5599cec570f0 msgr2=0x5599cec55a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5715ffb640 0 client.26168 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 shutdown_connections 2024-09-10T10:51:09.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.088+0000 7f5736748740 1 -- 172.21.15.143:0/882183754 wait complete. 2024-09-10T10:51:09.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f5736748740 20 client.26168 trim_cache size 0 max 16384 2024-09-10T10:51:09.094 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:51:09.094 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:51:09.094 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "ff3dde76-a4c5-42e7-a462-bb10880586a2", 2024-09-10T10:51:09.094 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:51:09.094 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f57180a7090 msgr2=0x7f57180a9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f57180a7090 0x7f57180a9550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5728031440 tx=0x7f5728005b50 comp rx=0 tx=0).stop 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 msgr2=0x7f572c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 0x7f572c198960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f571c00e9f0 tx=0x7f571c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 shutdown_connections 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f57180a7090 0x7f57180a9550 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f572c106f10 0x7f572c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f572c102790 0x7f572c198960 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 --2- 172.21.15.143:0/1062530632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f572c101de0 0x7f572c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.089+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 >> 172.21.15.143:0/1062530632 conn(0x7f572c0fd8e0 msgr2=0x7f572c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.090+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 shutdown_connections 2024-09-10T10:51:09.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:09.090+0000 7f573436f640 1 -- 172.21.15.143:0/1062530632 wait complete. 2024-09-10T10:51:09.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:09 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:51:09.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:09 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:09 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:51:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:09 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:51:09.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:09 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:51:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:09 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:09 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:51:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:09 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:51:10.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:09 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:51:10.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:09 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:09 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:51:10.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:09 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:51:10.206 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:51:10.584 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:10.888 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.889 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:10 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:51:10.889 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.889 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:10 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:51:10.889 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.889 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:10 smithi143 ceph-mon[31057]: pgmap v534: 129 pgs: 129 active+clean; 407 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-10T10:51:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:10 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:51:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:10 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:51:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:10 smithi178 ceph-mon[34813]: pgmap v534: 129 pgs: 129 active+clean; 407 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-10T10:51:10.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:10 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:51:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:10 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:51:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:10 smithi144 ceph-mon[34230]: pgmap v534: 129 pgs: 129 active+clean; 407 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-10T10:51:11.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 -- 172.21.15.143:0/4018284736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48101440 msgr2=0x7f1c48105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/4018284736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48101440 0x7f1c48105910 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02a470 comp rx=0 tx=0).stop 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 -- 172.21.15.143:0/4018284736 shutdown_connections 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/4018284736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c48105f30 0x7f1c481063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/4018284736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48101440 0x7f1c48105910 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/4018284736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c48100b20 0x7f1c48100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 -- 172.21.15.143:0/4018284736 >> 172.21.15.143:0/4018284736 conn(0x7f1c480fc790 msgr2=0x7f1c480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 -- 172.21.15.143:0/4018284736 shutdown_connections 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.500+0000 7f1c4cfae640 1 -- 172.21.15.143:0/4018284736 wait complete. 2024-09-10T10:51:11.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.501+0000 7f1c4cfae640 1 Processor -- start 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.501+0000 7f1c4cfae640 1 -- start start 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4cfae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c48100b20 0x7f1c481a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4cfae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c48101440 0x7f1c481a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4cfae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 0x7f1c4819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4cfae640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48118400 con 0x7f1c48105f30 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4cfae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48118280 con 0x7f1c48100b20 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4cfae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48118580 con 0x7f1c48101440 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 0x7f1c4819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c45d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c48101440 0x7f1c481a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 0x7f1c4819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52488/0 (socket says 172.21.15.143:52488) 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 -- 172.21.15.143:0/2754328596 learned_addr learned my addr 172.21.15.143:0/2754328596 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:11.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c4659c640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c48100b20 0x7f1c481a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 -- 172.21.15.143:0/2754328596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c48101440 msgr2=0x7f1c481a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c48101440 0x7f1c481a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 -- 172.21.15.143:0/2754328596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c48100b20 msgr2=0x7f1c481a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c48100b20 0x7f1c481a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.502+0000 7f1c46d9d640 1 -- 172.21.15.143:0/2754328596 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4819efd0 con 0x7f1c48105f30 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.503+0000 7f1c46d9d640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 0x7f1c4819e830 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000eaa0 tx=0x7f1c4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.503+0000 7f1c377fe640 1 -- 172.21.15.143:0/2754328596 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000cdb0 con 0x7f1c48105f30 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.503+0000 7f1c377fe640 1 -- 172.21.15.143:0/2754328596 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c40005020 con 0x7f1c48105f30 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.503+0000 7f1c377fe640 1 -- 172.21.15.143:0/2754328596 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400116d0 con 0x7f1c48105f30 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.503+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4819f2c0 con 0x7f1c48105f30 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.503+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481ab510 con 0x7f1c48105f30 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c377fe640 1 -- 172.21.15.143:0/2754328596 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1c400051d0 con 0x7f1c48105f30 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 Processor -- start 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 -- start start 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db87880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56052daed2e0 con 0x56052db92680 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56052da94ce0 con 0x56052db874a0 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.507+0000 7f1c4f387740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56052dae92a0 con 0x56052db87dc0 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db94a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52494/0 (socket says 172.21.15.143:52494) 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 -- 172.21.15.143:0/1241527534 learned_addr learned my addr 172.21.15.143:0/1241527534 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db87880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c4659c640 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c377fe640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c300a2cd0 0x7f1c300a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 -- 172.21.15.143:0/1241527534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 msgr2=0x56052db92140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db92140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 -- 172.21.15.143:0/1241527534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 msgr2=0x56052db87880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db87880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.508+0000 7f1c45d9b640 1 -- 172.21.15.143:0/1241527534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052da79440 con 0x56052db92680 2024-09-10T10:51:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.509+0000 7f1c377fe640 1 -- 172.21.15.143:0/2754328596 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1c400cf300 con 0x7f1c48105f30 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.509+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db94a70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c02a990 tx=0x7f1c3c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb11dc4ed6fcf9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.509+0000 7f1c34ff9640 1 -- 172.21.15.143:0/1241527534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c035030 con 0x56052db92680 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.509+0000 7f1c34ff9640 1 -- 172.21.15.143:0/1241527534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c0318a0 con 0x56052db92680 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.509+0000 7f1c46d9d640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c300a2cd0 0x7f1c300a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.509+0000 7f1c34ff9640 1 -- 172.21.15.143:0/1241527534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c003970 con 0x56052db92680 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 -- 172.21.15.143:0/1241527534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 msgr2=0x56052db94a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db94a70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c02a990 tx=0x7f1c3c005b50 comp rx=0 tx=0).stop 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c46d9d640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c300a2cd0 0x7f1c300a5190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c400ccb40 tx=0x7f1c4000cf50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 -- 172.21.15.143:0/1241527534 shutdown_connections 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db94a70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1241527534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db87880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 -- 172.21.15.143:0/1241527534 >> 172.21.15.143:0/1241527534 conn(0x56052db83a30 msgr2=0x56052db83e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 -- 172.21.15.143:0/1241527534 shutdown_connections 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.510+0000 7f1c4f387740 1 -- 172.21.15.143:0/1241527534 wait complete. 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.511+0000 7f1c4f387740 1 -- start start 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.511+0000 7f1c4f387740 1 Processor -- start 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db92010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db8f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56052db95250 con 0x56052db92680 2024-09-10T10:51:11.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56052daa8050 con 0x56052db874a0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56052db953d0 con 0x56052db87dc0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4f387740 10 client.0 _ll_register_callbacks cb 0x56052dae8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c45d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db92010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db8f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:56506/0 (socket says 172.21.15.143:56506) 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 -- 172.21.15.143:0/3835452725 learned_addr learned my addr 172.21.15.143:0/3835452725 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db8f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 -- 172.21.15.143:0/3835452725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 msgr2=0x56052db92010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db92010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 -- 172.21.15.143:0/3835452725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 msgr2=0x56052db8f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db8f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c4659c640 1 -- 172.21.15.143:0/3835452725 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052d85e3c0 con 0x56052db87dc0 2024-09-10T10:51:11.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.512+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db8f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c4659c640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db8f2c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c38012700 tx=0x7f1c38012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a6d26dcde780f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c2dffb640 1 -- 172.21.15.143:0/3835452725 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38004050 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c2dffb640 1 -- 172.21.15.143:0/3835452725 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1c380041f0 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c3c0318a0 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c2dffb640 1 -- 172.21.15.143:0/3835452725 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1c380043f0 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c4f387740 10 client.26245 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56052dba0ca0 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.513+0000 7f1c2dffb640 1 -- 172.21.15.143:0/3835452725 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c0318a0 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c2dffb640 1 -- 172.21.15.143:0/3835452725 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f1c38003bd0 con 0x56052db87dc0 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c4f387740 10 client.26245 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c4f387740 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c4f387740 20 client.26245 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:11.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dba21b0 0x56052dba45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c4f387740 4 client.26245 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052dafa850 con 0x56052dba21b0 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dba21b0 0x56052dba45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.514+0000 7f1c2dffb640 10 client.26245 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.515+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dba21b0 0x56052dba45a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16eca681b99fd2f server_cookie=c7efe0ea8b7260dc in_seq=0 out_seq=0 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.515+0000 7f1c2dffb640 1 -- 172.21.15.143:0/3835452725 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56052dafa850 con 0x56052dba21b0 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.515+0000 7f1c2dffb640 10 client.26245 handle_command_reply: tid=0 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.515+0000 7f1c4f387740 1 client.26245 shutdown 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 msgr2=0x56052db8f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db8f2c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c38012700 tx=0x7f1c38012bc0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 shutdown_connections 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db92680 0x56052db8f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dba21b0 0x56052dba45a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db87dc0 0x56052db8f2c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3835452725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db92010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 >> 172.21.15.143:0/3835452725 conn(0x56052db83a30 msgr2=0x56052db89710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c2dffb640 0 client.26245 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:11.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 shutdown_connections 2024-09-10T10:51:11.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 1 -- 172.21.15.143:0/3835452725 wait complete. 2024-09-10T10:51:11.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.516+0000 7f1c4f387740 20 client.26245 trim_cache size 0 max 16384 2024-09-10T10:51:11.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 Processor -- start 2024-09-10T10:51:11.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 -- start start 2024-09-10T10:51:11.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db8dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db8a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b50c0 con 0x56052db87dc0 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b6c00 con 0x56052db874a0 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4f387740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b5790 con 0x56052db92680 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c45d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c45d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:47920/0 (socket says 172.21.15.143:47920) 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c45d9b640 1 -- 172.21.15.143:0/3133111350 learned_addr learned my addr 172.21.15.143:0/3133111350 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.545+0000 7f1c4659c640 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db8dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db8a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 -- 172.21.15.143:0/3133111350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 msgr2=0x56052db8a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db8a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 -- 172.21.15.143:0/3133111350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 msgr2=0x56052db8dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db8dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 -- 172.21.15.143:0/3133111350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052db8e2b0 con 0x56052db874a0 2024-09-10T10:51:11.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9e4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c033000 tx=0x7f1c3c041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7fde441bb34dd10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c2ffff640 1 -- 172.21.15.143:0/3133111350 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c053040 con 0x56052db874a0 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c2ffff640 1 -- 172.21.15.143:0/3133111350 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c031e30 con 0x56052db874a0 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.546+0000 7f1c2ffff640 1 -- 172.21.15.143:0/3133111350 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c035040 con 0x56052db874a0 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 -- 172.21.15.143:0/3133111350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 msgr2=0x56052db9e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9e4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c033000 tx=0x7f1c3c041040 comp rx=0 tx=0).stop 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 -- 172.21.15.143:0/3133111350 shutdown_connections 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db8a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db8dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 --2- 172.21.15.143:0/3133111350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9e4b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 -- 172.21.15.143:0/3133111350 >> 172.21.15.143:0/3133111350 conn(0x56052db83a30 msgr2=0x56052db8a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 -- 172.21.15.143:0/3133111350 shutdown_connections 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.547+0000 7f1c4f387740 1 -- 172.21.15.143:0/3133111350 wait complete. 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 -- start start 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 Processor -- start 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db96060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b6530 con 0x56052db87dc0 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b5e60 con 0x56052db874a0 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b30a0 con 0x56052db92680 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4f387740 10 client.0 _ll_register_callbacks cb 0x7f1c38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db96060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db96060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52518/0 (socket says 172.21.15.143:52518) 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.548+0000 7f1c4659c640 1 -- 172.21.15.143:0/1718906614 learned_addr learned my addr 172.21.15.143:0/1718906614 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:11.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4659c640 1 -- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 msgr2=0x56052db965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4659c640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4659c640 1 -- 172.21.15.143:0/1718906614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 msgr2=0x56052db9ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4659c640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4659c640 1 -- 172.21.15.143:0/1718906614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052db92380 con 0x56052db87dc0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4659c640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db96060 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1c38012700 tx=0x7f1c38010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44d3d9a84dd9a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c2dffb640 1 -- 172.21.15.143:0/1718906614 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800fb30 con 0x56052db87dc0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c2dffb640 1 -- 172.21.15.143:0/1718906614 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1c3800fcd0 con 0x56052db87dc0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.549+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c3c031e30 con 0x56052db87dc0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c2dffb640 1 -- 172.21.15.143:0/1718906614 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1c38005030 con 0x56052db87dc0 2024-09-10T10:51:11.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 10 client.16839 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56052dc0cc80 con 0x56052db87dc0 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c2dffb640 1 -- 172.21.15.143:0/1718906614 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c38004590 con 0x56052db87dc0 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c2dffb640 1 -- 172.21.15.143:0/1718906614 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f1c3802a430 con 0x56052db87dc0 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 10 client.16839 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 20 client.16839 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dc0d3e0 0x56052dc0f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 4 client.16839 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:51:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.550+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1c3c031e30 con 0x56052dc0d3e0 2024-09-10T10:51:11.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.555+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dc0d3e0 0x56052dc0f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.555+0000 7f1c45d9b640 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dc0d3e0 0x56052dc0f800 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a21bb2a5087363e server_cookie=d7bde4a288e6ac42 in_seq=0 out_seq=0 2024-09-10T10:51:11.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.555+0000 7f1c2dffb640 10 client.16839 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:11.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c2dffb640 1 -- 172.21.15.143:0/1718906614 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f1c3c031e30 con 0x56052dc0d3e0 2024-09-10T10:51:11.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c2dffb640 10 client.16839 handle_command_reply: tid=0 2024-09-10T10:51:11.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 client.16839 shutdown 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 msgr2=0x56052db96060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db96060 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1c38012700 tx=0x7f1c38010f60 comp rx=0 tx=0).stop 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 shutdown_connections 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56052dc0d3e0 0x56052dc0f800 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56052db92680 0x56052db965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c2dffb640 0 client.16839 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56052db87dc0 0x56052db96060 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 --2- 172.21.15.143:0/1718906614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56052db874a0 0x56052db9ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 >> 172.21.15.143:0/1718906614 conn(0x56052db83a30 msgr2=0x56052db90b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 shutdown_connections 2024-09-10T10:51:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 1 -- 172.21.15.143:0/1718906614 wait complete. 2024-09-10T10:51:11.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.556+0000 7f1c4f387740 20 client.16839 trim_cache size 0 max 16384 2024-09-10T10:51:11.560 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:51:11.560 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:51:11.560 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:51:11.560 INFO:teuthology.orchestra.run.smithi143.stdout: "ff3dde76-a4c5-42e7-a462-bb10880586a2": { 2024-09-10T10:51:11.560 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:51:11.561 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "ff3dde76-a4c5-42e7-a462-bb10880586a2", 2024-09-10T10:51:11.561 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:51:11.561 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:11.561 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:11.561 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:51:11.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.557+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c300a2cd0 msgr2=0x7f1c300a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.557+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c300a2cd0 0x7f1c300a5190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c400ccb40 tx=0x7f1c4000cf50 comp rx=0 tx=0).stop 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.557+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 msgr2=0x7f1c4819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.557+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 0x7f1c4819e830 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000eaa0 tx=0x7f1c4000ef60 comp rx=0 tx=0).stop 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 shutdown_connections 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f1c300a2cd0 0x7f1c300a5190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c48105f30 0x7f1c4819e830 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c48101440 0x7f1c481a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 --2- 172.21.15.143:0/2754328596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c48100b20 0x7f1c481a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 >> 172.21.15.143:0/2754328596 conn(0x7f1c480fc790 msgr2=0x7f1c4810f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 shutdown_connections 2024-09-10T10:51:11.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:11.558+0000 7f1c4cfae640 1 -- 172.21.15.143:0/2754328596 wait complete. 2024-09-10T10:51:11.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:11 smithi178 ceph-mon[34813]: pgmap v535: 129 pgs: 129 active+clean; 404 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-09-10T10:51:11.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:11 smithi144 ceph-mon[34230]: pgmap v535: 129 pgs: 129 active+clean; 404 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-09-10T10:51:11.953 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:11 smithi143 ceph-mon[31057]: pgmap v535: 129 pgs: 129 active+clean; 404 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-09-10T10:51:12.569 INFO:tasks.cephfs.filesystem:scrub status for tag:ff3dde76-a4c5-42e7-a462-bb10880586a2 - {'path': '/', 'tag': 'ff3dde76-a4c5-42e7-a462-bb10880586a2', 'options': 'recursive,force'} 2024-09-10T10:51:13.592 INFO:tasks.workunit.client.0.smithi143.stdout: 88 9017 133 6427 101 4626 135 2024-09-10T10:51:14.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:13 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:14.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:13 smithi143 ceph-mon[31057]: pgmap v536: 129 pgs: 129 active+clean; 401 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 747 op/s 2024-09-10T10:51:14.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:13 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:14.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:13 smithi178 ceph-mon[34813]: pgmap v536: 129 pgs: 129 active+clean; 401 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 747 op/s 2024-09-10T10:51:14.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:13 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:51:14.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:13 smithi144 ceph-mon[34230]: pgmap v536: 129 pgs: 129 active+clean; 401 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 747 op/s 2024-09-10T10:51:16.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:16 smithi178 ceph-mon[34813]: pgmap v537: 129 pgs: 129 active+clean; 402 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 818 op/s 2024-09-10T10:51:16.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:16 smithi144 ceph-mon[34230]: pgmap v537: 129 pgs: 129 active+clean; 402 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 818 op/s 2024-09-10T10:51:17.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:16 smithi143 ceph-mon[31057]: pgmap v537: 129 pgs: 129 active+clean; 402 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 818 op/s 2024-09-10T10:51:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:18 smithi178 ceph-mon[34813]: pgmap v538: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 970 op/s 2024-09-10T10:51:18.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:18 smithi144 ceph-mon[34230]: pgmap v538: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 970 op/s 2024-09-10T10:51:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:18 smithi143 ceph-mon[31057]: pgmap v538: 129 pgs: 129 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 970 op/s 2024-09-10T10:51:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:20 smithi178 ceph-mon[34813]: pgmap v539: 129 pgs: 129 active+clean; 400 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 979 op/s 2024-09-10T10:51:20.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:20 smithi144 ceph-mon[34230]: pgmap v539: 129 pgs: 129 active+clean; 400 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 979 op/s 2024-09-10T10:51:21.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:20 smithi143 ceph-mon[31057]: pgmap v539: 129 pgs: 129 active+clean; 400 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 979 op/s 2024-09-10T10:51:22.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:22 smithi178 ceph-mon[34813]: pgmap v540: 129 pgs: 129 active+clean; 405 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 973 op/s 2024-09-10T10:51:22.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:22 smithi144 ceph-mon[34230]: pgmap v540: 129 pgs: 129 active+clean; 405 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 973 op/s 2024-09-10T10:51:23.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:22 smithi143 ceph-mon[31057]: pgmap v540: 129 pgs: 129 active+clean; 405 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 973 op/s 2024-09-10T10:51:23.592 INFO:tasks.workunit.client.0.smithi143.stdout: 91 11153 140 7970 125 4341 136 2024-09-10T10:51:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:24 smithi178 ceph-mon[34813]: pgmap v541: 129 pgs: 129 active+clean; 412 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 932 op/s 2024-09-10T10:51:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:24 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:51:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:24 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:51:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:24 smithi144 ceph-mon[34230]: pgmap v541: 129 pgs: 129 active+clean; 412 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 932 op/s 2024-09-10T10:51:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:24 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:51:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:24 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:51:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:24 smithi143 ceph-mon[31057]: pgmap v541: 129 pgs: 129 active+clean; 412 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 932 op/s 2024-09-10T10:51:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:24 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:51:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:24 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:51:26.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:26 smithi178 ceph-mon[34813]: pgmap v542: 129 pgs: 129 active+clean; 404 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-10T10:51:26.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:26 smithi144 ceph-mon[34230]: pgmap v542: 129 pgs: 129 active+clean; 404 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-10T10:51:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:26 smithi143 ceph-mon[31057]: pgmap v542: 129 pgs: 129 active+clean; 404 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-10T10:51:28.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:28 smithi178 ceph-mon[34813]: pgmap v543: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-09-10T10:51:28.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:28 smithi144 ceph-mon[34230]: pgmap v543: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-09-10T10:51:29.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:28 smithi143 ceph-mon[31057]: pgmap v543: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 341 op/s 2024-09-10T10:51:30.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:29 smithi143 ceph-mon[31057]: pgmap v544: 129 pgs: 129 active+clean; 414 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-10T10:51:30.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:29 smithi178 ceph-mon[34813]: pgmap v544: 129 pgs: 129 active+clean; 414 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-10T10:51:30.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:29 smithi144 ceph-mon[34230]: pgmap v544: 129 pgs: 129 active+clean; 414 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-10T10:51:32.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:32 smithi178 ceph-mon[34813]: pgmap v545: 129 pgs: 129 active+clean; 416 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-10T10:51:32.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:32 smithi144 ceph-mon[34230]: pgmap v545: 129 pgs: 129 active+clean; 416 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-10T10:51:33.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:32 smithi143 ceph-mon[31057]: pgmap v545: 129 pgs: 129 active+clean; 416 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-10T10:51:33.592 INFO:tasks.workunit.client.0.smithi143.stdout: 93 11205 111 7879 151 4537 136 2024-09-10T10:51:34.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:34 smithi178 ceph-mon[34813]: pgmap v546: 129 pgs: 129 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-10T10:51:34.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:34 smithi144 ceph-mon[34230]: pgmap v546: 129 pgs: 129 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-10T10:51:35.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:34 smithi143 ceph-mon[31057]: pgmap v546: 129 pgs: 129 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-10T10:51:36.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:36 smithi178 ceph-mon[34813]: pgmap v547: 129 pgs: 129 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-10T10:51:36.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:36 smithi144 ceph-mon[34230]: pgmap v547: 129 pgs: 129 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-10T10:51:37.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:36 smithi143 ceph-mon[31057]: pgmap v547: 129 pgs: 129 active+clean; 419 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-10T10:51:38.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:38 smithi178 ceph-mon[34813]: pgmap v548: 129 pgs: 129 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-10T10:51:38.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:38 smithi144 ceph-mon[34230]: pgmap v548: 129 pgs: 129 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-10T10:51:39.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:38 smithi143 ceph-mon[31057]: pgmap v548: 129 pgs: 129 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-10T10:51:40.512 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T10:51:40.512 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T10:51:40.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:51:40.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:40 smithi143 ceph-mon[31057]: pgmap v549: 129 pgs: 129 active+clean; 419 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 222 op/s 2024-09-10T10:51:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:40.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:40 smithi178 ceph-mon[34813]: pgmap v549: 129 pgs: 129 active+clean; 419 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 222 op/s 2024-09-10T10:51:40.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:40 smithi144 ceph-mon[34230]: pgmap v549: 129 pgs: 129 active+clean; 419 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 222 op/s 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 -- 172.21.15.143:0/2843272522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 msgr2=0x7f28b41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 --2- 172.21.15.143:0/2843272522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b41043d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009860 tx=0x7f28a402a340 comp rx=0 tx=0).stop 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 -- 172.21.15.143:0/2843272522 shutdown_connections 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 --2- 172.21.15.143:0/2843272522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28b4109120 0x7f28b4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 --2- 172.21.15.143:0/2843272522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b41049a0 0x7f28b41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 --2- 172.21.15.143:0/2843272522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b41043d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 -- 172.21.15.143:0/2843272522 >> 172.21.15.143:0/2843272522 conn(0x7f28b40ffab0 msgr2=0x7f28b4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 -- 172.21.15.143:0/2843272522 shutdown_connections 2024-09-10T10:51:41.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.466+0000 7f28b923a640 1 -- 172.21.15.143:0/2843272522 wait complete. 2024-09-10T10:51:41.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.468+0000 7f28b923a640 1 Processor -- start 2024-09-10T10:51:41.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.468+0000 7f28b923a640 1 -- start start 2024-09-10T10:51:41.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.468+0000 7f28b923a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b410f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.468+0000 7f28b923a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b41049a0 0x7f28b410fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b923a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28b4109120 0x7f28b41a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b2d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b410f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b2d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b410f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:58934/0 (socket says 172.21.15.143:58934) 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b2d9d640 1 -- 172.21.15.143:0/778815626 learned_addr learned my addr 172.21.15.143:0/778815626 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4077800 con 0x7f28b4109120 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4077680 con 0x7f28b41049a0 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4077980 con 0x7f28b4103ff0 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b359e640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28b4109120 0x7f28b41a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b259c640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b41049a0 0x7f28b410fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b2d9d640 1 -- 172.21.15.143:0/778815626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b41049a0 msgr2=0x7f28b410fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.469+0000 7f28b2d9d640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b41049a0 0x7f28b410fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f28b2d9d640 1 -- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28b4109120 msgr2=0x7f28b41a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f28b2d9d640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28b4109120 0x7f28b41a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f28b2d9d640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a3a50 con 0x7f28b4103ff0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f28b2d9d640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b410f890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28a40303f0 tx=0x7f28a4030e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4037050 con 0x7f28b4103ff0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a3ce0 con 0x7f28b4103ff0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28a4043070 con 0x7f28b4103ff0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4003bf0 con 0x7f28b4103ff0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.470+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41ab5a0 con 0x7f28b4103ff0 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.472+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b41054d0 con 0x7f28b4103ff0 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.472+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f28a4030450 con 0x7f28b4103ff0 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.473+0000 7f289bfff640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f289c0a2c00 0x7f289c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.473+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f28a40ef590 con 0x7f28b4103ff0 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.473+0000 7f28b259c640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f289c0a2c00 0x7f289c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.474+0000 7f28b259c640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f289c0a2c00 0x7f289c0a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f28b4102460 tx=0x7f28a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.477+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28a404c020 con 0x7f28b4103ff0 2024-09-10T10:51:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.794+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b4110bf0 con 0x7f28b4103ff0 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.795+0000 7f289bfff640 1 -- 172.21.15.143:0/778815626 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11998 (secure 0 0 0) 0x7f28a40bc190 con 0x7f28b4103ff0 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"btime":"2024-09-10T10:51:06:617569+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:51:06.305790+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.798+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f289c0a2c00 msgr2=0x7f289c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.798+0000 7f28b923a640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f289c0a2c00 0x7f289c0a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f28b4102460 tx=0x7f28a8005f90 comp rx=0 tx=0).stop 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.798+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 msgr2=0x7f28b410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.798+0000 7f28b923a640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b410f890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28a40303f0 tx=0x7f28a4030e00 comp rx=0 tx=0).stop 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 shutdown_connections 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f289c0a2c00 0x7f289c0a50c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28b4109120 0x7f28b41a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b41049a0 0x7f28b410fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 --2- 172.21.15.143:0/778815626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28b4103ff0 0x7f28b410f890 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 >> 172.21.15.143:0/778815626 conn(0x7f28b40ffab0 msgr2=0x7f28b4100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.799+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 shutdown_connections 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:41.800+0000 7f28b923a640 1 -- 172.21.15.143:0/778815626 wait complete. 2024-09-10T10:51:42.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:51:42.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:42 smithi143 ceph-mon[31057]: pgmap v550: 129 pgs: 129 active+clean; 422 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2024-09-10T10:51:42.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:42 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/778815626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:51:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:42 smithi178 ceph-mon[34813]: pgmap v550: 129 pgs: 129 active+clean; 422 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2024-09-10T10:51:42.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:42 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/778815626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:51:42.930 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T10:51:42.930 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T10:51:42.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:42 smithi144 ceph-mon[34230]: pgmap v550: 129 pgs: 129 active+clean; 422 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2024-09-10T10:51:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:42 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/778815626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:51:43.004 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:43.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 -- 172.21.15.143:0/2088166930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60103ff0 msgr2=0x7fab601043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 --2- 172.21.15.143:0/2088166930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60103ff0 0x7fab601043d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab50009900 tx=0x7fab5002a410 comp rx=0 tx=0).stop 2024-09-10T10:51:43.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 -- 172.21.15.143:0/2088166930 shutdown_connections 2024-09-10T10:51:43.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 --2- 172.21.15.143:0/2088166930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60109120 0x7fab60110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 --2- 172.21.15.143:0/2088166930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 0x7fab601089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 --2- 172.21.15.143:0/2088166930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60103ff0 0x7fab601043d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.558+0000 7fab65f20640 1 -- 172.21.15.143:0/2088166930 >> 172.21.15.143:0/2088166930 conn(0x7fab600ffab0 msgr2=0x7fab60101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.559+0000 7fab65f20640 1 -- 172.21.15.143:0/2088166930 shutdown_connections 2024-09-10T10:51:43.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.559+0000 7fab65f20640 1 -- 172.21.15.143:0/2088166930 wait complete. 2024-09-10T10:51:43.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.559+0000 7fab65f20640 1 Processor -- start 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 -- start start 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60103ff0 0x7fab6019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 0x7fab6019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60109120 0x7fab6019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab60113fd0 con 0x7fab601049a0 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab60113e50 con 0x7fab60103ff0 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab65f20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab60114150 con 0x7fab60109120 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab5effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 0x7fab6019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab5effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 0x7fab6019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39302/0 (socket says 172.21.15.143:39302) 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab5effd640 1 -- 172.21.15.143:0/4194569933 learned_addr learned my addr 172.21.15.143:0/4194569933 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:43.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab5ffff640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60109120 0x7fab6019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.560+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60103ff0 0x7fab6019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5ffff640 1 -- 172.21.15.143:0/4194569933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60103ff0 msgr2=0x7fab6019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5ffff640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60103ff0 0x7fab6019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5ffff640 1 -- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 msgr2=0x7fab6019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5ffff640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 0x7fab6019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5ffff640 1 -- 172.21.15.143:0/4194569933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6019f5e0 con 0x7fab60109120 2024-09-10T10:51:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5ffff640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60109120 0x7fab6019ef00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab5800da10 tx=0x7fab5800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5cff9640 1 -- 172.21.15.143:0/4194569933 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5800bd70 con 0x7fab60109120 2024-09-10T10:51:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab5cff9640 1 -- 172.21.15.143:0/4194569933 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab58003e20 con 0x7fab60109120 2024-09-10T10:51:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.561+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6019f8d0 con 0x7fab60109120 2024-09-10T10:51:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.562+0000 7fab5cff9640 1 -- 172.21.15.143:0/4194569933 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab580106c0 con 0x7fab60109120 2024-09-10T10:51:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.562+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab601a71b0 con 0x7fab60109120 2024-09-10T10:51:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.565+0000 7fab5cff9640 1 -- 172.21.15.143:0/4194569933 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fab58003970 con 0x7fab60109120 2024-09-10T10:51:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.565+0000 7fab682f9740 1 Processor -- start 2024-09-10T10:51:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.565+0000 7fab682f9740 1 -- start start 2024-09-10T10:51:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab682f9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7e94880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab682f9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7e9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab682f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ea1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab682f9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eea7dfa2e0 con 0x55eea7e94dc0 2024-09-10T10:51:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab682f9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eea7da1ce0 con 0x55eea7e944a0 2024-09-10T10:51:43.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab682f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eea7df62a0 con 0x55eea7e9f680 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7e94880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7e94880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:57310/0 (socket says 172.21.15.143:57310) 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 -- 172.21.15.143:0/1032457101 learned_addr learned my addr 172.21.15.143:0/1032457101 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ea1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5cff9640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fab4c0a2d50 0x7fab4c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7e9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 -- 172.21.15.143:0/1032457101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 msgr2=0x55eea7ea1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ea1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 -- 172.21.15.143:0/1032457101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 msgr2=0x55eea7e9f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7e9f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5effd640 1 -- 172.21.15.143:0/1032457101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea7d86440 con 0x55eea7e944a0 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5cff9640 1 -- 172.21.15.143:0/4194569933 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fab580cf650 con 0x7fab60109120 2024-09-10T10:51:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.566+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7e9f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.567+0000 7fab5effd640 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7e94880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab48004650 tx=0x7fab48004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a88edcf954620510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.567+0000 7fab5ffff640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fab4c0a2d50 0x7fab4c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.567+0000 7fab55ffb640 1 -- 172.21.15.143:0/1032457101 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48005020 con 0x55eea7e944a0 2024-09-10T10:51:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.567+0000 7fab55ffb640 1 -- 172.21.15.143:0/1032457101 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab480047a0 con 0x55eea7e944a0 2024-09-10T10:51:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.567+0000 7fab55ffb640 1 -- 172.21.15.143:0/1032457101 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48005020 con 0x55eea7e944a0 2024-09-10T10:51:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 -- 172.21.15.143:0/1032457101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 msgr2=0x55eea7e94880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab5ffff640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fab4c0a2d50 0x7fab4c0a5210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fab5809b7f0 tx=0x7fab58004950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7e94880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab48004650 tx=0x7fab48004730 comp rx=0 tx=0).stop 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 -- 172.21.15.143:0/1032457101 shutdown_connections 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ea1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7e9f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 --2- 172.21.15.143:0/1032457101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7e94880 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 -- 172.21.15.143:0/1032457101 >> 172.21.15.143:0/1032457101 conn(0x55eea7e90a30 msgr2=0x55eea7e90e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 -- 172.21.15.143:0/1032457101 shutdown_connections 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 -- 172.21.15.143:0/1032457101 wait complete. 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.568+0000 7fab682f9740 1 -- start start 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 Processor -- start 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e944a0 0x55eea7e9eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7e9c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e9f680 0x55eea7e9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eea7ea2250 con 0x55eea7e944a0 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eea7db5050 con 0x55eea7e9f680 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eea7ea23d0 con 0x55eea7e94dc0 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab682f9740 10 client.0 _ll_register_callbacks cb 0x55eea7df5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab5effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e944a0 0x55eea7e9eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab5f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7e9c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab5f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7e9c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:58964/0 (socket says 172.21.15.143:58964) 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab5f7fe640 1 -- 172.21.15.143:0/304934995 learned_addr learned my addr 172.21.15.143:0/304934995 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.569+0000 7fab5effd640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e9f680 0x55eea7e9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5f7fe640 1 -- 172.21.15.143:0/304934995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e9f680 msgr2=0x55eea7e9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e9f680 0x55eea7e9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5f7fe640 1 -- 172.21.15.143:0/304934995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e944a0 msgr2=0x55eea7e9eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e944a0 0x55eea7e9eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5f7fe640 1 -- 172.21.15.143:0/304934995 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea7b6b3c0 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5effd640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e944a0 0x55eea7e9eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7e9c2a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab5002a930 tx=0x7fab50032e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de470d0a9c39daac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab3effd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab3effd640 1 -- 172.21.15.143:0/304934995 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50035040 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab480047a0 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.570+0000 7fab3effd640 1 -- 172.21.15.143:0/304934995 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fab500317e0 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab3effd640 1 -- 172.21.15.143:0/304934995 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab50004040 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 10 client.26269 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eea7eb6480 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab3effd640 1 -- 172.21.15.143:0/304934995 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab480047a0 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab3effd640 1 -- 172.21.15.143:0/304934995 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fab500047e0 con 0x55eea7e94dc0 2024-09-10T10:51:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 10 client.26269 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 20 client.26269 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7eba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 4 client.26269 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eea7e07850 con 0x55eea7eb7d00 2024-09-10T10:51:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.571+0000 7fab5effd640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7eba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.572+0000 7fab5effd640 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7eba120 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cb540a18cb3726 server_cookie=5c57dc492720af11 in_seq=0 out_seq=0 2024-09-10T10:51:43.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.572+0000 7fab3effd640 10 client.26269 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:43.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab3effd640 1 -- 172.21.15.143:0/304934995 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eea7e07850 con 0x55eea7eb7d00 2024-09-10T10:51:43.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab3effd640 10 client.26269 handle_command_reply: tid=0 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 client.26269 shutdown 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 msgr2=0x55eea7e9c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7e9c2a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab5002a930 tx=0x7fab50032e70 comp rx=0 tx=0).stop 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 shutdown_connections 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e9f680 0x55eea7e9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7eba120 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7e9c2a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab3effd640 0 client.26269 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 --2- 172.21.15.143:0/304934995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e944a0 0x55eea7e9eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 >> 172.21.15.143:0/304934995 conn(0x55eea7e90a30 msgr2=0x55eea7e96760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.573+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 shutdown_connections 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.574+0000 7fab682f9740 1 -- 172.21.15.143:0/304934995 wait complete. 2024-09-10T10:51:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.574+0000 7fab682f9740 20 client.26269 trim_cache size 0 max 16384 2024-09-10T10:51:43.592 INFO:tasks.workunit.client.0.smithi143.stdout: 94 11473 58 8506 154 4348 87 2024-09-10T10:51:43.595 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.600+0000 7fab682f9740 1 Processor -- start 2024-09-10T10:51:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.600+0000 7fab682f9740 1 -- start start 2024-09-10T10:51:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab682f9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab682f9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7ebf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab682f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ebff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab682f9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab601ac910 con 0x55eea7e94dc0 2024-09-10T10:51:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab682f9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab601ae450 con 0x55eea7e944a0 2024-09-10T10:51:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab682f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab601acfe0 con 0x55eea7e9f680 2024-09-10T10:51:43.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5f7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7ebf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:57324/0 (socket says 172.21.15.143:57324) 2024-09-10T10:51:43.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 -- 172.21.15.143:0/2888511425 learned_addr learned my addr 172.21.15.143:0/2888511425 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:43.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ebff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 -- 172.21.15.143:0/2888511425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 msgr2=0x55eea7ebff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ebff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 -- 172.21.15.143:0/2888511425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 msgr2=0x55eea7ebf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7ebf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 -- 172.21.15.143:0/2888511425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea7eba490 con 0x55eea7e944a0 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.601+0000 7fab5effd640 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb3d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fab48009dd0 tx=0x7fab48009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f53efca61f94801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.602+0000 7fab557fa640 1 -- 172.21.15.143:0/2888511425 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4800f7e0 con 0x55eea7e944a0 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.602+0000 7fab557fa640 1 -- 172.21.15.143:0/2888511425 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab4800fe30 con 0x55eea7e944a0 2024-09-10T10:51:43.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.602+0000 7fab557fa640 1 -- 172.21.15.143:0/2888511425 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48005370 con 0x55eea7e944a0 2024-09-10T10:51:43.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 -- 172.21.15.143:0/2888511425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 msgr2=0x55eea7eb3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb3d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fab48009dd0 tx=0x7fab48009e80 comp rx=0 tx=0).stop 2024-09-10T10:51:43.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 -- 172.21.15.143:0/2888511425 shutdown_connections 2024-09-10T10:51:43.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e9f680 0x55eea7ebff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e94dc0 0x55eea7ebf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 --2- 172.21.15.143:0/2888511425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb3d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 -- 172.21.15.143:0/2888511425 >> 172.21.15.143:0/2888511425 conn(0x55eea7e90a30 msgr2=0x55eea7e97900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 -- 172.21.15.143:0/2888511425 shutdown_connections 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.603+0000 7fab682f9740 1 -- 172.21.15.143:0/2888511425 wait complete. 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.604+0000 7fab682f9740 1 -- start start 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.604+0000 7fab682f9740 1 Processor -- start 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.604+0000 7fab682f9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.604+0000 7fab682f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7eb05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.604+0000 7fab682f9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e9f680 0x55eea7eab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.604+0000 7fab5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:57332/0 (socket says 172.21.15.143:57332) 2024-09-10T10:51:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 -- 172.21.15.143:0/3044117389 learned_addr learned my addr 172.21.15.143:0/3044117389 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fab601add80 con 0x55eea7e9f680 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab601ad6b0 con 0x55eea7e944a0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab601aa8f0 con 0x55eea7e94dc0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e9f680 0x55eea7eab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7eb05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab682f9740 10 client.0 _ll_register_callbacks cb 0x7fab50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 -- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 msgr2=0x55eea7eb05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7eb05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 -- 172.21.15.143:0/3044117389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e9f680 msgr2=0x55eea7eab450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e9f680 0x55eea7eab450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea7e9f360 con 0x55eea7e944a0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab5effd640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb0090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab48006010 tx=0x7fab48013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c54d33df89f1ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab3effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:51:43.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab3effd640 1 -- 172.21.15.143:0/3044117389 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48036070 con 0x55eea7e944a0 2024-09-10T10:51:43.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4800fe30 con 0x55eea7e944a0 2024-09-10T10:51:43.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.605+0000 7fab3effd640 1 -- 172.21.15.143:0/3044117389 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fab48004d10 con 0x55eea7e944a0 2024-09-10T10:51:43.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.606+0000 7fab3effd640 1 -- 172.21.15.143:0/3044117389 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab4803f050 con 0x55eea7e944a0 2024-09-10T10:51:43.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.606+0000 7fab682f9740 10 client.26210 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.606+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eea7f22300 con 0x55eea7e944a0 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.606+0000 7fab3effd640 1 -- 172.21.15.143:0/3044117389 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab48004f10 con 0x55eea7e944a0 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.606+0000 7fab3effd640 1 -- 172.21.15.143:0/3044117389 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fab48039020 con 0x55eea7e944a0 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab682f9740 10 client.26210 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab682f9740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab682f9740 20 client.26210 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab682f9740 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7f247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab682f9740 4 client.26210 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:51:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fab4800fe30 con 0x55eea7eb7d00 2024-09-10T10:51:43.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.607+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7f247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab3effd640 10 client.26210 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:43.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab5f7fe640 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7f247f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ad4697a338a77a server_cookie=1e3b5d871e05c055 in_seq=0 out_seq=0 2024-09-10T10:51:43.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab3effd640 1 -- 172.21.15.143:0/3044117389 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fab4800fe30 con 0x55eea7eb7d00 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab3effd640 10 client.26210 handle_command_reply: tid=0 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 client.26210 shutdown 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 msgr2=0x55eea7eb0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb0090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab48006010 tx=0x7fab48013040 comp rx=0 tx=0).stop 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 shutdown_connections 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eea7e9f680 0x55eea7eab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eea7eb7d00 0x55eea7f247f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eea7e94dc0 0x55eea7eb05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 --2- 172.21.15.143:0/3044117389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eea7e944a0 0x55eea7eb0090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 >> 172.21.15.143:0/3044117389 conn(0x55eea7e90a30 msgr2=0x55eea7e8f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.608+0000 7fab3effd640 0 client.26210 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.609+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 shutdown_connections 2024-09-10T10:51:43.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.609+0000 7fab682f9740 1 -- 172.21.15.143:0/3044117389 wait complete. 2024-09-10T10:51:43.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.609+0000 7fab682f9740 20 client.26210 trim_cache size 0 max 16384 2024-09-10T10:51:43.614 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:51:43.614 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:51:43.614 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:51:43.614 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fab4c0a2d50 msgr2=0x7fab4c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fab4c0a2d50 0x7fab4c0a5210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fab5809b7f0 tx=0x7fab58004950 comp rx=0 tx=0).stop 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60109120 msgr2=0x7fab6019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60109120 0x7fab6019ef00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab5800da10 tx=0x7fab5800ded0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 shutdown_connections 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fab4c0a2d50 0x7fab4c0a5210 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab60109120 0x7fab6019ef00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab601049a0 0x7fab6019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 --2- 172.21.15.143:0/4194569933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab60103ff0 0x7fab6019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 >> 172.21.15.143:0/4194569933 conn(0x7fab600ffab0 msgr2=0x7fab60100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 shutdown_connections 2024-09-10T10:51:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:43.610+0000 7fab65f20640 1 -- 172.21.15.143:0/4194569933 wait complete. 2024-09-10T10:51:43.726 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:43 smithi143 ceph-mon[31057]: pgmap v551: 129 pgs: 129 active+clean; 417 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-10T10:51:43.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:43 smithi178 ceph-mon[34813]: pgmap v551: 129 pgs: 129 active+clean; 417 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-10T10:51:43.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:43 smithi144 ceph-mon[34230]: pgmap v551: 129 pgs: 129 active+clean; 417 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-10T10:51:45.073 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:51:45.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:51:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.297+0000 7fd40180e640 1 -- 172.21.15.143:0/848178859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 msgr2=0x7fd3fc107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.297+0000 7fd40180e640 1 --2- 172.21.15.143:0/848178859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 0x7fd3fc107bc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009930 tx=0x7fd3e402a420 comp rx=0 tx=0).stop 2024-09-10T10:51:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.297+0000 7fd40180e640 1 -- 172.21.15.143:0/848178859 shutdown_connections 2024-09-10T10:51:45.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.297+0000 7fd40180e640 1 --2- 172.21.15.143:0/848178859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3fc108300 0x7fd3fc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.298+0000 7fd40180e640 1 --2- 172.21.15.143:0/848178859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 0x7fd3fc107bc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.298+0000 7fd40180e640 1 --2- 172.21.15.143:0/848178859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc102dd0 0x7fd3fc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.298+0000 7fd40180e640 1 -- 172.21.15.143:0/848178859 >> 172.21.15.143:0/848178859 conn(0x7fd3fc0fc7d0 msgr2=0x7fd3fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.298+0000 7fd40180e640 1 -- 172.21.15.143:0/848178859 shutdown_connections 2024-09-10T10:51:45.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.298+0000 7fd40180e640 1 -- 172.21.15.143:0/848178859 wait complete. 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.298+0000 7fd40180e640 1 Processor -- start 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 -- start start 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3fc102dd0 0x7fd3fc100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 0x7fd3fc100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 0x7fd3fc1014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc111f00 con 0x7fd3fc103780 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc111d80 con 0x7fd3fc102dd0 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd40180e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc112080 con 0x7fd3fc108300 2024-09-10T10:51:45.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd3fb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 0x7fd3fc1014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd3fb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 0x7fd3fc1014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:59002/0 (socket says 172.21.15.143:59002) 2024-09-10T10:51:45.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.299+0000 7fd3fb7fe640 1 -- 172.21.15.143:0/3125095837 learned_addr learned my addr 172.21.15.143:0/3125095837 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:45.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3fc102dd0 0x7fd3fc100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 0x7fd3fc100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fb7fe640 1 -- 172.21.15.143:0/3125095837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3fc102dd0 msgr2=0x7fd3fc100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3fc102dd0 0x7fd3fc100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fb7fe640 1 -- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 msgr2=0x7fd3fc100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 0x7fd3fc100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fb7fe640 1 -- 172.21.15.143:0/3125095837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1aaeb0 con 0x7fd3fc108300 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 0x7fd3fc1014b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d750 tx=0x7fd3ec00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.300+0000 7fd40080c640 1 -- 172.21.15.143:0/3125095837 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec017070 con 0x7fd3fc108300 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.301+0000 7fd40080c640 1 -- 172.21.15.143:0/3125095837 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3ec0054d0 con 0x7fd3fc108300 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.301+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc1ab0e0 con 0x7fd3fc108300 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.301+0000 7fd40080c640 1 -- 172.21.15.143:0/3125095837 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec004c30 con 0x7fd3fc108300 2024-09-10T10:51:45.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.301+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc1ab5f0 con 0x7fd3fc108300 2024-09-10T10:51:45.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.304+0000 7fd40080c640 1 -- 172.21.15.143:0/3125095837 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd3ec004e90 con 0x7fd3fc108300 2024-09-10T10:51:45.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.304+0000 7fd403be7740 1 Processor -- start 2024-09-10T10:51:45.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.304+0000 7fd403be7740 1 -- start start 2024-09-10T10:51:45.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.304+0000 7fd403be7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a2971bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.304+0000 7fd403be7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29726400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd403be7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a29728d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd403be7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557a296812a0 con 0x557a2971b760 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd403be7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a29628b10 con 0x557a29726940 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd403be7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557a2967d260 con 0x557a2971c080 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29726400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29726400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:59016/0 (socket says 172.21.15.143:59016) 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 -- 172.21.15.143:0/29633573 learned_addr learned my addr 172.21.15.143:0/29633573 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a2971bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 -- 172.21.15.143:0/29633573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 msgr2=0x557a29728d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a29728d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 -- 172.21.15.143:0/29633573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 msgr2=0x557a2971bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a2971bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 -- 172.21.15.143:0/29633573 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a293f2b90 con 0x557a2971c080 2024-09-10T10:51:45.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a2971bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:45.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.305+0000 7fd3faffd640 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29726400 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40114f0 tx=0x7fd3f40119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f9c9723249cf7f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.306+0000 7fd40080c640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd3e80a2c00 0x7fd3e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.306+0000 7fd3f1ffb640 1 -- 172.21.15.143:0/29633573 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f401e070 con 0x557a2971c080 2024-09-10T10:51:45.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.306+0000 7fd3f1ffb640 1 -- 172.21.15.143:0/29633573 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f400fad0 con 0x557a2971c080 2024-09-10T10:51:45.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.306+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd3e80a2c00 0x7fd3e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.306+0000 7fd3f1ffb640 1 -- 172.21.15.143:0/29633573 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4019b10 con 0x557a2971c080 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.306+0000 7fd40080c640 1 -- 172.21.15.143:0/3125095837 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3ec0cec60 con 0x7fd3fc108300 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 -- 172.21.15.143:0/29633573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 msgr2=0x557a29726400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd3e80a2c00 0x7fd3e80a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4031440 tx=0x7fd3e4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29726400 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40114f0 tx=0x7fd3f40119b0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 -- 172.21.15.143:0/29633573 shutdown_connections 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a29728d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29726400 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 --2- 172.21.15.143:0/29633573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a2971bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 -- 172.21.15.143:0/29633573 >> 172.21.15.143:0/29633573 conn(0x557a29717cf0 msgr2=0x557a297180d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 -- 172.21.15.143:0/29633573 shutdown_connections 2024-09-10T10:51:45.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 -- 172.21.15.143:0/29633573 wait complete. 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.307+0000 7fd403be7740 1 -- start start 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 Processor -- start 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a29723870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29723db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a297242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a29723870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557a29729510 con 0x557a2971b760 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a29723870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39398/0 (socket says 172.21.15.143:39398) 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd3faffd640 1 -- 172.21.15.143:0/3691201659 learned_addr learned my addr 172.21.15.143:0/3691201659 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:45.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a296096e0 con 0x557a29726940 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557a29729690 con 0x557a2971c080 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd403be7740 10 client.0 _ll_register_callbacks cb 0x557a2962da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29723db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.308+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a297242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3fa7fc640 1 -- 172.21.15.143:0/3691201659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 msgr2=0x557a297242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a297242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3fa7fc640 1 -- 172.21.15.143:0/3691201659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 msgr2=0x557a29723870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a29723870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3fa7fc640 1 -- 172.21.15.143:0/3691201659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a29732a50 con 0x557a2971c080 2024-09-10T10:51:45.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a29723870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29723db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402a940 tx=0x7fd3e4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9980b6ef3d0ed3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.309+0000 7fd3daffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd3daffd640 1 -- 172.21.15.143:0/3691201659 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e404d070 con 0x557a2971c080 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f400fad0 con 0x557a2971c080 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd3daffd640 1 -- 172.21.15.143:0/3691201659 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3e4004450 con 0x557a2971c080 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd3daffd640 1 -- 172.21.15.143:0/3691201659 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3e4059050 con 0x557a2971c080 2024-09-10T10:51:45.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd403be7740 10 client.26287 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557a29735430 con 0x557a2971c080 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd3daffd640 1 -- 172.21.15.143:0/3691201659 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f400fad0 con 0x557a2971c080 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.310+0000 7fd3daffd640 1 -- 172.21.15.143:0/3691201659 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fd3e4049650 con 0x557a2971c080 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd403be7740 10 client.26287 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd403be7740 10 client.26287 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd403be7740 20 client.26287 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd403be7740 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a29736c10 0x557a297390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd403be7740 4 client.26287 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a2968e9d0 con 0x557a29736c10 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.311+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a29736c10 0x557a297390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.312+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a29736c10 0x557a297390d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5302cd966e548e server_cookie=b639901ba164828e in_seq=0 out_seq=0 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.312+0000 7fd3daffd640 10 client.26287 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.312+0000 7fd3daffd640 1 -- 172.21.15.143:0/3691201659 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a2968e9d0 con 0x557a29736c10 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.312+0000 7fd3daffd640 10 client.26287 handle_command_reply: tid=0 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.312+0000 7fd403be7740 1 client.26287 shutdown 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 msgr2=0x557a29723db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29723db0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402a940 tx=0x7fd3e4009260 comp rx=0 tx=0).stop 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 shutdown_connections 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a297242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a29736c10 0x557a297390d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971c080 0x557a29723db0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 --2- 172.21.15.143:0/3691201659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971b760 0x557a29723870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 >> 172.21.15.143:0/3691201659 conn(0x557a29717cf0 msgr2=0x557a2971da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:45.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd3daffd640 0 client.26287 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:45.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 shutdown_connections 2024-09-10T10:51:45.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 1 -- 172.21.15.143:0/3691201659 wait complete. 2024-09-10T10:51:45.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.313+0000 7fd403be7740 20 client.26287 trim_cache size 0 max 16384 2024-09-10T10:51:45.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.339+0000 7fd403be7740 1 Processor -- start 2024-09-10T10:51:45.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.339+0000 7fd403be7740 1 -- start start 2024-09-10T10:51:45.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd403be7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a29725740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd403be7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a29722140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd403be7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2973e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd403be7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b5190 con 0x557a2971c080 2024-09-10T10:51:45.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd403be7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b6cd0 con 0x557a29726940 2024-09-10T10:51:45.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd403be7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b5860 con 0x557a2971b760 2024-09-10T10:51:45.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a29725740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a29725740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:59042/0 (socket says 172.21.15.143:59042) 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 -- 172.21.15.143:0/3893634565 learned_addr learned my addr 172.21.15.143:0/3893634565 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2973e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a29722140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 -- 172.21.15.143:0/3893634565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 msgr2=0x557a2973e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2973e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 -- 172.21.15.143:0/3893634565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 msgr2=0x557a29722140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a29722140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.340+0000 7fd3faffd640 1 -- 172.21.15.143:0/3893634565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a29717b20 con 0x557a2971b760 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.341+0000 7fd3faffd640 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a29725740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4019770 tx=0x7fd3f4009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd52e6eca629b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.341+0000 7fd3f17fa640 1 -- 172.21.15.143:0/3893634565 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f401e040 con 0x557a2971b760 2024-09-10T10:51:45.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.341+0000 7fd3f17fa640 1 -- 172.21.15.143:0/3893634565 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f402a070 con 0x557a2971b760 2024-09-10T10:51:45.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.341+0000 7fd3f17fa640 1 -- 172.21.15.143:0/3893634565 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4019a90 con 0x557a2971b760 2024-09-10T10:51:45.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 -- 172.21.15.143:0/3893634565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 msgr2=0x557a29725740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a29725740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4019770 tx=0x7fd3f4009990 comp rx=0 tx=0).stop 2024-09-10T10:51:45.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 -- 172.21.15.143:0/3893634565 shutdown_connections 2024-09-10T10:51:45.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2973e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a29722140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 --2- 172.21.15.143:0/3893634565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a29725740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 -- 172.21.15.143:0/3893634565 >> 172.21.15.143:0/3893634565 conn(0x557a29717cf0 msgr2=0x557a29716e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 -- 172.21.15.143:0/3893634565 shutdown_connections 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 -- 172.21.15.143:0/3893634565 wait complete. 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.342+0000 7fd403be7740 1 -- start start 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 Processor -- start 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a2972ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a2972f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2972bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a2972ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd3faffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a2972ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:59058/0 (socket says 172.21.15.143:59058) 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b6600 con 0x557a2971c080 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b5f30 con 0x557a29726940 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc1b3170 con 0x557a2971b760 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd3faffd640 1 -- 172.21.15.143:0/2043761205 learned_addr learned my addr 172.21.15.143:0/2043761205 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd403be7740 10 client.0 _ll_register_callbacks cb 0x7fd3e4032e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.343+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a2972f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2972bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3faffd640 1 -- 172.21.15.143:0/2043761205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 msgr2=0x557a2972bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3faffd640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2972bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3faffd640 1 -- 172.21.15.143:0/2043761205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 msgr2=0x557a2972f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3faffd640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a2972f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3faffd640 1 -- 172.21.15.143:0/2043761205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a29713260 con 0x557a2971b760 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3fb7fe640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2972bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a2972f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3faffd640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a2972ec70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400a010 tx=0x7fd3f4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ac90a5e5e98b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3daffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3daffd640 1 -- 172.21.15.143:0/2043761205 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4033060 con 0x557a2971b760 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f402a070 con 0x557a2971b760 2024-09-10T10:51:45.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3daffd640 1 -- 172.21.15.143:0/2043761205 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd3f4011040 con 0x557a2971b760 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd3daffd640 1 -- 172.21.15.143:0/2043761205 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3f401e070 con 0x557a2971b760 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd403be7740 10 client.26293 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.344+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557a297a24a0 con 0x557a2971b760 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd3daffd640 1 -- 172.21.15.143:0/2043761205 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f402a070 con 0x557a2971b760 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd3daffd640 1 -- 172.21.15.143:0/2043761205 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fd3f40197a0 con 0x557a2971b760 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd403be7740 10 client.26293 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd403be7740 10 client.26293 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd403be7740 20 client.26293 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd403be7740 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a297a2930 0x557a297a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd403be7740 4 client.26293 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T10:51:45.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x557a29688d20 con 0x557a297a2930 2024-09-10T10:51:45.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.345+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a297a2930 0x557a297a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.346+0000 7fd3daffd640 10 client.26293 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:45.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:45.346+0000 7fd3fa7fc640 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a297a2930 0x557a297a4d20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d673acbe2ed8d708 server_cookie=83d57d265ac2f1c1 in_seq=0 out_seq=0 2024-09-10T10:51:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:46.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 -- 172.21.15.143:0/2564618305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0102d90 msgr2=0x7f78c0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2564618305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0102d90 0x7f78c0103170 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009930 tx=0x7f78b002a420 comp rx=0 tx=0).stop 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 -- 172.21.15.143:0/2564618305 shutdown_connections 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2564618305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0108290 0x7f78c010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2564618305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78c0103740 0x7f78c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2564618305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0102d90 0x7f78c0103170 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 -- 172.21.15.143:0/2564618305 >> 172.21.15.143:0/2564618305 conn(0x7f78c00fc7d0 msgr2=0x7f78c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 -- 172.21.15.143:0/2564618305 shutdown_connections 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.199+0000 7f78c64d8640 1 -- 172.21.15.143:0/2564618305 wait complete. 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.200+0000 7f78c64d8640 1 Processor -- start 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.200+0000 7f78c64d8640 1 -- start start 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c64d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0102d90 0x7f78c0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c64d8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 0x7f78c0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c64d8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78c0108290 0x7f78c019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c64d8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111f40 con 0x7f78c0103740 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c64d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111dc0 con 0x7f78c0102d90 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c64d8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01120c0 con 0x7f78c0108290 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 0x7f78c0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0102d90 0x7f78c0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 0x7f78c0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39446/0 (socket says 172.21.15.143:39446) 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 -- 172.21.15.143:0/2936829367 learned_addr learned my addr 172.21.15.143:0/2936829367 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78c4a75640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78c0108290 0x7f78c019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 -- 172.21.15.143:0/2936829367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78c0108290 msgr2=0x7f78c019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78c0108290 0x7f78c019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 -- 172.21.15.143:0/2936829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0102d90 msgr2=0x7f78c0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0102d90 0x7f78c0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 -- 172.21.15.143:0/2936829367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c019d4b0 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bffff640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0102d90 0x7f78c0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bf7fe640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 0x7f78c0198a40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e9f0 tx=0x7f78b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.201+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b400ccc0 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.202+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78b40054d0 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.202+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c019d7a0 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.202+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4010430 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.202+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01a5080 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.203+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c006a7f0 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.204+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f78b4005020 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.205+0000 7f78bd7fa640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78a80a2b80 0x7f78a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.205+0000 7f78bffff640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78a80a2b80 0x7f78a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.205+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f78b40cf760 con 0x7f78c0103740 2024-09-10T10:51:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.205+0000 7f78bffff640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78a80a2b80 0x7f78a80a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009900 tx=0x7f78b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.209+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78b409c360 con 0x7f78c0103740 2024-09-10T10:51:46.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.454+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c010a2a0 con 0x7f78c0103740 2024-09-10T10:51:46.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.455+0000 7f78bd7fa640 1 -- 172.21.15.143:0/2936829367 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11998 (secure 0 0 0) 0x7f78b409bc60 con 0x7f78c0103740 2024-09-10T10:51:46.457 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:51:46.457 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"btime":"2024-09-10T10:51:06:617569+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:51:06.305790+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:51:46.458 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2024-09-10T10:51:46.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78a80a2b80 msgr2=0x7f78a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78a80a2b80 0x7f78a80a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009900 tx=0x7f78b0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 msgr2=0x7f78c0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 0x7f78c0198a40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e9f0 tx=0x7f78b400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 shutdown_connections 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f78a80a2b80 0x7f78a80a5040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78c0108290 0x7f78c019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78c0103740 0x7f78c0198a40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 --2- 172.21.15.143:0/2936829367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0102d90 0x7f78c0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 >> 172.21.15.143:0/2936829367 conn(0x7f78c00fc7d0 msgr2=0x7f78c0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 shutdown_connections 2024-09-10T10:51:46.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:46.457+0000 7f78c64d8640 1 -- 172.21.15.143:0/2936829367 wait complete. 2024-09-10T10:51:46.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:46 smithi143 ceph-mon[31057]: pgmap v552: 129 pgs: 129 active+clean; 417 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-10T10:51:46.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:46 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2936829367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:51:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:46 smithi178 ceph-mon[34813]: pgmap v552: 129 pgs: 129 active+clean; 417 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-10T10:51:46.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:46 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2936829367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:51:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:46 smithi144 ceph-mon[34230]: pgmap v552: 129 pgs: 129 active+clean; 417 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-09-10T10:51:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:46 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2936829367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:51:47.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:51:47.549 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 -- 172.21.15.143:0/2014393747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c101e10 msgr2=0x7f9c5c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 --2- 172.21.15.143:0/2014393747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c101e10 0x7f9c5c1021f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c02a410 comp rx=0 tx=0).stop 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 -- 172.21.15.143:0/2014393747 shutdown_connections 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 --2- 172.21.15.143:0/2014393747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c106f40 0x7f9c5c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 --2- 172.21.15.143:0/2014393747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 0x7f9c5c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 --2- 172.21.15.143:0/2014393747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c101e10 0x7f9c5c1021f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 -- 172.21.15.143:0/2014393747 >> 172.21.15.143:0/2014393747 conn(0x7f9c5c0fd8d0 msgr2=0x7f9c5c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 -- 172.21.15.143:0/2014393747 shutdown_connections 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.389+0000 7f9c60f61640 1 -- 172.21.15.143:0/2014393747 wait complete. 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 Processor -- start 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 -- start start 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c101e10 0x7f9c5c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 0x7f9c5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 0x7f9c5c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c111e10 con 0x7f9c5c106f40 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c111c90 con 0x7f9c5c101e10 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c60f61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c111f90 con 0x7f9c5c1027c0 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c5ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 0x7f9c5c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c5a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c101e10 0x7f9c5c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 0x7f9c5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c5ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 0x7f9c5c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51380/0 (socket says 172.21.15.143:51380) 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c5ad9d640 1 -- 172.21.15.143:0/4215036420 learned_addr learned my addr 172.21.15.143:0/4215036420 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c5ad9d640 1 -- 172.21.15.143:0/4215036420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 msgr2=0x7f9c5c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.390+0000 7f9c5ad9d640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 0x7f9c5c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c5ad9d640 1 -- 172.21.15.143:0/4215036420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c101e10 msgr2=0x7f9c5c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c5ad9d640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c101e10 0x7f9c5c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c5ad9d640 1 -- 172.21.15.143:0/4215036420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c19d340 con 0x7f9c5c106f40 2024-09-10T10:51:48.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c53fff640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 0x7f9c5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c5ad9d640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 0x7f9c5c19cc60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400cad0 tx=0x7f9c5400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c537fe640 1 -- 172.21.15.143:0/4215036420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400ae20 con 0x7f9c5c106f40 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c537fe640 1 -- 172.21.15.143:0/4215036420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c54005020 con 0x7f9c5c106f40 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c537fe640 1 -- 172.21.15.143:0/4215036420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400f6b0 con 0x7f9c5c106f40 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c19d630 con 0x7f9c5c106f40 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.391+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1a4f10 con 0x7f9c5c106f40 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.393+0000 7f9c537fe640 1 -- 172.21.15.143:0/4215036420 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9c540051d0 con 0x7f9c5c106f40 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 Processor -- start 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 -- start start 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 0x55cab2f5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f65370 0x55cab2f67760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cab2ec02f0 con 0x55cab2f5bb40 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cab2e67cf0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c6333a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cab2ebc2b0 con 0x55cab2f65370 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 0x55cab2f5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c537fe640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9c480a2b80 0x7f9c480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 0x55cab2f5a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51382/0 (socket says 172.21.15.143:51382) 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c53fff640 1 -- 172.21.15.143:0/2567067809 learned_addr learned my addr 172.21.15.143:0/2567067809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f65370 0x55cab2f67760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c5ad9d640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9c480a2b80 0x7f9c480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.394+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c53fff640 1 -- 172.21.15.143:0/2567067809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f65370 msgr2=0x55cab2f67760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f65370 0x55cab2f67760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c53fff640 1 -- 172.21.15.143:0/2567067809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 msgr2=0x55cab2f5aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f5aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c53fff640 1 -- 172.21.15.143:0/2567067809 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab2e4c3a0 con 0x55cab2f5bb40 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c537fe640 1 -- 172.21.15.143:0/4215036420 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9c540cf310 con 0x7f9c5c106f40 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 0x55cab2f5a090 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005bc0 tx=0x7f9c4400bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e09b8f2c875885c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c5ad9d640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9c480a2b80 0x7f9c480a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c54096360 tx=0x7f9c5406e980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c50ff9640 1 -- 172.21.15.143:0/2567067809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44012c50 con 0x55cab2f5bb40 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c50ff9640 1 -- 172.21.15.143:0/2567067809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c44012df0 con 0x55cab2f5bb40 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.395+0000 7f9c50ff9640 1 -- 172.21.15.143:0/2567067809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440054b0 con 0x55cab2f5bb40 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 -- 172.21.15.143:0/2567067809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 msgr2=0x55cab2f5a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 0x55cab2f5a090 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005bc0 tx=0x7f9c4400bb80 comp rx=0 tx=0).stop 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 -- 172.21.15.143:0/2567067809 shutdown_connections 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f65370 0x55cab2f67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2567067809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5bb40 0x55cab2f5a090 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 -- 172.21.15.143:0/2567067809 >> 172.21.15.143:0/2567067809 conn(0x55cab2f23310 msgr2=0x55cab2f56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 -- 172.21.15.143:0/2567067809 shutdown_connections 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 -- 172.21.15.143:0/2567067809 wait complete. 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 -- start start 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.396+0000 7f9c6333a740 1 Processor -- start 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f64cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2f624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cab2f68010 con 0x55cab2f5a5d0 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cab2e7b060 con 0x55cab2f65370 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cab2f68190 con 0x55cab2f5bb40 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 10 client.0 _ll_register_callbacks cb 0x55cab2bce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:48.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f64cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f64cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51394/0 (socket says 172.21.15.143:51394) 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 -- 172.21.15.143:0/541892512 learned_addr learned my addr 172.21.15.143:0/541892512 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 -- 172.21.15.143:0/541892512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 msgr2=0x55cab2f61f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 -- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 msgr2=0x55cab2f624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2f624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 -- 172.21.15.143:0/541892512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab2c313c0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c53fff640 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f64cd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9c44015000 tx=0x7f9c4400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f293c4a3bfa0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c41ffb640 1 -- 172.21.15.143:0/541892512 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004020 con 0x55cab2f5a5d0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c41ffb640 1 -- 172.21.15.143:0/541892512 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9c440041c0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.397+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44012df0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c41ffb640 1 -- 172.21.15.143:0/541892512 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9c440043c0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 10 client.16899 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cab2f73a20 con 0x55cab2f5a5d0 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c41ffb640 1 -- 172.21.15.143:0/541892512 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c44012df0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c41ffb640 1 -- 172.21.15.143:0/541892512 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f9c440046c0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 10 client.16899 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 10 client.16899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 20 client.16899 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2f74e80 0x55cab2f77270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 4 client.16899 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cab2ecd860 con 0x55cab2f74e80 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.398+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2f74e80 0x55cab2f77270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.399+0000 7f9c41ffb640 10 client.16899 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.399+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2f74e80 0x55cab2f77270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9231e2f9c995341 server_cookie=a27a5426106e78d6 in_seq=0 out_seq=0 2024-09-10T10:51:48.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c41ffb640 1 -- 172.21.15.143:0/541892512 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cab2ecd860 con 0x55cab2f74e80 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c41ffb640 10 client.16899 handle_command_reply: tid=0 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 client.16899 shutdown 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 msgr2=0x55cab2f64cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f64cd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9c44015000 tx=0x7f9c4400b540 comp rx=0 tx=0).stop 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 shutdown_connections 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2f74e80 0x55cab2f77270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2f624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 --2- 172.21.15.143:0/541892512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f64cd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 >> 172.21.15.143:0/541892512 conn(0x55cab2f23310 msgr2=0x55cab2f5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c41ffb640 0 client.16899 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 shutdown_connections 2024-09-10T10:51:48.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 1 -- 172.21.15.143:0/541892512 wait complete. 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.400+0000 7f9c6333a740 20 client.16899 trim_cache size 0 max 16384 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 Processor -- start 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 -- start start 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 0x55cab2f60f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1aeb70 con 0x55cab2f65370 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1b06b0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c6333a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1af240 con 0x55cab2f5bb40 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 0x55cab2f60f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 0x55cab2f60f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51400/0 (socket says 172.21.15.143:51400) 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 -- 172.21.15.143:0/2795952248 learned_addr learned my addr 172.21.15.143:0/2795952248 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 -- 172.21.15.143:0/2795952248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 msgr2=0x55cab2f61840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 -- 172.21.15.143:0/2795952248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 msgr2=0x55cab2f712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 -- 172.21.15.143:0/2795952248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab2f604d0 con 0x55cab2f65370 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.413+0000 7f9c53fff640 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 0x55cab2f60f60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400c010 tx=0x7f9c440123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572f303ad9538215 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:48.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c43fff640 1 -- 172.21.15.143:0/2795952248 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44010de0 con 0x55cab2f65370 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c43fff640 1 -- 172.21.15.143:0/2795952248 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c44004660 con 0x55cab2f65370 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c43fff640 1 -- 172.21.15.143:0/2795952248 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44003970 con 0x55cab2f65370 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 -- 172.21.15.143:0/2795952248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 msgr2=0x55cab2f60f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 0x55cab2f60f60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400c010 tx=0x7f9c440123f0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 -- 172.21.15.143:0/2795952248 shutdown_connections 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f65370 0x55cab2f60f60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 --2- 172.21.15.143:0/2795952248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f5a5d0 0x55cab2f712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 -- 172.21.15.143:0/2795952248 >> 172.21.15.143:0/2795952248 conn(0x55cab2f23310 msgr2=0x55cab2f60850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 -- 172.21.15.143:0/2795952248 shutdown_connections 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.414+0000 7f9c6333a740 1 -- 172.21.15.143:0/2795952248 wait complete. 2024-09-10T10:51:48.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 -- start start 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 Processor -- start 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f68b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f69070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2fd7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1affe0 con 0x55cab2f5a5d0 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1af910 con 0x55cab2f65370 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1acb50 con 0x55cab2f5bb40 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c6333a740 10 client.0 _ll_register_callbacks cb 0x7f9c4c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f68b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c53fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f68b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51404/0 (socket says 172.21.15.143:51404) 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c53fff640 1 -- 172.21.15.143:0/1413764754 learned_addr learned my addr 172.21.15.143:0/1413764754 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f69070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c53fff640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2fd7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c5a59c640 1 -- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 msgr2=0x55cab2fd7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2fd7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c5a59c640 1 -- 172.21.15.143:0/1413764754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 msgr2=0x55cab2f68b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f68b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c5a59c640 1 -- 172.21.15.143:0/1413764754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab2fd7ea0 con 0x55cab2f5bb40 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.415+0000 7f9c53fff640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f68b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c5a59c640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f69070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009230 tx=0x7f9c4c032ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=165b9c15d75ab8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c41ffb640 1 -- 172.21.15.143:0/1413764754 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c035040 con 0x55cab2f5bb40 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44004660 con 0x55cab2f5bb40 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c41ffb640 1 -- 172.21.15.143:0/1413764754 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9c4c005040 con 0x55cab2f5bb40 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c41ffb640 1 -- 172.21.15.143:0/1413764754 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c005240 con 0x55cab2f5bb40 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 10 client.26317 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:48.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cab2fdf2e0 con 0x55cab2f5bb40 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c41ffb640 1 -- 172.21.15.143:0/1413764754 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c44004660 con 0x55cab2f5bb40 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c41ffb640 1 -- 172.21.15.143:0/1413764754 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f9c4c0035f0 con 0x55cab2f5bb40 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 10 client.26317 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 10 client.26317 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 20 client.26317 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2fdfa60 0x55cab2fe1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 4 client.26317 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.416+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cab2ec7d70 con 0x55cab2fdfa60 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c53fff640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2fdfa60 0x55cab2fe1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c41ffb640 10 client.26317 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c53fff640 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2fdfa60 0x55cab2fe1e80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac769f6b0de09406 server_cookie=5a5d197842f3699f in_seq=0 out_seq=0 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c41ffb640 1 -- 172.21.15.143:0/1413764754 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cab2ec7d70 con 0x55cab2fdfa60 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c41ffb640 10 client.26317 handle_command_reply: tid=0 2024-09-10T10:51:48.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 client.26317 shutdown 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 msgr2=0x55cab2f69070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f69070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009230 tx=0x7f9c4c032ed0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 shutdown_connections 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55cab2fdfa60 0x55cab2fe1e80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cab2f65370 0x55cab2fd7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c41ffb640 0 client.26317 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cab2f5bb40 0x55cab2f69070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.417+0000 7f9c6333a740 1 --2- 172.21.15.143:0/1413764754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cab2f5a5d0 0x55cab2f68b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 >> 172.21.15.143:0/1413764754 conn(0x55cab2f23310 msgr2=0x55cab2f61070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 shutdown_connections 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c6333a740 1 -- 172.21.15.143:0/1413764754 wait complete. 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c6333a740 20 client.26317 trim_cache size 0 max 16384 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9c480a2b80 msgr2=0x7f9c480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9c480a2b80 0x7f9c480a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c54096360 tx=0x7f9c5406e980 comp rx=0 tx=0).stop 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 msgr2=0x7f9c5c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 0x7f9c5c19cc60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400cad0 tx=0x7f9c5400cf90 comp rx=0 tx=0).stop 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 shutdown_connections 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9c480a2b80 0x7f9c480a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c5c106f40 0x7f9c5c19cc60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c5c1027c0 0x7f9c5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 --2- 172.21.15.143:0/4215036420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c101e10 0x7f9c5c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 >> 172.21.15.143:0/4215036420 conn(0x7f9c5c0fd8d0 msgr2=0x7f9c5c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.418+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 shutdown_connections 2024-09-10T10:51:48.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:48.419+0000 7f9c60f61640 1 -- 172.21.15.143:0/4215036420 wait complete. 2024-09-10T10:51:48.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:48 smithi143 ceph-mon[31057]: pgmap v553: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 223 op/s 2024-09-10T10:51:48.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:48 smithi178 ceph-mon[34813]: pgmap v553: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 223 op/s 2024-09-10T10:51:48.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:48 smithi144 ceph-mon[34230]: pgmap v553: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 223 op/s 2024-09-10T10:51:49.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:51:49.669 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 -- 172.21.15.143:0/2800183176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 msgr2=0x7facf8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 --2- 172.21.15.143:0/2800183176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf8103120 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7facf4009900 tx=0x7facf402a480 comp rx=0 tx=0).stop 2024-09-10T10:51:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 -- 172.21.15.143:0/2800183176 shutdown_connections 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 --2- 172.21.15.143:0/2800183176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facf810c170 0x7facf810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 --2- 172.21.15.143:0/2800183176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 0x7facf810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 --2- 172.21.15.143:0/2800183176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf8103120 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 -- 172.21.15.143:0/2800183176 >> 172.21.15.143:0/2800183176 conn(0x7facf80fc7a0 msgr2=0x7facf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 -- 172.21.15.143:0/2800183176 shutdown_connections 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.440+0000 7facffb5a640 1 -- 172.21.15.143:0/2800183176 wait complete. 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 Processor -- start 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 -- start start 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 0x7facf81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facf810c170 0x7facf819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7facf8111e20 con 0x7facf8102d40 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facf8111ca0 con 0x7facf810c170 2024-09-10T10:51:50.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facffb5a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facf8111fa0 con 0x7facf8103660 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facfd8f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facfd8f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51422/0 (socket says 172.21.15.143:51422) 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facfd8f6640 1 -- 172.21.15.143:0/3376942958 learned_addr learned my addr 172.21.15.143:0/3376942958 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 0x7facf81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.441+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facf810c170 0x7facf819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd8f6640 1 -- 172.21.15.143:0/3376942958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 msgr2=0x7facf81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 0x7facf81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd8f6640 1 -- 172.21.15.143:0/3376942958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facf810c170 msgr2=0x7facf819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facf810c170 0x7facf819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd8f6640 1 -- 172.21.15.143:0/3376942958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf819d350 con 0x7facf8102d40 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 0x7facf81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf81983a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7facf4005940 tx=0x7facf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7faceeffd640 1 -- 172.21.15.143:0/3376942958 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf403f030 con 0x7facf8102d40 2024-09-10T10:51:50.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7faceeffd640 1 -- 172.21.15.143:0/3376942958 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf4003cb0 con 0x7facf8102d40 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf819d5e0 con 0x7facf8102d40 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7faceeffd640 1 -- 172.21.15.143:0/3376942958 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4035070 con 0x7facf8102d40 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.442+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81a4f20 con 0x7facf8102d40 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.444+0000 7faceeffd640 1 -- 172.21.15.143:0/3376942958 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7facf4042030 con 0x7facf8102d40 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 Processor -- start 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 -- start start 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb09f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55deb09522f0 con 0x55deb09ec5d0 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55deb08f9cf0 con 0x55deb09f7370 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7fad01f33740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55deb094e2b0 con 0x55deb09edb40 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7faceeffd640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7face40a2c00 0x7face40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7facfd8f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7facfd8f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09eca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51424/0 (socket says 172.21.15.143:51424) 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7facfd8f6640 1 -- 172.21.15.143:0/3079431891 learned_addr learned my addr 172.21.15.143:0/3079431891 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.445+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7face40a2c00 0x7face40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb09f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7faceeffd640 1 -- 172.21.15.143:0/3376942958 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7facf40efac0 con 0x7facf8102d40 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 -- 172.21.15.143:0/3079431891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 msgr2=0x55deb09f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb09f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 -- 172.21.15.143:0/3079431891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 msgr2=0x55deb09eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 -- 172.21.15.143:0/3079431891 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb08de3a0 con 0x55deb09edb40 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09ec090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7facf0005fd0 tx=0x7facf0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f51e4f185d403426 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7face40a2c00 0x7face40a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7facf0010c40 tx=0x7facf000c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7face3fff640 1 -- 172.21.15.143:0/3079431891 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0024070 con 0x55deb09edb40 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7face3fff640 1 -- 172.21.15.143:0/3079431891 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf0009c40 con 0x55deb09edb40 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.446+0000 7face3fff640 1 -- 172.21.15.143:0/3079431891 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0005280 con 0x55deb09edb40 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 -- 172.21.15.143:0/3079431891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 msgr2=0x55deb09ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09ec090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7facf0005fd0 tx=0x7facf0010740 comp rx=0 tx=0).stop 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 -- 172.21.15.143:0/3079431891 shutdown_connections 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb09f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 --2- 172.21.15.143:0/3079431891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09ec090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 -- 172.21.15.143:0/3079431891 >> 172.21.15.143:0/3079431891 conn(0x55deb09b5310 msgr2=0x55deb09e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 -- 172.21.15.143:0/3079431891 shutdown_connections 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 -- 172.21.15.143:0/3079431891 wait complete. 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.447+0000 7fad01f33740 1 -- start start 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 Processor -- start 2024-09-10T10:51:50.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09f4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 0x55deb09f4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55deb09fa010 con 0x55deb09f7370 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55deb090d060 con 0x55deb09ec5d0 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55deb09fa190 con 0x55deb09edb40 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7fad01f33740 10 client.0 _ll_register_callbacks cb 0x55deb06608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 0x55deb09f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 0x55deb09f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51428/0 (socket says 172.21.15.143:51428) 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 -- 172.21.15.143:0/2382529122 learned_addr learned my addr 172.21.15.143:0/2382529122 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfd0f5640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfd8f6640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 -- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 msgr2=0x55deb09f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 -- 172.21.15.143:0/2382529122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 msgr2=0x55deb09f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfe0f7640 1 -- 172.21.15.143:0/2382529122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb06c33c0 con 0x55deb09f7370 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfd8f6640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.448+0000 7facfd0f5640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7facfe0f7640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 0x55deb09f4810 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7facf0012e70 tx=0x7facf0012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276d1d203b7276fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7face17fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7face17fa640 1 -- 172.21.15.143:0/2382529122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0024040 con 0x55deb09f7370 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf0009c40 con 0x55deb09f7370 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7face17fa640 1 -- 172.21.15.143:0/2382529122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7facf0004c10 con 0x55deb09f7370 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7face17fa640 1 -- 172.21.15.143:0/2382529122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7facf0004e10 con 0x55deb09f7370 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 10 client.16917 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55deb0a05b30 con 0x55deb09f7370 2024-09-10T10:51:50.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7face17fa640 1 -- 172.21.15.143:0/2382529122 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf0009c40 con 0x55deb09f7370 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7face17fa640 1 -- 172.21.15.143:0/2382529122 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7facf000f040 con 0x55deb09f7370 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 10 client.16917 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 10 client.16917 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 20 client.16917 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a06ff0 0x55deb0a093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 4 client.16917 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.449+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb095f860 con 0x55deb0a06ff0 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.450+0000 7facfd8f6640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a06ff0 0x55deb0a093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.450+0000 7facfd8f6640 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a06ff0 0x55deb0a093e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1550b7fdf77c80f0 server_cookie=b38bb884e5d22ace in_seq=0 out_seq=0 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.450+0000 7face17fa640 10 client.16917 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.451+0000 7face17fa640 1 -- 172.21.15.143:0/2382529122 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deb095f860 con 0x55deb0a06ff0 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.451+0000 7face17fa640 10 client.16917 handle_command_reply: tid=0 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.451+0000 7fad01f33740 1 client.16917 shutdown 2024-09-10T10:51:50.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.451+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 msgr2=0x55deb09f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.451+0000 7fad01f33740 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 0x55deb09f4810 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7facf0012e70 tx=0x7facf0012ea0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 shutdown_connections 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a06ff0 0x55deb0a093e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09f7370 0x55deb09f4810 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7face17fa640 0 client.16917 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 --2- 172.21.15.143:0/2382529122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 >> 172.21.15.143:0/2382529122 conn(0x55deb09b5310 msgr2=0x55deb09ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 shutdown_connections 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 1 -- 172.21.15.143:0/2382529122 wait complete. 2024-09-10T10:51:50.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.452+0000 7fad01f33740 20 client.16917 trim_cache size 0 max 16384 2024-09-10T10:51:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 Processor -- start 2024-09-10T10:51:50.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 -- start start 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 0x55deb09f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09f7370 0x55deb0a0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7facf81aeb10 con 0x55deb09edb40 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b0650 con 0x55deb09ec5d0 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7fad01f33740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facf81af1e0 con 0x55deb09f7370 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 0x55deb09f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 0x55deb09f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51442/0 (socket says 172.21.15.143:51442) 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 -- 172.21.15.143:0/3390065736 learned_addr learned my addr 172.21.15.143:0/3390065736 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09f7370 0x55deb0a0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 -- 172.21.15.143:0/3390065736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09f7370 msgr2=0x55deb0a0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09f7370 0x55deb0a0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 -- 172.21.15.143:0/3390065736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 msgr2=0x55deb09f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd0f5640 1 -- 172.21.15.143:0/3390065736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb09e8510 con 0x55deb09edb40 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.466+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.467+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 0x55deb09f2e90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7face80128f0 tx=0x7face8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4bef309d5ba515 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.467+0000 7face37fe640 1 -- 172.21.15.143:0/3390065736 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8022070 con 0x55deb09edb40 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.467+0000 7face37fe640 1 -- 172.21.15.143:0/3390065736 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face800f8a0 con 0x55deb09edb40 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.467+0000 7face37fe640 1 -- 172.21.15.143:0/3390065736 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8003850 con 0x55deb09edb40 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 -- 172.21.15.143:0/3390065736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 msgr2=0x55deb09f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 0x55deb09f2e90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7face80128f0 tx=0x7face8005f90 comp rx=0 tx=0).stop 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 -- 172.21.15.143:0/3390065736 shutdown_connections 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09f7370 0x55deb0a0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09edb40 0x55deb09f2e90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 --2- 172.21.15.143:0/3390065736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09ec5d0 0x55deb09f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 -- 172.21.15.143:0/3390065736 >> 172.21.15.143:0/3390065736 conn(0x55deb09b5310 msgr2=0x55deb09e3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 -- 172.21.15.143:0/3390065736 shutdown_connections 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 -- 172.21.15.143:0/3390065736 wait complete. 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 -- start start 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.468+0000 7fad01f33740 1 Processor -- start 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09fa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb0a73e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7facf81aff80 con 0x55deb09ec5d0 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facf81af8b0 con 0x55deb09f7370 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facf81acaf0 con 0x55deb09edb40 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7fad01f33740 10 client.0 _ll_register_callbacks cb 0x7facf40b7bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09fa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51448/0 (socket says 172.21.15.143:51448) 2024-09-10T10:51:50.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 -- 172.21.15.143:0/3056696442 learned_addr learned my addr 172.21.15.143:0/3056696442 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfe0f7640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb0a73e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 -- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 msgr2=0x55deb09faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 -- 172.21.15.143:0/3056696442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 msgr2=0x55deb0a73e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb0a73e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 -- 172.21.15.143:0/3056696442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb0a74480 con 0x55deb09ec5d0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7facfd8f6640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09fa980 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7facf40ed5d0 tx=0x7facf4031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6044466ac62d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7face17fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.469+0000 7face17fa640 1 -- 172.21.15.143:0/3056696442 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4049040 con 0x55deb09ec5d0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7face17fa640 1 -- 172.21.15.143:0/3056696442 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7facf40b86e0 con 0x55deb09ec5d0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face800f8a0 con 0x55deb09ec5d0 2024-09-10T10:51:50.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7face17fa640 1 -- 172.21.15.143:0/3056696442 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7facf403b460 con 0x55deb09ec5d0 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 10 client.16923 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55deb0a75780 con 0x55deb09ec5d0 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7face17fa640 1 -- 172.21.15.143:0/3056696442 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7face800f8a0 con 0x55deb09ec5d0 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7face17fa640 1 -- 172.21.15.143:0/3056696442 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7facf40be040 con 0x55deb09ec5d0 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 10 client.16923 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 10 client.16923 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 20 client.16923 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a75e50 0x55deb0a78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 4 client.16923 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deb0959d70 con 0x55deb0a75e50 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.470+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a75e50 0x55deb0a78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.471+0000 7facfd0f5640 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a75e50 0x55deb0a78260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=832cf3806b9b3518 server_cookie=6304e6bba17e5283 in_seq=0 out_seq=0 2024-09-10T10:51:50.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.471+0000 7face17fa640 10 client.16923 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:50.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7face17fa640 1 -- 172.21.15.143:0/3056696442 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deb0959d70 con 0x55deb0a75e50 2024-09-10T10:51:50.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7face17fa640 10 client.16923 handle_command_reply: tid=0 2024-09-10T10:51:50.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 client.16923 shutdown 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 msgr2=0x55deb09fa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09fa980 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7facf40ed5d0 tx=0x7facf4031a10 comp rx=0 tx=0).stop 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 shutdown_connections 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55deb0a75e50 0x55deb0a78260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb09f7370 0x55deb0a73e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7face17fa640 0 client.16923 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55deb09edb40 0x55deb09faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 --2- 172.21.15.143:0/3056696442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55deb09ec5d0 0x55deb09fa980 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 >> 172.21.15.143:0/3056696442 conn(0x55deb09b5310 msgr2=0x55deb09f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 shutdown_connections 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 1 -- 172.21.15.143:0/3056696442 wait complete. 2024-09-10T10:51:50.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.473+0000 7fad01f33740 20 client.16923 trim_cache size 0 max 16384 2024-09-10T10:51:50.482 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7face40a2c00 msgr2=0x7face40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7face40a2c00 0x7face40a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7facf0010c40 tx=0x7facf000c000 comp rx=0 tx=0).stop 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 msgr2=0x7facf81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf81983a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7facf4005940 tx=0x7facf40042f0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 shutdown_connections 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7face40a2c00 0x7face40a50c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facf810c170 0x7facf819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facf8103660 0x7facf81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 --2- 172.21.15.143:0/3376942958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facf8102d40 0x7facf81983a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 >> 172.21.15.143:0/3376942958 conn(0x7facf80fc7a0 msgr2=0x7facf80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 shutdown_connections 2024-09-10T10:51:50.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:50.474+0000 7facffb5a640 1 -- 172.21.15.143:0/3376942958 wait complete. 2024-09-10T10:51:50.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:50 smithi178 ceph-mon[34813]: pgmap v554: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-10T10:51:50.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:50 smithi144 ceph-mon[34230]: pgmap v554: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-10T10:51:51.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:50 smithi143 ceph-mon[31057]: pgmap v554: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-10T10:51:51.303 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:51:51.552 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:52.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.016+0000 7fa2505ad640 1 -- 172.21.15.143:0/151159922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2481027a0 msgr2=0x7fa2481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.016+0000 7fa2505ad640 1 --2- 172.21.15.143:0/151159922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2481027a0 0x7fa2481067f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa238009900 tx=0x7fa23802a410 comp rx=0 tx=0).stop 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.016+0000 7fa2505ad640 1 -- 172.21.15.143:0/151159922 shutdown_connections 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.016+0000 7fa2505ad640 1 --2- 172.21.15.143:0/151159922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa248106f20 0x7fa24810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.016+0000 7fa2505ad640 1 --2- 172.21.15.143:0/151159922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2481027a0 0x7fa2481067f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.016+0000 7fa2505ad640 1 --2- 172.21.15.143:0/151159922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa248101df0 0x7fa2481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 -- 172.21.15.143:0/151159922 >> 172.21.15.143:0/151159922 conn(0x7fa2480fd8f0 msgr2=0x7fa2480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 -- 172.21.15.143:0/151159922 shutdown_connections 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 -- 172.21.15.143:0/151159922 wait complete. 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 Processor -- start 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 -- start start 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa248101df0 0x7fa2481983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.017+0000 7fa2505ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2481027a0 0x7fa2481988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa2505ad640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 0x7fa24819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa2505ad640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa248111e40 con 0x7fa248106f20 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa2505ad640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa248111cc0 con 0x7fa2481027a0 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa2505ad640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa248111fc0 con 0x7fa248101df0 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 0x7fa24819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 0x7fa24819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51466/0 (socket says 172.21.15.143:51466) 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 -- 172.21.15.143:0/3791034900 learned_addr learned my addr 172.21.15.143:0/3791034900 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24e349640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa248101df0 0x7fa2481983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24db48640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2481027a0 0x7fa2481988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 -- 172.21.15.143:0/3791034900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa248101df0 msgr2=0x7fa2481983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa248101df0 0x7fa2481983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 -- 172.21.15.143:0/3791034900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2481027a0 msgr2=0x7fa2481988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2481027a0 0x7fa2481988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 -- 172.21.15.143:0/3791034900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24819d350 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa24eb4a640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 0x7fa24819cc70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa24000eb10 tx=0x7fa24000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa23f7fe640 1 -- 172.21.15.143:0/3791034900 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240004b20 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa23f7fe640 1 -- 172.21.15.143:0/3791034900 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa240004cc0 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24819d640 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.018+0000 7fa23f7fe640 1 -- 172.21.15.143:0/3791034900 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240010690 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.019+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a4f20 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.021+0000 7fa23f7fe640 1 -- 172.21.15.143:0/3791034900 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa240005020 con 0x7fa248106f20 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.021+0000 7fa252986740 1 Processor -- start 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.021+0000 7fa252986740 1 -- start start 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa252986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d3b40 0x562f367d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa252986740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 0x562f367d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa252986740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa252986740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562f367382f0 con 0x562f367d25d0 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa252986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f366dfcf0 con 0x562f367d3b40 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa252986740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f367342b0 con 0x562f367dd370 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa23f7fe640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2340a2b80 0x7fa2340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 0x562f367d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 0x562f367d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51468/0 (socket says 172.21.15.143:51468) 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 -- 172.21.15.143:0/149868845 learned_addr learned my addr 172.21.15.143:0/149868845 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24db48640 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24db48640 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d3b40 0x562f367d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24eb4a640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2340a2b80 0x7fa2340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 -- 172.21.15.143:0/149868845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 msgr2=0x562f367df760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367df760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 -- 172.21.15.143:0/149868845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d3b40 msgr2=0x562f367d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d3b40 0x562f367d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24e349640 1 -- 172.21.15.143:0/149868845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f366c43a0 con 0x562f367d25d0 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa24db48640 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d3b40 0x562f367d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.022+0000 7fa23f7fe640 1 -- 172.21.15.143:0/3791034900 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2400cf8b0 con 0x7fa248106f20 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.023+0000 7fa24e349640 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 0x562f367d2a10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa24400bd10 tx=0x7fa2440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e279439784a7cabc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.023+0000 7fa24eb4a640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2340a2b80 0x7fa2340a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa2400cd0a0 tx=0x7fa240097a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.023+0000 7fa23cff9640 1 -- 172.21.15.143:0/149868845 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244004030 con 0x562f367d25d0 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.023+0000 7fa23cff9640 1 -- 172.21.15.143:0/149868845 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2440035f0 con 0x562f367d25d0 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.023+0000 7fa23cff9640 1 -- 172.21.15.143:0/149868845 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440117b0 con 0x562f367d25d0 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 -- 172.21.15.143:0/149868845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 msgr2=0x562f367d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 0x562f367d2a10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa24400bd10 tx=0x7fa2440107a0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 -- 172.21.15.143:0/149868845 shutdown_connections 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367df760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d25d0 0x562f367d2a10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 --2- 172.21.15.143:0/149868845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d3b40 0x562f367d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 -- 172.21.15.143:0/149868845 >> 172.21.15.143:0/149868845 conn(0x562f3679b310 msgr2=0x562f367cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 -- 172.21.15.143:0/149868845 shutdown_connections 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 -- 172.21.15.143:0/149868845 wait complete. 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 -- start start 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.024+0000 7fa252986740 1 Processor -- start 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 0x562f367da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367dd370 0x562f367da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562f367e0070 con 0x562f367dd370 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f366f3060 con 0x562f367d25d0 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f367e01f0 con 0x562f367d3b40 2024-09-10T10:51:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa252986740 10 client.0 _ll_register_callbacks cb 0x562f364468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24e349640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 0x562f367da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367dd370 0x562f367da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24e349640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 0x562f367da060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:60738/0 (socket says 172.21.15.143:60738) 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24e349640 1 -- 172.21.15.143:0/194032252 learned_addr learned my addr 172.21.15.143:0/194032252 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 -- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 msgr2=0x562f367da060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 0x562f367da060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 -- 172.21.15.143:0/194032252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 msgr2=0x562f367dcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 -- 172.21.15.143:0/194032252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f364a93c0 con 0x562f367dd370 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24e349640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 0x562f367da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa24db48640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367dd370 0x562f367da7f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa23802aa00 tx=0x7fa238009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4f47a92a99f768 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.025+0000 7fa231ffb640 1 -- 172.21.15.143:0/194032252 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238035040 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa231ffb640 1 -- 172.21.15.143:0/194032252 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2380410f0 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2440035f0 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa231ffb640 1 -- 172.21.15.143:0/194032252 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa238041e50 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 10 client.16935 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562f367ebb60 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa231ffb640 1 -- 172.21.15.143:0/194032252 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2440035f0 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa231ffb640 1 -- 172.21.15.143:0/194032252 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa2380035f0 con 0x562f367dd370 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 10 client.16935 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 10 client.16935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 20 client.16935 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f367ecf70 0x562f367ef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 4 client.16935 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:52.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f36745860 con 0x562f367ecf70 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.026+0000 7fa24e349640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f367ecf70 0x562f367ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.027+0000 7fa24e349640 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f367ecf70 0x562f367ef380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65952c3732c09d57 server_cookie=3d750524a5762e09 in_seq=0 out_seq=0 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.027+0000 7fa231ffb640 10 client.16935 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa231ffb640 1 -- 172.21.15.143:0/194032252 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f36745860 con 0x562f367ecf70 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa231ffb640 10 client.16935 handle_command_reply: tid=0 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 client.16935 shutdown 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367dd370 msgr2=0x562f367da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367dd370 0x562f367da7f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa23802aa00 tx=0x7fa238009260 comp rx=0 tx=0).stop 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 shutdown_connections 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f367ecf70 0x562f367ef380 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa231ffb640 0 client.16935 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367dd370 0x562f367da7f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d3b40 0x562f367da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 --2- 172.21.15.143:0/194032252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 >> 172.21.15.143:0/194032252 conn(0x562f3679b310 msgr2=0x562f367d4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 shutdown_connections 2024-09-10T10:51:52.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 1 -- 172.21.15.143:0/194032252 wait complete. 2024-09-10T10:51:52.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.028+0000 7fa252986740 20 client.16935 trim_cache size 0 max 16384 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 Processor -- start 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 -- start start 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367dd370 0x562f367d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ac8b0 con 0x562f367d3b40 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ae3f0 con 0x562f367dd370 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.043+0000 7fa252986740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481acf80 con 0x562f367d25d0 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367d5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51488/0 (socket says 172.21.15.143:51488) 2024-09-10T10:51:52.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 -- 172.21.15.143:0/1096636735 learned_addr learned my addr 172.21.15.143:0/1096636735 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24db48640 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24db48640 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367dd370 0x562f367d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 -- 172.21.15.143:0/1096636735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d25d0 msgr2=0x562f367dcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 -- 172.21.15.143:0/1096636735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367dd370 msgr2=0x562f367d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367dd370 0x562f367d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 -- 172.21.15.143:0/1096636735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f367d85b0 con 0x562f367d3b40 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa24e349640 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367d5120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa244010ca0 tx=0x7fa24400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdca9631276de0be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa233fff640 1 -- 172.21.15.143:0/1096636735 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244005240 con 0x562f367d3b40 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa233fff640 1 -- 172.21.15.143:0/1096636735 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2440053e0 con 0x562f367d3b40 2024-09-10T10:51:52.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.044+0000 7fa233fff640 1 -- 172.21.15.143:0/1096636735 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2440116a0 con 0x562f367d3b40 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 -- 172.21.15.143:0/1096636735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 msgr2=0x562f367d5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367d5120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa244010ca0 tx=0x7fa24400b790 comp rx=0 tx=0).stop 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 -- 172.21.15.143:0/1096636735 shutdown_connections 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367dd370 0x562f367d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367d5120 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 --2- 172.21.15.143:0/1096636735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367d25d0 0x562f367dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 -- 172.21.15.143:0/1096636735 >> 172.21.15.143:0/1096636735 conn(0x562f3679b310 msgr2=0x562f367c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 -- 172.21.15.143:0/1096636735 shutdown_connections 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.045+0000 7fa252986740 1 -- 172.21.15.143:0/1096636735 wait complete. 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 -- start start 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 Processor -- start 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481add20 con 0x562f367d3b40 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ad650 con 0x562f367d25d0 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481aa890 con 0x562f367dd370 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa252986740 10 client.0 _ll_register_callbacks cb 0x7fa2380040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367e5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51502/0 (socket says 172.21.15.143:51502) 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 -- 172.21.15.143:0/1985415325 learned_addr learned my addr 172.21.15.143:0/1985415325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24db48640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24db48640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 -- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 msgr2=0x562f367e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 -- 172.21.15.143:0/1985415325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 msgr2=0x562f367e5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367e5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.046+0000 7fa24e349640 1 -- 172.21.15.143:0/1985415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f367e1610 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa24e349640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367e5da0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa2440107e0 tx=0x7fa244005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6482ac46ebef8f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa231ffb640 1 -- 172.21.15.143:0/1985415325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244034070 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa231ffb640 1 -- 172.21.15.143:0/1985415325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa2440037f0 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2440053e0 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa231ffb640 1 -- 172.21.15.143:0/1985415325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa244042050 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa252986740 10 client.16941 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562f368578d0 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa231ffb640 1 -- 172.21.15.143:0/1985415325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2440053e0 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.047+0000 7fa231ffb640 1 -- 172.21.15.143:0/1985415325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa244039b40 con 0x562f367d3b40 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa252986740 10 client.16941 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa252986740 10 client.16941 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:51:52.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa252986740 20 client.16941 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa252986740 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f36857db0 0x562f3685a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa252986740 4 client.16941 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f3673fd70 con 0x562f36857db0 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa24db48640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f36857db0 0x562f3685a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa24db48640 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f36857db0 0x562f3685a1d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94e7f1b83bbfad1e server_cookie=5440d95b09f9c253 in_seq=0 out_seq=0 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.048+0000 7fa231ffb640 10 client.16941 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa231ffb640 1 -- 172.21.15.143:0/1985415325 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f3673fd70 con 0x562f36857db0 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa231ffb640 10 client.16941 handle_command_reply: tid=0 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 client.16941 shutdown 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 msgr2=0x562f367e5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367e5da0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa2440107e0 tx=0x7fa244005c20 comp rx=0 tx=0).stop 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 shutdown_connections 2024-09-10T10:51:52.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x562f36857db0 0x562f3685a1d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f367dd370 0x562f367e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f367d3b40 0x562f367e5da0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 --2- 172.21.15.143:0/1985415325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f367d25d0 0x562f367e5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 >> 172.21.15.143:0/1985415325 conn(0x562f3679b310 msgr2=0x562f367cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa231ffb640 0 client.16941 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 shutdown_connections 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 1 -- 172.21.15.143:0/1985415325 wait complete. 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.049+0000 7fa252986740 20 client.16941 trim_cache size 0 max 16384 2024-09-10T10:51:52.055 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:52.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2340a2b80 msgr2=0x7fa2340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2340a2b80 0x7fa2340a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa2400cd0a0 tx=0x7fa240097a50 comp rx=0 tx=0).stop 2024-09-10T10:51:52.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 msgr2=0x7fa24819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 0x7fa24819cc70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa24000eb10 tx=0x7fa24000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 shutdown_connections 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa2340a2b80 0x7fa2340a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa248106f20 0x7fa24819cc70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2481027a0 0x7fa2481988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 --2- 172.21.15.143:0/3791034900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa248101df0 0x7fa2481983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 >> 172.21.15.143:0/3791034900 conn(0x7fa2480fd8f0 msgr2=0x7fa2480fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 shutdown_connections 2024-09-10T10:51:52.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.050+0000 7fa2505ad640 1 -- 172.21.15.143:0/3791034900 wait complete. 2024-09-10T10:51:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd3daffd640 1 -- 172.21.15.143:0/2043761205 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557a29688d20 con 0x557a297a2930 2024-09-10T10:51:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd3daffd640 10 client.26293 handle_command_reply: tid=0 2024-09-10T10:51:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 client.26293 shutdown 2024-09-10T10:51:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 msgr2=0x557a2972ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a2972ec70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400a010 tx=0x7fd3f4009580 comp rx=0 tx=0).stop 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 shutdown_connections 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a29726940 0x557a2972bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557a2971c080 0x557a2972f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557a297a2930 0x557a297a4d20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 --2- 172.21.15.143:0/2043761205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a2971b760 0x557a2972ec70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd3daffd640 0 client.26293 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.226+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 >> 172.21.15.143:0/2043761205 conn(0x557a29717cf0 msgr2=0x557a29724050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.227+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 shutdown_connections 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.227+0000 7fd403be7740 1 -- 172.21.15.143:0/2043761205 wait complete. 2024-09-10T10:51:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.227+0000 7fd403be7740 20 client.26293 trim_cache size 0 max 16384 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 35, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 18, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "ddc3abbd": { 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 18, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:51:52.230 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 140.1, 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:51:52.231 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd3e80a2c00 msgr2=0x7fd3e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd3e80a2c00 0x7fd3e80a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4031440 tx=0x7fd3e4005fd0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 msgr2=0x7fd3fc1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 0x7fd3fc1014b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d750 tx=0x7fd3ec00dc10 comp rx=0 tx=0).stop 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 shutdown_connections 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fd3e80a2c00 0x7fd3e80a50c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3fc108300 0x7fd3fc1014b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3fc103780 0x7fd3fc100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 --2- 172.21.15.143:0/3125095837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3fc102dd0 0x7fd3fc100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 >> 172.21.15.143:0/3125095837 conn(0x7fd3fc0fc7d0 msgr2=0x7fd3fc105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 shutdown_connections 2024-09-10T10:51:52.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:52.228+0000 7fd40180e640 1 -- 172.21.15.143:0/3125095837 wait complete. 2024-09-10T10:51:52.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:52 smithi178 ceph-mon[34813]: pgmap v555: 129 pgs: 129 active+clean; 430 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-10T10:51:52.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:52 smithi144 ceph-mon[34230]: pgmap v555: 129 pgs: 129 active+clean; 430 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-10T10:51:53.023 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:52 smithi143 ceph-mon[31057]: pgmap v555: 129 pgs: 129 active+clean; 430 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-10T10:51:53.102 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ddc3abbd', quiesce took 12.6 seconds. Will release after: 7.500000000000002 2024-09-10T10:51:53.592 INFO:tasks.workunit.client.0.smithi143.stdout: 96 3326 12 1935 37 1089 20 2024-09-10T10:51:53.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:51:53.979 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:51:54.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:53 smithi143 ceph-mon[31057]: pgmap v556: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-10T10:51:54.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:53 smithi178 ceph-mon[34813]: pgmap v556: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-10T10:51:54.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:53 smithi144 ceph-mon[34230]: pgmap v556: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-10T10:51:54.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 -- 172.21.15.143:0/295990724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102ef0 msgr2=0x7f71d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 --2- 172.21.15.143:0/295990724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102ef0 0x7f71d010da40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009ca0 tx=0x7f71c802a950 comp rx=0 tx=0).stop 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 -- 172.21.15.143:0/295990724 shutdown_connections 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 --2- 172.21.15.143:0/295990724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102ef0 0x7f71d010da40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 --2- 172.21.15.143:0/295990724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d0102570 0x7f71d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 --2- 172.21.15.143:0/295990724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 -- 172.21.15.143:0/295990724 >> 172.21.15.143:0/295990724 conn(0x7f71d00fc7d0 msgr2=0x7f71d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 -- 172.21.15.143:0/295990724 shutdown_connections 2024-09-10T10:51:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.519+0000 7f71d855c640 1 -- 172.21.15.143:0/295990724 wait complete. 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 Processor -- start 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 -- start start 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102570 0x7f71d01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d0102ef0 0x7f71d01a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0110e30 con 0x7f71d0102570 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0110cb0 con 0x7f71d0102ef0 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.520+0000 7f71d855c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0110fb0 con 0x7f71d0100e20 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d5af7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102570 0x7f71d01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d5af7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102570 0x7f71d01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51518/0 (socket says 172.21.15.143:51518) 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:60768/0 (socket says 172.21.15.143:60768) 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d5af7640 1 -- 172.21.15.143:0/2282995484 learned_addr learned my addr 172.21.15.143:0/2282995484 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d6af9640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d0102ef0 0x7f71d01a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 -- 172.21.15.143:0/2282995484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d0102ef0 msgr2=0x7f71d01a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d0102ef0 0x7f71d01a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 -- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102570 msgr2=0x7f71d01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102570 0x7f71d01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 -- 172.21.15.143:0/2282995484 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d01a4860 con 0x7f71d0100e20 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71d62f8640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d01992a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00e9f0 tx=0x7f71cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.521+0000 7f71c77fe640 1 -- 172.21.15.143:0/2282995484 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00ccc0 con 0x7f71d0100e20 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.522+0000 7f71c77fe640 1 -- 172.21.15.143:0/2282995484 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc0108e0 con 0x7f71d0100e20 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.522+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d0197090 con 0x7f71d0100e20 2024-09-10T10:51:54.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.522+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d0197510 con 0x7f71d0100e20 2024-09-10T10:51:54.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.522+0000 7f71c77fe640 1 -- 172.21.15.143:0/2282995484 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc018880 con 0x7f71d0100e20 2024-09-10T10:51:54.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.525+0000 7f71c77fe640 1 -- 172.21.15.143:0/2282995484 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f71cc010a90 con 0x7f71d0100e20 2024-09-10T10:51:54.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.525+0000 7f71da935740 1 Processor -- start 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.525+0000 7f71da935740 1 -- start start 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71da935740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71da935740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf60de0 0x55e3ccf61220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71da935740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71da935740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3ccebe2f0 con 0x55e3ccf62350 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71da935740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3cce65cf0 con 0x55e3ccf6bb80 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71da935740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e3cceba2b0 con 0x55e3ccf60de0 2024-09-10T10:51:54.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf60de0 0x55e3ccf61220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d5af7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d6af9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d5af7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf6df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:51070/0 (socket says 172.21.15.143:51070) 2024-09-10T10:51:54.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d5af7640 1 -- 172.21.15.143:0/1133365141 learned_addr learned my addr 172.21.15.143:0/1133365141 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d62f8640 1 -- 172.21.15.143:0/1133365141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 msgr2=0x55e3ccf6df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.526+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf6df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71d62f8640 1 -- 172.21.15.143:0/1133365141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 msgr2=0x55e3ccf608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71c77fe640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71bc0a2c00 0x7f71bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71c77fe640 1 -- 172.21.15.143:0/2282995484 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71cc0cf6b0 con 0x7f71d0100e20 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71d62f8640 1 -- 172.21.15.143:0/1133365141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3cce4a3a0 con 0x55e3ccf60de0 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71d6af9640 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:54.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71d5af7640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71bc0a2c00 0x7f71bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf60de0 0x55e3ccf61220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0ccd90 tx=0x7f71cc009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=133a5757698f5b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71c4ff9640 1 -- 172.21.15.143:0/1133365141 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc09b560 con 0x55e3ccf60de0 2024-09-10T10:51:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71c4ff9640 1 -- 172.21.15.143:0/1133365141 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc010e00 con 0x55e3ccf60de0 2024-09-10T10:51:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.527+0000 7f71c4ff9640 1 -- 172.21.15.143:0/1133365141 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0cf3b0 con 0x55e3ccf60de0 2024-09-10T10:51:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 -- 172.21.15.143:0/1133365141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf60de0 msgr2=0x55e3ccf61220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71d5af7640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71bc0a2c00 0x7f71bc0a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e3ccf61880 tx=0x7f71c000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf60de0 0x55e3ccf61220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0ccd90 tx=0x7f71cc009700 comp rx=0 tx=0).stop 2024-09-10T10:51:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 -- 172.21.15.143:0/1133365141 shutdown_connections 2024-09-10T10:51:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf6df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf60de0 0x55e3ccf61220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 --2- 172.21.15.143:0/1133365141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.528+0000 7f71da935740 1 -- 172.21.15.143:0/1133365141 >> 172.21.15.143:0/1133365141 conn(0x55e3ccf21310 msgr2=0x55e3ccf5d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.529+0000 7f71da935740 1 -- 172.21.15.143:0/1133365141 shutdown_connections 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.529+0000 7f71da935740 1 -- 172.21.15.143:0/1133365141 wait complete. 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.529+0000 7f71da935740 1 -- start start 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.529+0000 7f71da935740 1 Processor -- start 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 0x55e3ccf6b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf62350 0x55e3ccf68850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3ccf6e820 con 0x55e3ccf6bb80 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3cce79060 con 0x55e3ccf60de0 2024-09-10T10:51:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e3ccf6e9a0 con 0x55e3ccf62350 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71da935740 10 client.0 _ll_register_callbacks cb 0x55e3ccbcc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 0x55e3ccf6b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d6af9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 0x55e3ccf6b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:51072/0 (socket says 172.21.15.143:51072) 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 -- 172.21.15.143:0/1662837269 learned_addr learned my addr 172.21.15.143:0/1662837269 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf62350 0x55e3ccf68850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 -- 172.21.15.143:0/1662837269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf62350 msgr2=0x55e3ccf68850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf62350 0x55e3ccf68850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 -- 172.21.15.143:0/1662837269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf6bb80 msgr2=0x55e3ccf68e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf68e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d62f8640 1 -- 172.21.15.143:0/1662837269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ccc2f3c0 con 0x55e3ccf60de0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.530+0000 7f71d6af9640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 0x55e3ccf6b5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f71cc09b900 tx=0x7f71cc09b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43af76fa5ac4a61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:51:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1662837269 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0b3020 con 0x55e3ccf60de0 2024-09-10T10:51:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e3ccecb860 con 0x55e3ccf60de0 2024-09-10T10:51:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1662837269 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71cc010430 con 0x55e3ccf60de0 2024-09-10T10:51:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1662837269 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f71cc09b560 con 0x55e3ccf60de0 2024-09-10T10:51:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71da935740 10 client.26282 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3ccf7a330 con 0x55e3ccf60de0 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1662837269 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc09bc00 con 0x55e3ccf60de0 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.531+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1662837269 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f71cc097950 con 0x55e3ccf60de0 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71da935740 10 client.26282 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71da935740 10 client.26282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71da935740 20 client.26282 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71da935740 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccf7b920 0x55e3ccf7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71da935740 4 client.26282 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ccecb860 con 0x55e3ccf7b920 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.532+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccf7b920 0x55e3ccf7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.533+0000 7f71b9ffb640 10 client.26282 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.533+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccf7b920 0x55e3ccf7dd40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e0390c0a01b0ba4 server_cookie=1abc8e2a374134dd in_seq=0 out_seq=0 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.533+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1662837269 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3ccecb860 con 0x55e3ccf7b920 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.533+0000 7f71b9ffb640 10 client.26282 handle_command_reply: tid=0 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.533+0000 7f71da935740 1 client.26282 shutdown 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 msgr2=0x55e3ccf6b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 0x55e3ccf6b5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f71cc09b900 tx=0x7f71cc09b870 comp rx=0 tx=0).stop 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 shutdown_connections 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf62350 0x55e3ccf68850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccf7b920 0x55e3ccf7dd40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 --2- 172.21.15.143:0/1662837269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf60de0 0x55e3ccf6b5a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 >> 172.21.15.143:0/1662837269 conn(0x55e3ccf21310 msgr2=0x55e3ccf63140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71b9ffb640 0 client.26282 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 shutdown_connections 2024-09-10T10:51:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 1 -- 172.21.15.143:0/1662837269 wait complete. 2024-09-10T10:51:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.534+0000 7f71da935740 20 client.26282 trim_cache size 0 max 16384 2024-09-10T10:51:54.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.559+0000 7f71da935740 1 Processor -- start 2024-09-10T10:51:54.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.559+0000 7f71da935740 1 -- start start 2024-09-10T10:51:54.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.559+0000 7f71da935740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf63930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.559+0000 7f71da935740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf63e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.559+0000 7f71da935740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf835f0 0x55e3ccf83aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71da935740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01ac940 con 0x55e3ccf62350 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71da935740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01ae480 con 0x55e3ccf835f0 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71da935740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01ad010 con 0x55e3ccf6bb80 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf63930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d5af7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf63e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf63930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51542/0 (socket says 172.21.15.143:51542) 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 -- 172.21.15.143:0/1861916009 learned_addr learned my addr 172.21.15.143:0/1861916009 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d6af9640 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf835f0 0x55e3ccf83aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 -- 172.21.15.143:0/1861916009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf6bb80 msgr2=0x55e3ccf63e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf63e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 -- 172.21.15.143:0/1861916009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf835f0 msgr2=0x55e3ccf83aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf835f0 0x55e3ccf83aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 -- 172.21.15.143:0/1861916009 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ccf5e170 con 0x55e3ccf62350 2024-09-10T10:51:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d6af9640 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf835f0 0x55e3ccf83aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf63e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf63930 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f71cc099eb0 tx=0x7f71cc009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7745ecf542b03f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.560+0000 7f71bbfff640 1 -- 172.21.15.143:0/1861916009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0b3070 con 0x55e3ccf62350 2024-09-10T10:51:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.561+0000 7f71bbfff640 1 -- 172.21.15.143:0/1861916009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc09bc00 con 0x55e3ccf62350 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.561+0000 7f71bbfff640 1 -- 172.21.15.143:0/1861916009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc09ad20 con 0x55e3ccf62350 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.561+0000 7f71da935740 1 -- 172.21.15.143:0/1861916009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 msgr2=0x55e3ccf63930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.561+0000 7f71da935740 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf63930 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f71cc099eb0 tx=0x7f71cc009700 comp rx=0 tx=0).stop 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 -- 172.21.15.143:0/1861916009 shutdown_connections 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf835f0 0x55e3ccf83aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf63e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 --2- 172.21.15.143:0/1861916009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf63930 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 -- 172.21.15.143:0/1861916009 >> 172.21.15.143:0/1861916009 conn(0x55e3ccf21310 msgr2=0x55e3ccf53d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 -- 172.21.15.143:0/1861916009 shutdown_connections 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 -- 172.21.15.143:0/1861916009 wait complete. 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 -- start start 2024-09-10T10:51:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.562+0000 7f71da935740 1 Processor -- start 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf6fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf835f0 0x55e3ccf705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01addb0 con 0x55e3ccf62350 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01ad6e0 con 0x55e3ccf6bb80 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01aa920 con 0x55e3ccf835f0 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf6fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71da935740 10 client.0 _ll_register_callbacks cb 0x7f71c8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf6fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51544/0 (socket says 172.21.15.143:51544) 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 -- 172.21.15.143:0/1598985019 learned_addr learned my addr 172.21.15.143:0/1598985019 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d6af9640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf835f0 0x55e3ccf705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 -- 172.21.15.143:0/1598985019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf835f0 msgr2=0x55e3ccf705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf835f0 0x55e3ccf705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 -- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 msgr2=0x55e3ccf700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d62f8640 1 -- 172.21.15.143:0/1598985019 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ccfe5720 con 0x55e3ccf62350 2024-09-10T10:51:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.563+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71d62f8640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf6fb70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f71cc098580 tx=0x7f71cc097fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d30975a4da3d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:51:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1598985019 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc082060 con 0x55e3ccf62350 2024-09-10T10:51:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1598985019 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f71cc0d5620 con 0x55e3ccf62350 2024-09-10T10:51:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71cc09bc00 con 0x55e3ccf62350 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1598985019 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f71cc0a9050 con 0x55e3ccf62350 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71da935740 10 client.16959 fetch_fsmap learned FSMap version 35 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3ccfe5720 con 0x55e3ccf62350 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1598985019 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc09bc00 con 0x55e3ccf62350 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.564+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1598985019 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f71cc09ab10 con 0x55e3ccf62350 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 10 client.16959 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 20 client.16959 populate_metadata read hostname 'smithi143' 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccfe6cc0 0x55e3ccfe90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 4 client.16959 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3ccec5d70 con 0x55e3ccfe6cc0 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccfe6cc0 0x55e3ccfe90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71d5af7640 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccfe6cc0 0x55e3ccfe90b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb4aeff7fc791996 server_cookie=ee4a2eb9d75e7654 in_seq=0 out_seq=0 2024-09-10T10:51:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71b9ffb640 10 client.16959 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71b9ffb640 1 -- 172.21.15.143:0/1598985019 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3ccec5d70 con 0x55e3ccfe6cc0 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71b9ffb640 10 client.16959 handle_command_reply: tid=0 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.565+0000 7f71da935740 1 client.16959 shutdown 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 msgr2=0x55e3ccf6fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf6fb70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f71cc098580 tx=0x7f71cc097fd0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 shutdown_connections 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e3ccf835f0 0x55e3ccf705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55e3ccfe6cc0 0x55e3ccfe90b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3ccf6bb80 0x55e3ccf700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 --2- 172.21.15.143:0/1598985019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e3ccf62350 0x55e3ccf6fb70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71b9ffb640 0 client.16959 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:51:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 >> 172.21.15.143:0/1598985019 conn(0x55e3ccf21310 msgr2=0x55e3ccf82e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 shutdown_connections 2024-09-10T10:51:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 1 -- 172.21.15.143:0/1598985019 wait complete. 2024-09-10T10:51:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.566+0000 7f71da935740 20 client.16959 trim_cache size 0 max 16384 2024-09-10T10:51:54.572 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71bc0a2c00 msgr2=0x7f71bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71bc0a2c00 0x7f71bc0a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e3ccf61880 tx=0x7f71c000b480 comp rx=0 tx=0).stop 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 msgr2=0x7f71d01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d01992a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00e9f0 tx=0x7f71cc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 shutdown_connections 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f71bc0a2c00 0x7f71bc0a50c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d0102ef0 0x7f71d01a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71d0102570 0x7f71d01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 --2- 172.21.15.143:0/2282995484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71d0100e20 0x7f71d01992a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 >> 172.21.15.143:0/2282995484 conn(0x7f71d00fc7d0 msgr2=0x7f71d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 shutdown_connections 2024-09-10T10:51:54.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:51:54.567+0000 7f71d855c640 1 -- 172.21.15.143:0/2282995484 wait complete. 2024-09-10T10:51:56.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:56 smithi178 ceph-mon[34813]: pgmap v557: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-10T10:51:56.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:56 smithi144 ceph-mon[34230]: pgmap v557: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-10T10:51:57.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:56 smithi143 ceph-mon[31057]: pgmap v557: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-10T10:51:58.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:51:58 smithi178 ceph-mon[34813]: pgmap v558: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T10:51:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:51:58 smithi144 ceph-mon[34230]: pgmap v558: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T10:51:59.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:51:58 smithi143 ceph-mon[31057]: pgmap v558: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T10:51:59.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:00.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:52:00.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:00 smithi178 ceph-mon[34813]: pgmap v559: 129 pgs: 129 active+clean; 423 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 703 KiB/s wr, 149 op/s 2024-09-10T10:52:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:00 smithi144 ceph-mon[34230]: pgmap v559: 129 pgs: 129 active+clean; 423 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 703 KiB/s wr, 149 op/s 2024-09-10T10:52:01.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:00 smithi143 ceph-mon[31057]: pgmap v559: 129 pgs: 129 active+clean; 423 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 703 KiB/s wr, 149 op/s 2024-09-10T10:52:02.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:01 smithi143 ceph-mon[31057]: pgmap v560: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 957 KiB/s wr, 143 op/s 2024-09-10T10:52:02.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:01 smithi178 ceph-mon[34813]: pgmap v560: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 957 KiB/s wr, 143 op/s 2024-09-10T10:52:02.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:01 smithi144 ceph-mon[34230]: pgmap v560: 129 pgs: 129 active+clean; 427 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 957 KiB/s wr, 143 op/s 2024-09-10T10:52:03.020 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:52:03.020 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.020 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.020 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.020 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi144 to 176.0M 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi178 to 175.9M 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: Unable to set osd_memory_target on smithi144 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T10:52:03.021 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:02 smithi143 ceph-mon[31057]: Unable to set osd_memory_target on smithi178 to 184547328: error parsing value: Value '184547328' is below minimum 939524096 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi144 to 176.0M 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi178 to 175.9M 2024-09-10T10:52:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: Unable to set osd_memory_target on smithi144 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T10:52:03.176 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:02 smithi178 ceph-mon[34813]: Unable to set osd_memory_target on smithi178 to 184547328: error parsing value: Value '184547328' is below minimum 939524096 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi144 to 176.0M 2024-09-10T10:52:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi178 to 175.9M 2024-09-10T10:52:03.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: Unable to set osd_memory_target on smithi144 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-10T10:52:03.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:02 smithi144 ceph-mon[34230]: Unable to set osd_memory_target on smithi178 to 184547328: error parsing value: Value '184547328' is below minimum 939524096 2024-09-10T10:52:03.592 INFO:tasks.workunit.client.0.smithi143.stdout: 96 9625 0 6455 0 3722 0 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 -- 172.21.15.143:0/3632600873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c100a70 msgr2=0x7fdd2c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 --2- 172.21.15.143:0/3632600873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c100a70 0x7fdd2c100ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009930 tx=0x7fdd1402a420 comp rx=0 tx=0).stop 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 -- 172.21.15.143:0/3632600873 shutdown_connections 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 --2- 172.21.15.143:0/3632600873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c101410 0x7fdd2c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 --2- 172.21.15.143:0/3632600873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c100a70 0x7fdd2c100ed0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 --2- 172.21.15.143:0/3632600873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c106a70 0x7fdd2c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 -- 172.21.15.143:0/3632600873 >> 172.21.15.143:0/3632600873 conn(0x7fdd2c0fc7d0 msgr2=0x7fdd2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 -- 172.21.15.143:0/3632600873 shutdown_connections 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.018+0000 7fdd31507640 1 -- 172.21.15.143:0/3632600873 wait complete. 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.019+0000 7fdd31507640 1 Processor -- start 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.019+0000 7fdd31507640 1 -- start start 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd31507640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 0x7fdd2c10d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd31507640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c101410 0x7fdd2c10a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 0x7fdd2c10d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 0x7fdd2c10d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:34374/0 (socket says 172.21.15.143:34374) 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd31507640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c106a70 0x7fdd2c10ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 -- 172.21.15.143:0/2060379195 learned_addr learned my addr 172.21.15.143:0/2060379195 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c106a70 0x7fdd2c10ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c10fbf0 con 0x7fdd2c106a70 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c10fa70 con 0x7fdd2c101410 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c10fd70 con 0x7fdd2c100a70 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd227fc640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c101410 0x7fdd2c10a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 -- 172.21.15.143:0/2060379195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c101410 msgr2=0x7fdd2c10a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c101410 0x7fdd2c10a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 -- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c106a70 msgr2=0x7fdd2c10ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c106a70 0x7fdd2c10ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2affd640 1 -- 172.21.15.143:0/2060379195 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c10b5c0 con 0x7fdd2c100a70 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd227fc640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c101410 0x7fdd2c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.020+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c106a70 0x7fdd2c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.021+0000 7fdd2affd640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 0x7fdd2c10d780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00db80 tx=0x7fdd1c00b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.021+0000 7fdd28ff9640 1 -- 172.21.15.143:0/2060379195 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c017070 con 0x7fdd2c100a70 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.021+0000 7fdd28ff9640 1 -- 172.21.15.143:0/2060379195 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd1c003b70 con 0x7fdd2c100a70 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.021+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c1aaf60 con 0x7fdd2c100a70 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.021+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c1ab420 con 0x7fdd2c100a70 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.021+0000 7fdd28ff9640 1 -- 172.21.15.143:0/2060379195 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c010480 con 0x7fdd2c100a70 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.024+0000 7fdd28ff9640 1 -- 172.21.15.143:0/2060379195 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdd1c004020 con 0x7fdd2c100a70 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.024+0000 7fdd338e0740 1 Processor -- start 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.024+0000 7fdd338e0740 1 -- start start 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b10b40 0x55def5b0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b0fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 0x55def5b1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd338e0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55def5a752f0 con 0x55def5b0f5d0 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd338e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55def5a1ccf0 con 0x55def5b10b40 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd338e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55def5a712b0 con 0x55def5b1a370 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b0fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd2b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b10b40 0x55def5b0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 0x55def5b1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 0x55def5b1c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:34384/0 (socket says 172.21.15.143:34384) 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 -- 172.21.15.143:0/3973266193 learned_addr learned my addr 172.21.15.143:0/3973266193 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 -- 172.21.15.143:0/3973266193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b10b40 msgr2=0x55def5b0f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b10b40 0x55def5b0f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 -- 172.21.15.143:0/3973266193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 msgr2=0x55def5b0fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b0fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 -- 172.21.15.143:0/3973266193 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55def5a013a0 con 0x55def5b1a370 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd2affd640 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd28ff9640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd180a2b80 0x7fdd180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.025+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 0x55def5b1c760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd14006010 tx=0x7fdd14032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25d1bc7b423e250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.026+0000 7fdd217fa640 1 -- 172.21.15.143:0/3973266193 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14035040 con 0x55def5b1a370 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.026+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd180a2b80 0x7fdd180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.026+0000 7fdd217fa640 1 -- 172.21.15.143:0/3973266193 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd140318e0 con 0x55def5b1a370 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.026+0000 7fdd217fa640 1 -- 172.21.15.143:0/3973266193 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004a00 con 0x55def5b1a370 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.026+0000 7fdd28ff9640 1 -- 172.21.15.143:0/2060379195 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdd1c0cf240 con 0x7fdd2c100a70 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 -- 172.21.15.143:0/3973266193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 msgr2=0x55def5b1c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd180a2b80 0x7fdd180a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0feec0 tx=0x7fdd24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 0x55def5b1c760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd14006010 tx=0x7fdd14032f60 comp rx=0 tx=0).stop 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 -- 172.21.15.143:0/3973266193 shutdown_connections 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b1a370 0x55def5b1c760 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3973266193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b10b40 0x55def5b0f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 -- 172.21.15.143:0/3973266193 >> 172.21.15.143:0/3973266193 conn(0x55def5ad8310 msgr2=0x55def5b0bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 -- 172.21.15.143:0/3973266193 shutdown_connections 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 -- 172.21.15.143:0/3973266193 wait complete. 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 -- start start 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.027+0000 7fdd338e0740 1 Processor -- start 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b19cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b16fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b19cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b19cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59196/0 (socket says 172.21.15.143:59196) 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b17520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 -- 172.21.15.143:0/3486441804 learned_addr learned my addr 172.21.15.143:0/3486441804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55def5b1d010 con 0x55def5b0f5d0 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55def5a30060 con 0x55def5b1a370 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55def5b1d190 con 0x55def5b10b40 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd338e0740 10 client.0 _ll_register_callbacks cb 0x55def57838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b16fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b17520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 -- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 msgr2=0x55def5b16fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b16fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 -- 172.21.15.143:0/3486441804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 msgr2=0x55def5b17520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b17520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2affd640 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55def57e63c0 con 0x55def5b0f5d0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b17520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.028+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b16fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd2affd640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b19cf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000f80 tx=0x7fdd1c097aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a258ebe26e7cf93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd127fc640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd127fc640 1 -- 172.21.15.143:0/3486441804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00be60 con 0x55def5b0f5d0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd127fc640 1 -- 172.21.15.143:0/3486441804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdd1c003d20 con 0x55def5b0f5d0 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55def5a82860 con 0x55def5b0f5d0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd127fc640 1 -- 172.21.15.143:0/3486441804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd1c004310 con 0x55def5b0f5d0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd338e0740 10 client.16971 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55def5b28ad0 con 0x55def5b0f5d0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.029+0000 7fdd127fc640 1 -- 172.21.15.143:0/3486441804 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd1c09be40 con 0x55def5b0f5d0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd127fc640 1 -- 172.21.15.143:0/3486441804 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fdd1c023060 con 0x55def5b0f5d0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd338e0740 10 client.16971 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd338e0740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd338e0740 20 client.16971 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b2a4a0 0x55def5b2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd338e0740 4 client.16971 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55def5a82860 con 0x55def5b2a4a0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b2a4a0 0x55def5b2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd127fc640 10 client.16971 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.030+0000 7fdd227fc640 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b2a4a0 0x55def5b2c940 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d7904f1529e3ee server_cookie=1c56c54b980b4ab1 in_seq=0 out_seq=0 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd127fc640 1 -- 172.21.15.143:0/3486441804 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55def5a82860 con 0x55def5b2a4a0 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd127fc640 10 client.16971 handle_command_reply: tid=0 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 client.16971 shutdown 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 msgr2=0x55def5b19cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b19cf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000f80 tx=0x7fdd1c097aa0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 shutdown_connections 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b17520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b2a4a0 0x55def5b2c940 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b16fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 --2- 172.21.15.143:0/3486441804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b19cf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 >> 172.21.15.143:0/3486441804 conn(0x55def5ad8310 msgr2=0x55def5b11930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd127fc640 0 client.16971 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 shutdown_connections 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.031+0000 7fdd338e0740 1 -- 172.21.15.143:0/3486441804 wait complete. 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.032+0000 7fdd338e0740 20 client.16971 trim_cache size 0 max 16384 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 Processor -- start 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 -- start start 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b0f5d0 0x55def5b26340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 0x55def5b12120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b12660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c00fad0 con 0x55def5b10b40 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c011610 con 0x55def5b1a370 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd338e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c0101a0 con 0x55def5b0f5d0 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd227fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 0x55def5b12120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd227fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 0x55def5b12120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59200/0 (socket says 172.21.15.143:59200) 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd227fc640 1 -- 172.21.15.143:0/626148375 learned_addr learned my addr 172.21.15.143:0/626148375 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd2affd640 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b0f5d0 0x55def5b26340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.049+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b12660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd227fc640 1 -- 172.21.15.143:0/626148375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b0f5d0 msgr2=0x55def5b26340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd227fc640 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b0f5d0 0x55def5b26340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd227fc640 1 -- 172.21.15.143:0/626148375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 msgr2=0x55def5b12660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd227fc640 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b12660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd227fc640 1 -- 172.21.15.143:0/626148375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55def5b1a060 con 0x55def5b10b40 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd2affd640 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b0f5d0 0x55def5b26340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd227fc640 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 0x55def5b12120 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009930 tx=0x7fdd14005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecc9006d4d90e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd20ff9640 1 -- 172.21.15.143:0/626148375 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14035040 con 0x55def5b10b40 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd20ff9640 1 -- 172.21.15.143:0/626148375 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd14005500 con 0x55def5b10b40 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.050+0000 7fdd20ff9640 1 -- 172.21.15.143:0/626148375 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004af0 con 0x55def5b10b40 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 -- 172.21.15.143:0/626148375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 msgr2=0x55def5b12120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 0x55def5b12120 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009930 tx=0x7fdd14005d50 comp rx=0 tx=0).stop 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 -- 172.21.15.143:0/626148375 shutdown_connections 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b12660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b10b40 0x55def5b12120 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- 172.21.15.143:0/626148375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b0f5d0 0x55def5b26340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 -- 172.21.15.143:0/626148375 >> 172.21.15.143:0/626148375 conn(0x55def5ad8310 msgr2=0x55def5b2c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 -- 172.21.15.143:0/626148375 shutdown_connections 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 -- 172.21.15.143:0/626148375 wait complete. 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 -- start start 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 Processor -- start 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b13b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.051+0000 7fdd338e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c010f40 con 0x55def5b0f5d0 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c010870 con 0x55def5b1a370 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c00dab0 con 0x55def5b10b40 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 10 client.0 _ll_register_callbacks cb 0x7fdd1c097320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b22a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59204/0 (socket says 172.21.15.143:59204) 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 -- 172.21.15.143:0/2791066587 learned_addr learned my addr 172.21.15.143:0/2791066587 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd227fc640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b13b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 -- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 msgr2=0x55def5b13b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b13b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 -- 172.21.15.143:0/2791066587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 msgr2=0x55def5b140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 -- 172.21.15.143:0/2791066587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55def5b14710 con 0x55def5b0f5d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2b7fe640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd2affd640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b22a60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00db50 tx=0x7fdd1c0862c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fe39799ee88f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd127fc640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd127fc640 1 -- 172.21.15.143:0/2791066587 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c09be40 con 0x55def5b0f5d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd127fc640 1 -- 172.21.15.143:0/2791066587 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fdd1c0976c0 con 0x55def5b0f5d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd14005500 con 0x55def5b0f5d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd127fc640 1 -- 172.21.15.143:0/2791066587 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd1c09b740 con 0x55def5b0f5d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 10 client.16977 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.052+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55def5b95090 con 0x55def5b0f5d0 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd127fc640 1 -- 172.21.15.143:0/2791066587 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd1c0a43b0 con 0x55def5b0f5d0 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd127fc640 1 -- 172.21.15.143:0/2791066587 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fdd1c0a0c80 con 0x55def5b0f5d0 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd338e0740 10 client.16977 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd338e0740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd338e0740 20 client.16977 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd338e0740 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b955d0 0x55def5b979c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd338e0740 4 client.16977 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdd14005500 con 0x55def5b955d0 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.053+0000 7fdd227fc640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b955d0 0x55def5b979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd127fc640 10 client.16977 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd227fc640 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b955d0 0x55def5b979c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a50d2b377368ee server_cookie=6135e221ac4c2e24 in_seq=0 out_seq=0 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd127fc640 1 -- 172.21.15.143:0/2791066587 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdd14005500 con 0x55def5b955d0 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd127fc640 10 client.16977 handle_command_reply: tid=0 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 client.16977 shutdown 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 msgr2=0x55def5b22a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b22a60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00db50 tx=0x7fdd1c0862c0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 shutdown_connections 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55def5b1a370 0x55def5b140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55def5b955d0 0x55def5b979c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def5b10b40 0x55def5b13b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd127fc640 0 client.16977 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 --2- 172.21.15.143:0/2791066587 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55def5b0f5d0 0x55def5b22a60 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 >> 172.21.15.143:0/2791066587 conn(0x55def5ad8310 msgr2=0x55def5b18800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 shutdown_connections 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 1 -- 172.21.15.143:0/2791066587 wait complete. 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.054+0000 7fdd338e0740 20 client.16977 trim_cache size 0 max 16384 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd180a2b80 msgr2=0x7fdd180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd180a2b80 0x7fdd180a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0feec0 tx=0x7fdd24005f90 comp rx=0 tx=0).stop 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 msgr2=0x7fdd2c10d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 0x7fdd2c10d780 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00db80 tx=0x7fdd1c00b630 comp rx=0 tx=0).stop 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 shutdown_connections 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fdd180a2b80 0x7fdd180a5040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c106a70 0x7fdd2c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd2c101410 0x7fdd2c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 --2- 172.21.15.143:0/2060379195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2c100a70 0x7fdd2c10d780 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 >> 172.21.15.143:0/2060379195 conn(0x7fdd2c0fc7d0 msgr2=0x7fdd2c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 shutdown_connections 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:03.055+0000 7fdd31507640 1 -- 172.21.15.143:0/2060379195 wait complete. 2024-09-10T10:52:04.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:03 smithi143 ceph-mon[31057]: pgmap v561: 129 pgs: 129 active+clean; 430 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-10T10:52:04.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:03 smithi178 ceph-mon[34813]: pgmap v561: 129 pgs: 129 active+clean; 430 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-10T10:52:04.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:03 smithi144 ceph-mon[34230]: pgmap v561: 129 pgs: 129 active+clean; 430 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: pgmap v562: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:52:06.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:52:06.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: pgmap v562: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:52:06.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:52:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: pgmap v562: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T10:52:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:52:06.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:52:06.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:52:06.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:52:07.030 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 -- 172.21.15.143:0/2389486842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100100 msgr2=0x7f279c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/2389486842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100100 0x7f279c100560 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2788009860 tx=0x7f278802a340 comp rx=0 tx=0).stop 2024-09-10T10:52:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 -- 172.21.15.143:0/2389486842 shutdown_connections 2024-09-10T10:52:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/2389486842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100aa0 0x7f279c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/2389486842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100100 0x7f279c100560 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/2389486842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 0x7f279c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 -- 172.21.15.143:0/2389486842 >> 172.21.15.143:0/2389486842 conn(0x7f279c077230 msgr2=0x7f279c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 -- 172.21.15.143:0/2389486842 shutdown_connections 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.574+0000 7f27a2b8b640 1 -- 172.21.15.143:0/2389486842 wait complete. 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 Processor -- start 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 -- start start 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 0x7f279c1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100aa0 0x7f279c101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 0x7f279c101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f279c111ec0 con 0x7f279c100100 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.575+0000 7f27a2b8b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f279c111d40 con 0x7f279c106b80 2024-09-10T10:52:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a2b8b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f279c112040 con 0x7f279c100aa0 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 0x7f279c1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 0x7f279c1012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60736/0 (socket says 172.21.15.143:60736) 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 -- 172.21.15.143:0/3380459748 learned_addr learned my addr 172.21.15.143:0/3380459748 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a1128640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 0x7f279c101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f279bfff640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100aa0 0x7f279c101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 -- 172.21.15.143:0/3380459748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100aa0 msgr2=0x7f279c101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100aa0 0x7f279c101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 -- 172.21.15.143:0/3380459748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 msgr2=0x7f279c101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 0x7f279c101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 -- 172.21.15.143:0/3380459748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c1aad90 con 0x7f279c100100 2024-09-10T10:52:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f279bfff640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100aa0 0x7f279c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a1128640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 0x7f279c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.576+0000 7f27a0927640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 0x7f279c1012e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f279000d950 tx=0x7f279000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.577+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790017070 con 0x7f279c100100 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.577+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2790009c30 con 0x7f279c100100 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.577+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c1aafc0 con 0x7f279c100100 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.577+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000f620 con 0x7f279c100100 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.577+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c1ab3c0 con 0x7f279c100100 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.579+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279c103420 con 0x7f279c100100 2024-09-10T10:52:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.580+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f27900109a0 con 0x7f279c100100 2024-09-10T10:52:07.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.580+0000 7f2799ffb640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f278c0a2b80 0x7f278c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.581+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f27900cee90 con 0x7f279c100100 2024-09-10T10:52:07.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.581+0000 7f279bfff640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f278c0a2b80 0x7f278c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.581+0000 7f279bfff640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f278c0a2b80 0x7f278c0a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2788009830 tx=0x7f27880058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.586+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2790009de0 con 0x7f279c100100 2024-09-10T10:52:07.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:07 smithi143 ceph-mon[31057]: Adjusting osd_memory_target on smithi143 to 176.0M 2024-09-10T10:52:07.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:07 smithi143 ceph-mon[31057]: Unable to set osd_memory_target on smithi143 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-10T10:52:07.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:07 smithi178 ceph-mon[34813]: Adjusting osd_memory_target on smithi143 to 176.0M 2024-09-10T10:52:07.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:07 smithi178 ceph-mon[34813]: Unable to set osd_memory_target on smithi143 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-10T10:52:07.929 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:52:07.929 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:52:07.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:07 smithi144 ceph-mon[34230]: Adjusting osd_memory_target on smithi143 to 176.0M 2024-09-10T10:52:07.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:07 smithi144 ceph-mon[34230]: Unable to set osd_memory_target on smithi143 to 184578150: error parsing value: Value '184578150' is below minimum 939524096 2024-09-10T10:52:07.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.954+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f279c02d7c0 con 0x7f279c100100 2024-09-10T10:52:07.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.955+0000 7f2799ffb640 1 -- 172.21.15.143:0/3380459748 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11998 (secure 0 0 0) 0x7f279009b440 con 0x7f279c100100 2024-09-10T10:52:07.957 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:52:07.957 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"btime":"2024-09-10T10:51:06:617569+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:51:06.305790+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:52:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2024-09-10T10:52:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f278c0a2b80 msgr2=0x7f278c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f278c0a2b80 0x7f278c0a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2788009830 tx=0x7f27880058d0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 msgr2=0x7f279c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 0x7f279c1012e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f279000d950 tx=0x7f279000de10 comp rx=0 tx=0).stop 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 shutdown_connections 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f278c0a2b80 0x7f278c0a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c106b80 0x7f279c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f279c100aa0 0x7f279c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 --2- 172.21.15.143:0/3380459748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f279c100100 0x7f279c1012e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 >> 172.21.15.143:0/3380459748 conn(0x7f279c077230 msgr2=0x7f279c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 shutdown_connections 2024-09-10T10:52:07.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:07.959+0000 7f27a2b8b640 1 -- 172.21.15.143:0/3380459748 wait complete. 2024-09-10T10:52:08.657 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id ddc3abbd --release --await' 2024-09-10T10:52:08.657 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id ddc3abbd --release --await 2024-09-10T10:52:08.737 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:08.901 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:08 smithi143 ceph-mon[31057]: pgmap v563: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-10T10:52:08.901 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:08 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3380459748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:08.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:08 smithi178 ceph-mon[34813]: pgmap v563: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-10T10:52:08.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:08 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3380459748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:08.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:08 smithi144 ceph-mon[34230]: pgmap v563: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 143 op/s 2024-09-10T10:52:08.937 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:08 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3380459748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.335+0000 7efc67fd4640 1 -- 172.21.15.143:0/2079768346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60100e20 msgr2=0x7efc60101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.335+0000 7efc67fd4640 1 --2- 172.21.15.143:0/2079768346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60100e20 0x7efc60101200 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009900 tx=0x7efc5c02a480 comp rx=0 tx=0).stop 2024-09-10T10:52:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 -- 172.21.15.143:0/2079768346 shutdown_connections 2024-09-10T10:52:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 --2- 172.21.15.143:0/2079768346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60102ef0 0x7efc6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 --2- 172.21.15.143:0/2079768346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60102570 0x7efc601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 --2- 172.21.15.143:0/2079768346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60100e20 0x7efc60101200 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 -- 172.21.15.143:0/2079768346 >> 172.21.15.143:0/2079768346 conn(0x7efc600fc7d0 msgr2=0x7efc600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 -- 172.21.15.143:0/2079768346 shutdown_connections 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.336+0000 7efc67fd4640 1 -- 172.21.15.143:0/2079768346 wait complete. 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 Processor -- start 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 -- start start 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60100e20 0x7efc60198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 0x7efc601989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60102ef0 0x7efc6019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc60111f40 con 0x7efc60102570 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc60111dc0 con 0x7efc60100e20 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc67fd4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc601120c0 con 0x7efc60102ef0 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 0x7efc601989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 0x7efc601989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60756/0 (socket says 172.21.15.143:60756) 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 -- 172.21.15.143:0/3943384520 learned_addr learned my addr 172.21.15.143:0/3943384520 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc66571640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60102ef0 0x7efc6019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc65d70640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60100e20 0x7efc60198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 -- 172.21.15.143:0/3943384520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60102ef0 msgr2=0x7efc6019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60102ef0 0x7efc6019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 -- 172.21.15.143:0/3943384520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60100e20 msgr2=0x7efc60198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60100e20 0x7efc60198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc6556f640 1 -- 172.21.15.143:0/3943384520 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6019d410 con 0x7efc60102570 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.337+0000 7efc65d70640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60100e20 0x7efc60198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.338+0000 7efc6556f640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 0x7efc601989a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b2f0 tx=0x7efc5000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:09.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.338+0000 7efc56ffd640 1 -- 172.21.15.143:0/3943384520 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50022070 con 0x7efc60102570 2024-09-10T10:52:09.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.338+0000 7efc56ffd640 1 -- 172.21.15.143:0/3943384520 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc5001b9d0 con 0x7efc60102570 2024-09-10T10:52:09.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.338+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6019d700 con 0x7efc60102570 2024-09-10T10:52:09.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.338+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6010e280 con 0x7efc60102570 2024-09-10T10:52:09.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.339+0000 7efc56ffd640 1 -- 172.21.15.143:0/3943384520 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5001db20 con 0x7efc60102570 2024-09-10T10:52:09.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 Processor -- start 2024-09-10T10:52:09.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 -- start start 2024-09-10T10:52:09.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc56ffd640 1 -- 172.21.15.143:0/3943384520 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efc50025070 con 0x7efc60102570 2024-09-10T10:52:09.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0751280 con 0x55dbe07ebb60 2024-09-10T10:52:09.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dbe06f8b10 con 0x55dbe07ec480 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dbe074d240 con 0x55dbe07f6d40 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc66571640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc65d70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc66571640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07ebf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60766/0 (socket says 172.21.15.143:60766) 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc66571640 1 -- 172.21.15.143:0/3753852639 learned_addr learned my addr 172.21.15.143:0/3753852639 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.342+0000 7efc6556f640 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc66571640 1 -- 172.21.15.143:0/3753852639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 msgr2=0x55dbe07f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc66571640 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc66571640 1 -- 172.21.15.143:0/3753852639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 msgr2=0x55dbe07f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc66571640 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc66571640 1 -- 172.21.15.143:0/3753852639 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe07051f0 con 0x55dbe07ebb60 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc65d70640 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc6556f640 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc66571640 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07ebf40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efc58009f50 tx=0x7efc580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b143d878f566caf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc4bfff640 1 -- 172.21.15.143:0/3753852639 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004b20 con 0x55dbe07ebb60 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc4bfff640 1 -- 172.21.15.143:0/3753852639 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc58004cc0 con 0x55dbe07ebb60 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.343+0000 7efc4bfff640 1 -- 172.21.15.143:0/3753852639 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58005020 con 0x55dbe07ebb60 2024-09-10T10:52:09.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc56ffd640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc4c0a2c80 0x7efc4c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc65d70640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc4c0a2c80 0x7efc4c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc56ffd640 1 -- 172.21.15.143:0/3943384520 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc500d6e80 con 0x7efc60102570 2024-09-10T10:52:09.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 -- 172.21.15.143:0/3753852639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 msgr2=0x55dbe07ebf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07ebf40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efc58009f50 tx=0x7efc580118d0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 -- 172.21.15.143:0/3753852639 shutdown_connections 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/3753852639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07ebf40 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.344+0000 7efc6a3ad740 1 -- 172.21.15.143:0/3753852639 >> 172.21.15.143:0/3753852639 conn(0x55dbe07e80f0 msgr2=0x55dbe07e84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.345+0000 7efc6a3ad740 1 -- 172.21.15.143:0/3753852639 shutdown_connections 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.345+0000 7efc6a3ad740 1 -- 172.21.15.143:0/3753852639 wait complete. 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.345+0000 7efc65d70640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc4c0a2c80 0x7efc4c0a5140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc5c005c40 tx=0x7efc5c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.345+0000 7efc6a3ad740 1 -- start start 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.345+0000 7efc6a3ad740 1 Processor -- start 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 0x55dbe07f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 0x55dbe07f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55dbe07f9910 con 0x55dbe07ec480 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dbe070bf10 con 0x55dbe07ebb60 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dbe07f9a90 con 0x55dbe07f6d40 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6a3ad740 10 client.0 _ll_register_callbacks cb 0x55dbe06d4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6556f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 0x55dbe07f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc65d70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 0x55dbe07f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6556f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 0x55dbe07f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60774/0 (socket says 172.21.15.143:60774) 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc65d70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 0x55dbe07f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38176/0 (socket says 172.21.15.143:38176) 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc6556f640 1 -- 172.21.15.143:0/971977319 learned_addr learned my addr 172.21.15.143:0/971977319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.346+0000 7efc66571640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc65d70640 1 -- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 msgr2=0x55dbe07f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc65d70640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc65d70640 1 -- 172.21.15.143:0/971977319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 msgr2=0x55dbe07f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc65d70640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 0x55dbe07f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc65d70640 1 -- 172.21.15.143:0/971977319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe04c26a0 con 0x55dbe07ebb60 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc66571640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc6556f640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 0x55dbe07f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc65d70640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 0x55dbe07f3c00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006010 tx=0x7efc5c02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b7499819a76d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc497fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc497fa640 1 -- 172.21.15.143:0/971977319 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c04b020 con 0x55dbe07ebb60 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc58004cc0 con 0x55dbe07ebb60 2024-09-10T10:52:09.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc497fa640 1 -- 172.21.15.143:0/971977319 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc5c005260 con 0x55dbe07ebb60 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.347+0000 7efc497fa640 1 -- 172.21.15.143:0/971977319 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efc5c056050 con 0x55dbe07ebb60 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 10 client.26315 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dbe08055d0 con 0x55dbe07ebb60 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc497fa640 1 -- 172.21.15.143:0/971977319 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc58004cc0 con 0x55dbe07ebb60 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc497fa640 1 -- 172.21.15.143:0/971977319 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7efc5c0035f0 con 0x55dbe07ebb60 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 10 client.26315 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 20 client.26315 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08069e0 0x55dbe0808ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 4 client.26315 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe075e7f0 con 0x55dbe08069e0 2024-09-10T10:52:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.348+0000 7efc6556f640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08069e0 0x55dbe0808ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.349+0000 7efc497fa640 10 client.26315 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.349+0000 7efc6556f640 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08069e0 0x55dbe0808ea0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57979cbf670aae4d server_cookie=4bccc86bb3c245ec in_seq=0 out_seq=0 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc497fa640 1 -- 172.21.15.143:0/971977319 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbe075e7f0 con 0x55dbe08069e0 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc497fa640 10 client.26315 handle_command_reply: tid=0 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 client.26315 shutdown 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 msgr2=0x55dbe07f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 0x55dbe07f3c00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006010 tx=0x7efc5c02a480 comp rx=0 tx=0).stop 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 shutdown_connections 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08069e0 0x55dbe0808ea0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc497fa640 0 client.26315 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ec480 0x55dbe07f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/971977319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ebb60 0x55dbe07f3c00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.350+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 >> 172.21.15.143:0/971977319 conn(0x55dbe07e80f0 msgr2=0x55dbe07ede20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.351+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 shutdown_connections 2024-09-10T10:52:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.351+0000 7efc6a3ad740 1 -- 172.21.15.143:0/971977319 wait complete. 2024-09-10T10:52:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.351+0000 7efc6a3ad740 20 client.26315 trim_cache size 0 max 16384 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.375+0000 7efc6a3ad740 1 Processor -- start 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.375+0000 7efc6a3ad740 1 -- start start 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.375+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.375+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe080e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.375+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe080ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc4400faa0 con 0x55dbe07ebb60 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc440115e0 con 0x55dbe07ec480 2024-09-10T10:52:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc44010170 con 0x55dbe07f6d40 2024-09-10T10:52:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe080ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe080ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:41826/0 (socket says 172.21.15.143:41826) 2024-09-10T10:52:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 -- 172.21.15.143:0/2195680979 learned_addr learned my addr 172.21.15.143:0/2195680979 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc65d70640 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc6556f640 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe080e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 -- 172.21.15.143:0/2195680979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 msgr2=0x55dbe080e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe080e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 -- 172.21.15.143:0/2195680979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 msgr2=0x55dbe07f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 -- 172.21.15.143:0/2195680979 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe07ec2b0 con 0x55dbe07f6d40 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.376+0000 7efc66571640 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe080ed00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc58004610 tx=0x7efc58004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfeb475b9f8a2ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.377+0000 7efc4b7fe640 1 -- 172.21.15.143:0/2195680979 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004b70 con 0x55dbe07f6d40 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.377+0000 7efc4b7fe640 1 -- 172.21.15.143:0/2195680979 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc58003ee0 con 0x55dbe07f6d40 2024-09-10T10:52:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.377+0000 7efc4b7fe640 1 -- 172.21.15.143:0/2195680979 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800f870 con 0x55dbe07f6d40 2024-09-10T10:52:09.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 -- 172.21.15.143:0/2195680979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 msgr2=0x55dbe080ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe080ed00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc58004610 tx=0x7efc58004a00 comp rx=0 tx=0).stop 2024-09-10T10:52:09.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 -- 172.21.15.143:0/2195680979 shutdown_connections 2024-09-10T10:52:09.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe080ed00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe080e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/2195680979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 -- 172.21.15.143:0/2195680979 >> 172.21.15.143:0/2195680979 conn(0x55dbe07e80f0 msgr2=0x55dbe07f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 -- 172.21.15.143:0/2195680979 shutdown_connections 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 -- 172.21.15.143:0/2195680979 wait complete. 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 -- start start 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.378+0000 7efc6a3ad740 1 Processor -- start 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efc44010f10 con 0x55dbe07ebb60 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc44010840 con 0x55dbe07ec480 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc4400da80 con 0x55dbe07f6d40 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6a3ad740 10 client.0 _ll_register_callbacks cb 0x7efc5c03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:09.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07fb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38196/0 (socket says 172.21.15.143:38196) 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 -- 172.21.15.143:0/920773516 learned_addr learned my addr 172.21.15.143:0/920773516 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc65d70640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc66571640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 -- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 msgr2=0x55dbe07fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 -- 172.21.15.143:0/920773516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 msgr2=0x55dbe07fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07fabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.379+0000 7efc6556f640 1 -- 172.21.15.143:0/920773516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe0870840 con 0x55dbe07ec480 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc6556f640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07fb0f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b2c0 tx=0x7efc50016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead0775a0d595103 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc497fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc497fa640 1 -- 172.21.15.143:0/920773516 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc500a33d0 con 0x55dbe07ec480 2024-09-10T10:52:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc58003ee0 con 0x55dbe07ec480 2024-09-10T10:52:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc497fa640 1 -- 172.21.15.143:0/920773516 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efc500a16d0 con 0x55dbe07ec480 2024-09-10T10:52:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc497fa640 1 -- 172.21.15.143:0/920773516 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efc500a76d0 con 0x55dbe07ec480 2024-09-10T10:52:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc6a3ad740 10 client.26321 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dbe0871b70 con 0x55dbe07ec480 2024-09-10T10:52:09.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc497fa640 1 -- 172.21.15.143:0/920773516 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc58003ee0 con 0x55dbe07ec480 2024-09-10T10:52:09.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc497fa640 1 -- 172.21.15.143:0/920773516 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7efc500a2820 con 0x55dbe07ec480 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.380+0000 7efc6a3ad740 10 client.26321 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc6a3ad740 10 client.26321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc6a3ad740 20 client.26321 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08722f0 0x55dbe0874710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc6a3ad740 4 client.26321 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dbe0758d00 con 0x55dbe08722f0 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc65d70640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08722f0 0x55dbe0874710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.381+0000 7efc497fa640 10 client.26321 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.382+0000 7efc65d70640 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08722f0 0x55dbe0874710 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c662b35a5a5241ff server_cookie=e6cc523f4529c216 in_seq=0 out_seq=0 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc497fa640 1 -- 172.21.15.143:0/920773516 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dbe0758d00 con 0x55dbe08722f0 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc497fa640 10 client.26321 handle_command_reply: tid=0 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc6a3ad740 1 client.26321 shutdown 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 msgr2=0x55dbe07fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07fb0f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b2c0 tx=0x7efc50016040 comp rx=0 tx=0).stop 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 shutdown_connections 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.383+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55dbe08722f0 0x55dbe0874710 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc497fa640 0 client.26321 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dbe07f6d40 0x55dbe07fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dbe07ec480 0x55dbe07fb0f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 1 --2- 172.21.15.143:0/920773516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55dbe07ebb60 0x55dbe07fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 >> 172.21.15.143:0/920773516 conn(0x55dbe07e80f0 msgr2=0x55dbe07e7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 shutdown_connections 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 1 -- 172.21.15.143:0/920773516 wait complete. 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.384+0000 7efc6a3ad740 20 client.26321 trim_cache size 0 max 16384 2024-09-10T10:52:09.387 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:09.388 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:52:09.388 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "1ee854bf-7351-4a9d-90b6-c52613701ba7", 2024-09-10T10:52:09.388 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:52:09.388 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc4c0a2c80 msgr2=0x7efc4c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc4c0a2c80 0x7efc4c0a5140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc5c005c40 tx=0x7efc5c005b50 comp rx=0 tx=0).stop 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 msgr2=0x7efc601989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 0x7efc601989a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b2f0 tx=0x7efc5000b7b0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 shutdown_connections 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efc4c0a2c80 0x7efc4c0a5140 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60102ef0 0x7efc6019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc60102570 0x7efc601989a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 --2- 172.21.15.143:0/3943384520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc60100e20 0x7efc60198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 >> 172.21.15.143:0/3943384520 conn(0x7efc600fc7d0 msgr2=0x7efc600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.385+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 shutdown_connections 2024-09-10T10:52:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:09.386+0000 7efc67fd4640 1 -- 172.21.15.143:0/3943384520 wait complete. 2024-09-10T10:52:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:09.904 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:09 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:09.904 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:09 smithi143 ceph-mon[31057]: pgmap v564: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T10:52:09.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:09 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:09 smithi178 ceph-mon[34813]: pgmap v564: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T10:52:09.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:09 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:09 smithi144 ceph-mon[34230]: pgmap v564: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T10:52:10.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3124793290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a8108da0 msgr2=0x7fb8a810cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3124793290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a8108da0 0x7fb8a810cdf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009860 tx=0x7fb89c02a340 comp rx=0 tx=0).stop 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3124793290 shutdown_connections 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3124793290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a810d520 0x7fb8a81150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3124793290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a8108da0 0x7fb8a810cdf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3124793290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a81083f0 0x7fb8a81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3124793290 >> 172.21.15.143:0/3124793290 conn(0x7fb8a8077af0 msgr2=0x7fb8a8079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3124793290 shutdown_connections 2024-09-10T10:52:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.594+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3124793290 wait complete. 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.595+0000 7fb8aef2e640 1 Processor -- start 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.595+0000 7fb8aef2e640 1 -- start start 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8aef2e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a81083f0 0x7fb8a81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8aef2e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 0x7fb8a81a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8aef2e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 0x7fb8a819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8aef2e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81183d0 con 0x7fb8a810d520 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8aef2e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a8118250 con 0x7fb8a81083f0 2024-09-10T10:52:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8aef2e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a8118550 con 0x7fb8a8108da0 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 0x7fb8a81a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8accca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a81083f0 0x7fb8a81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 0x7fb8a819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 0x7fb8a819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60808/0 (socket says 172.21.15.143:60808) 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 0x7fb8a81a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:41874/0 (socket says 172.21.15.143:41874) 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8accca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a81083f0 0x7fb8a81a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38214/0 (socket says 172.21.15.143:38214) 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8ad4cb640 1 -- 172.21.15.143:0/3518620442 learned_addr learned my addr 172.21.15.143:0/3518620442 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 -- 172.21.15.143:0/3518620442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a81083f0 msgr2=0x7fb8a81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a81083f0 0x7fb8a81a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 -- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 msgr2=0x7fb8a819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 0x7fb8a819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8a7fff640 1 -- 172.21.15.143:0/3518620442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a819efc0 con 0x7fb8a8108da0 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.596+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 0x7fb8a819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.597+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 0x7fb8a81a4be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb89c02a860 tx=0x7fb89c030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.597+0000 7fb8a5ffb640 1 -- 172.21.15.143:0/3518620442 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c040020 con 0x7fb8a8108da0 2024-09-10T10:52:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.597+0000 7fb8a5ffb640 1 -- 172.21.15.143:0/3518620442 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb89c038040 con 0x7fb8a8108da0 2024-09-10T10:52:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.597+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a819f250 con 0x7fb8a8108da0 2024-09-10T10:52:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.597+0000 7fb8a5ffb640 1 -- 172.21.15.143:0/3518620442 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c037070 con 0x7fb8a8108da0 2024-09-10T10:52:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.597+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a819f690 con 0x7fb8a8108da0 2024-09-10T10:52:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.600+0000 7fb8a5ffb640 1 -- 172.21.15.143:0/3518620442 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb89c043020 con 0x7fb8a8108da0 2024-09-10T10:52:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.601+0000 7fb8b1307740 1 Processor -- start 2024-09-10T10:52:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.601+0000 7fb8b1307740 1 -- start start 2024-09-10T10:52:10.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.601+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.601+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325dfad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.601+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325dfd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8b1307740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560325d55620 con 0x560325df0060 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8b1307740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560325cfce50 con 0x560325df0980 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8b1307740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560325d515e0 con 0x560325dfb240 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60824/0 (socket says 172.21.15.143:60824) 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8ad4cb640 1 -- 172.21.15.143:0/2535826964 learned_addr learned my addr 172.21.15.143:0/2535826964 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8accca640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325dfad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8accca640 1 -- 172.21.15.143:0/2535826964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 msgr2=0x560325dfd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8a5ffb640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8900a2c00 0x7fb8900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325dfd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8accca640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325dfd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8accca640 1 -- 172.21.15.143:0/2535826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 msgr2=0x560325df0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8accca640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8accca640 1 -- 172.21.15.143:0/2535826964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560325cedd40 con 0x560325df0980 2024-09-10T10:52:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325dfd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.602+0000 7fb8a5ffb640 1 -- 172.21.15.143:0/3518620442 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb89c0eed60 con 0x7fb8a8108da0 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.603+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8900a2c00 0x7fb8900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.603+0000 7fb8accca640 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325dfad00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb898009c30 tx=0x7fb898011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9589c866be3f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.603+0000 7fb88effd640 1 -- 172.21.15.143:0/2535826964 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89801e070 con 0x560325df0980 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.603+0000 7fb88effd640 1 -- 172.21.15.143:0/2535826964 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb898009d20 con 0x560325df0980 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.603+0000 7fb88effd640 1 -- 172.21.15.143:0/2535826964 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898019410 con 0x560325df0980 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 -- 172.21.15.143:0/2535826964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 msgr2=0x560325dfad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325dfad00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb898009c30 tx=0x7fb898011810 comp rx=0 tx=0).stop 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8900a2c00 0x7fb8900a50c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0011880 tx=0x7fb8a0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 -- 172.21.15.143:0/2535826964 shutdown_connections 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325dfd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325dfad00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 --2- 172.21.15.143:0/2535826964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.604+0000 7fb8b1307740 1 -- 172.21.15.143:0/2535826964 >> 172.21.15.143:0/2535826964 conn(0x560325dec5f0 msgr2=0x560325dec9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.605+0000 7fb8b1307740 1 -- 172.21.15.143:0/2535826964 shutdown_connections 2024-09-10T10:52:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.605+0000 7fb8b1307740 1 -- 172.21.15.143:0/2535826964 wait complete. 2024-09-10T10:52:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.605+0000 7fb8b1307740 1 -- start start 2024-09-10T10:52:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.605+0000 7fb8b1307740 1 Processor -- start 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325df0980 0x560325df8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325dfb240 0x560325df8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x560325dfde10 con 0x560325df0060 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560325cddf10 con 0x560325dfb240 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560325dfdf90 con 0x560325df0980 2024-09-10T10:52:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8b1307740 10 client.0 _ll_register_callbacks cb 0x560325d01db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325dfb240 0x560325df8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60828/0 (socket says 172.21.15.143:60828) 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 -- 172.21.15.143:0/3929880241 learned_addr learned my addr 172.21.15.143:0/3929880241 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325df0980 0x560325df8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 -- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325df0980 msgr2=0x560325df8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325df0980 0x560325df8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 -- 172.21.15.143:0/3929880241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325dfb240 msgr2=0x560325df8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325dfb240 0x560325df8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.606+0000 7fb8accca640 1 -- 172.21.15.143:0/3929880241 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560325a64580 con 0x560325df0060 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb8accca640 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df80e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb89801c040 tx=0x7fb898009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c5611f93f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb887fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb887fff640 1 -- 172.21.15.143:0/3929880241 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89801e040 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb898009d20 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb887fff640 1 -- 172.21.15.143:0/3929880241 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb8980035f0 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb887fff640 1 -- 172.21.15.143:0/3929880241 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb898032050 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb8b1307740 10 client.17013 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.607+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560325b2e960 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb887fff640 1 -- 172.21.15.143:0/3929880241 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb898009d20 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb887fff640 1 -- 172.21.15.143:0/3929880241 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb898021030 con 0x560325df0060 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb8b1307740 10 client.17013 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb8b1307740 10 client.17013 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb8b1307740 20 client.17013 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e0b000 0x560325e0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb8b1307740 4 client.17013 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.608+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560325d62b90 con 0x560325e0b000 2024-09-10T10:52:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.609+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e0b000 0x560325e0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.609+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e0b000 0x560325e0d490 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72dc99f91690ab0a server_cookie=2ea36f171d1794c9 in_seq=0 out_seq=0 2024-09-10T10:52:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.609+0000 7fb887fff640 10 client.17013 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb887fff640 1 -- 172.21.15.143:0/3929880241 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560325d62b90 con 0x560325e0b000 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb887fff640 10 client.17013 handle_command_reply: tid=0 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 client.17013 shutdown 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 msgr2=0x560325df80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df80e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb89801c040 tx=0x7fb898009f70 comp rx=0 tx=0).stop 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 shutdown_connections 2024-09-10T10:52:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325dfb240 0x560325df8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e0b000 0x560325e0d490 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325df0980 0x560325df8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3929880241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0060 0x560325df80e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb887fff640 0 client.17013 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 >> 172.21.15.143:0/3929880241 conn(0x560325dec5f0 msgr2=0x560325df2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.610+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 shutdown_connections 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.611+0000 7fb8b1307740 1 -- 172.21.15.143:0/3929880241 wait complete. 2024-09-10T10:52:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.611+0000 7fb8b1307740 20 client.17013 trim_cache size 0 max 16384 2024-09-10T10:52:10.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 Processor -- start 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 -- start start 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0980 0x560325df9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325e12d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325e132a0 0x560325e031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81b50d0 con 0x560325df0980 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81b6c10 con 0x560325e132a0 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8b1307740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81b57a0 con 0x560325dfb240 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325e12d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325e132a0 0x560325e031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8ad4cb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325e132a0 0x560325e031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38246/0 (socket says 172.21.15.143:38246) 2024-09-10T10:52:10.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325e12d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:41906/0 (socket says 172.21.15.143:41906) 2024-09-10T10:52:10.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8ad4cb640 1 -- 172.21.15.143:0/3779438264 learned_addr learned my addr 172.21.15.143:0/3779438264 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:10.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.638+0000 7fb8accca640 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0980 0x560325df9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8a7fff640 1 -- 172.21.15.143:0/3779438264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325e132a0 msgr2=0x560325e031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325e132a0 0x560325e031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8a7fff640 1 -- 172.21.15.143:0/3779438264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0980 msgr2=0x560325df9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0980 0x560325df9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8a7fff640 1 -- 172.21.15.143:0/3779438264 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560325e0db10 con 0x560325dfb240 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8accca640 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0980 0x560325df9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.639+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325e12d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0baf60 tx=0x7fb89c0db040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e83dfd52da9031c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.640+0000 7fb88e7fc640 1 -- 172.21.15.143:0/3779438264 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c0bf610 con 0x560325dfb240 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.640+0000 7fb88e7fc640 1 -- 172.21.15.143:0/3779438264 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb89c0c3ed0 con 0x560325dfb240 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.640+0000 7fb88e7fc640 1 -- 172.21.15.143:0/3779438264 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c0042e0 con 0x560325dfb240 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 -- 172.21.15.143:0/3779438264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 msgr2=0x560325e12d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325e12d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0baf60 tx=0x7fb89c0db040 comp rx=0 tx=0).stop 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 -- 172.21.15.143:0/3779438264 shutdown_connections 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325e132a0 0x560325e031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325dfb240 0x560325e12d60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 --2- 172.21.15.143:0/3779438264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325df0980 0x560325df9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 -- 172.21.15.143:0/3779438264 >> 172.21.15.143:0/3779438264 conn(0x560325dec5f0 msgr2=0x560325deb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:10.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 -- 172.21.15.143:0/3779438264 shutdown_connections 2024-09-10T10:52:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.641+0000 7fb8b1307740 1 -- 172.21.15.143:0/3779438264 wait complete. 2024-09-10T10:52:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.642+0000 7fb8b1307740 1 -- start start 2024-09-10T10:52:10.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.642+0000 7fb8b1307740 1 Processor -- start 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325e031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 0x560325e03720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8b1307740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325e132a0 0x560325e001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 0x560325e03720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8accca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325e031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 0x560325e03720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60848/0 (socket says 172.21.15.143:60848) 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8accca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325e031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38250/0 (socket says 172.21.15.143:38250) 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8a7fff640 1 -- 172.21.15.143:0/206742326 learned_addr learned my addr 172.21.15.143:0/206742326 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81b6540 con 0x560325dfb240 2024-09-10T10:52:10.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81b5e70 con 0x560325df0980 2024-09-10T10:52:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81b30b0 con 0x560325e132a0 2024-09-10T10:52:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.643+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325e132a0 0x560325e001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8ad4cb640 1 -- 172.21.15.143:0/206742326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 msgr2=0x560325e031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325e031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8ad4cb640 1 -- 172.21.15.143:0/206742326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 msgr2=0x560325e03720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:10.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 0x560325e03720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8ad4cb640 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560325e007a0 con 0x560325e132a0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8a7fff640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 0x560325e03720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8ad4cb640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325e132a0 0x560325e001b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0010800 tx=0x7fb8a0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98b1529ce0b67a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb887fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8b1307740 10 client.0 _ll_register_callbacks cb 0x7fb898005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb89c0c3ed0 con 0x560325e132a0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.644+0000 7fb887fff640 1 -- 172.21.15.143:0/206742326 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a002f070 con 0x560325e132a0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.645+0000 7fb887fff640 1 -- 172.21.15.143:0/206742326 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb8a002b650 con 0x560325e132a0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.645+0000 7fb887fff640 1 -- 172.21.15.143:0/206742326 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8a003b050 con 0x560325e132a0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.645+0000 7fb8b1307740 10 client.26428 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.645+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=35}) -- 0x560325e76430 con 0x560325e132a0 2024-09-10T10:52:10.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.645+0000 7fb887fff640 1 -- 172.21.15.143:0/206742326 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb8a002b850 con 0x560325e132a0 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.645+0000 7fb8b1307740 10 client.26428 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8b1307740 10 client.26428 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8b1307740 20 client.26428 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8b1307740 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e76af0 0x560325e78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8b1307740 4 client.26428 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ddc3abbd", "release": true, "await": true}] 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ddc3abbd", "release": true, "await": true}) -- 0x7fb89c0c3ed0 con 0x560325e76af0 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8accca640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e76af0 0x560325e78fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb8accca640 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e76af0 0x560325e78fb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4483bd1d37dbce42 server_cookie=6492cb4ab0779271 in_seq=0 out_seq=0 2024-09-10T10:52:10.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:10.646+0000 7fb887fff640 10 client.26428 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:10.750 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:10.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:10 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:10 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:10 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 -- 172.21.15.143:0/3369227858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c10c170 msgr2=0x7f6d2c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 --2- 172.21.15.143:0/3369227858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c10c170 0x7f6d2c10e610 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009ca0 tx=0x7f6d2402a950 comp rx=0 tx=0).stop 2024-09-10T10:52:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 -- 172.21.15.143:0/3369227858 shutdown_connections 2024-09-10T10:52:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 --2- 172.21.15.143:0/3369227858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c10c170 0x7f6d2c10e610 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 --2- 172.21.15.143:0/3369227858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 --2- 172.21.15.143:0/3369227858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c102d40 0x7f6d2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 -- 172.21.15.143:0/3369227858 >> 172.21.15.143:0/3369227858 conn(0x7f6d2c0fc7a0 msgr2=0x7f6d2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 -- 172.21.15.143:0/3369227858 shutdown_connections 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.292+0000 7f6d30f03640 1 -- 172.21.15.143:0/3369227858 wait complete. 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 Processor -- start 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 -- start start 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 0x7f6d2c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c10c170 0x7f6d2c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c111e20 con 0x7f6d2c102d40 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c111ca0 con 0x7f6d2c103660 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d30f03640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c111fa0 con 0x7f6d2c10c170 2024-09-10T10:52:11.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 0x7f6d2c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38262/0 (socket says 172.21.15.143:38262) 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2420797778 learned_addr learned my addr 172.21.15.143:0/2420797778 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 0x7f6d2c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60862/0 (socket says 172.21.15.143:60862) 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c10c170 0x7f6d2c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2420797778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c10c170 msgr2=0x7f6d2c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c10c170 0x7f6d2c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 msgr2=0x7f6d2c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 0x7f6d2c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2420797778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c19d2b0 con 0x7f6d2c103660 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.293+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 0x7f6d2c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.294+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c198840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000e9f0 tx=0x7f6d2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.294+0000 7f6d1b7fe640 1 -- 172.21.15.143:0/2420797778 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000ccc0 con 0x7f6d2c103660 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.294+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2c19d5a0 con 0x7f6d2c103660 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.294+0000 7f6d1b7fe640 1 -- 172.21.15.143:0/2420797778 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d200108e0 con 0x7f6d2c103660 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.294+0000 7f6d1b7fe640 1 -- 172.21.15.143:0/2420797778 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200187c0 con 0x7f6d2c103660 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.294+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2c1a4e80 con 0x7f6d2c103660 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.296+0000 7f6d1b7fe640 1 -- 172.21.15.143:0/2420797778 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6d20010430 con 0x7f6d2c103660 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.296+0000 7f6d332dc740 1 Processor -- start 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 -- start start 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea05c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea067140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea069a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5579e9fc22e0 con 0x5579ea05cdc0 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579e9f69ce0 con 0x5579ea067680 2024-09-10T10:52:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d332dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5579e9fbe2a0 con 0x5579ea05c4a0 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d2ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea05c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea067140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d2ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea05c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:41942/0 (socket says 172.21.15.143:41942) 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea067140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60874/0 (socket says 172.21.15.143:60874) 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/2338214831 learned_addr learned my addr 172.21.15.143:0/2338214831 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea069a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d1b7fe640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d140a2b80 0x7f6d140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.297+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/2338214831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 msgr2=0x5579ea069a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea069a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/2338214831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 msgr2=0x5579ea067140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea067140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/2338214831 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e9f4e440 con 0x5579ea05c4a0 2024-09-10T10:52:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d140a2b80 0x7f6d140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d1b7fe640 1 -- 172.21.15.143:0/2420797778 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d200cf640 con 0x7f6d2c103660 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea05c880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d24035000 tx=0x7f6d24005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17923cdea59d8d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d18ff9640 1 -- 172.21.15.143:0/2338214831 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24034040 con 0x5579ea05c4a0 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d140a2b80 0x7f6d140a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d2404d000 tx=0x7f6d2404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d18ff9640 1 -- 172.21.15.143:0/2338214831 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d24003c30 con 0x5579ea05c4a0 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.298+0000 7f6d18ff9640 1 -- 172.21.15.143:0/2338214831 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24046670 con 0x5579ea05c4a0 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 -- 172.21.15.143:0/2338214831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 msgr2=0x5579ea05c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea05c880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d24035000 tx=0x7f6d24005ef0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 -- 172.21.15.143:0/2338214831 shutdown_connections 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea069a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea067140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2338214831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea05c880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 -- 172.21.15.143:0/2338214831 >> 172.21.15.143:0/2338214831 conn(0x5579ea058a30 msgr2=0x5579ea058e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 -- 172.21.15.143:0/2338214831 shutdown_connections 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 -- 172.21.15.143:0/2338214831 wait complete. 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 -- start start 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.299+0000 7f6d332dc740 1 Processor -- start 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea0642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea064800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5579ea06a250 con 0x5579ea05cdc0 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579e9f7d050 con 0x5579ea067680 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5579ea06a3d0 con 0x5579ea05c4a0 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 10 client.0 _ll_register_callbacks cb 0x5579e9fbd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:11.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea0642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea0642a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60882/0 (socket says 172.21.15.143:60882) 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2588422673 learned_addr learned my addr 172.21.15.143:0/2588422673 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea064800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 msgr2=0x5579ea066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2588422673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 msgr2=0x5579ea064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea064800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 -- 172.21.15.143:0/2588422673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e9d333c0 con 0x5579ea05cdc0 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea0642a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000e9c0 tx=0x7f6d20009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dcd4f9f0d23be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d11ffb640 1 -- 172.21.15.143:0/2588422673 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20010d90 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d11ffb640 1 -- 172.21.15.143:0/2588422673 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d200cf340 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d24003c30 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d11ffb640 1 -- 172.21.15.143:0/2588422673 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d20010720 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 10 client.17031 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.300+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5579ea075cb0 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d11ffb640 1 -- 172.21.15.143:0/2588422673 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d24003c30 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d11ffb640 1 -- 172.21.15.143:0/2588422673 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f6d200978e0 con 0x5579ea05cdc0 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d332dc740 10 client.17031 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d332dc740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d332dc740 20 client.17031 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea077170 0x5579ea079560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d332dc740 4 client.17031 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e9fcf850 con 0x5579ea077170 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea077170 0x5579ea079560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea077170 0x5579ea079560 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72bedf32d79d017 server_cookie=56af7c60db60f68e in_seq=0 out_seq=0 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.301+0000 7f6d11ffb640 10 client.17031 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.302+0000 7f6d11ffb640 1 -- 172.21.15.143:0/2588422673 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579e9fcf850 con 0x5579ea077170 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.302+0000 7f6d11ffb640 10 client.17031 handle_command_reply: tid=0 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.302+0000 7f6d332dc740 1 client.17031 shutdown 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 msgr2=0x5579ea0642a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea0642a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000e9c0 tx=0x7f6d20009650 comp rx=0 tx=0).stop 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 shutdown_connections 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05cdc0 0x5579ea0642a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea077170 0x5579ea079560 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 --2- 172.21.15.143:0/2588422673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d11ffb640 0 client.17031 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 >> 172.21.15.143:0/2588422673 conn(0x5579ea058a30 msgr2=0x5579ea057b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 shutdown_connections 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 1 -- 172.21.15.143:0/2588422673 wait complete. 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.303+0000 7f6d332dc740 20 client.17031 trim_cache size 0 max 16384 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 Processor -- start 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 -- start start 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea0734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea05cdc0 0x5579ea07ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 0x5579ea07f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ac8c0 con 0x5579ea067680 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ae400 con 0x5579ea05cdc0 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d332dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1acf90 con 0x5579ea05c4a0 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d2ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 0x5579ea07f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d2ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 0x5579ea07f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60890/0 (socket says 172.21.15.143:60890) 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/3312017156 learned_addr learned my addr 172.21.15.143:0/3312017156 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.320+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea0734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea05cdc0 0x5579ea07ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/3312017156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 msgr2=0x5579ea0734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea0734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/3312017156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea05cdc0 msgr2=0x5579ea07ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea05cdc0 0x5579ea07ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d2ad9d640 1 -- 172.21.15.143:0/3312017156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579ea0798d0 con 0x5579ea067680 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 0x5579ea07f370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d2402ae70 tx=0x7f6d240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fbc8277235d32f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d13fff640 1 -- 172.21.15.143:0/3312017156 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24005bd0 con 0x5579ea067680 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d13fff640 1 -- 172.21.15.143:0/3312017156 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d24004800 con 0x5579ea067680 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.321+0000 7f6d13fff640 1 -- 172.21.15.143:0/3312017156 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24046ae0 con 0x5579ea067680 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 -- 172.21.15.143:0/3312017156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 msgr2=0x5579ea07f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 0x5579ea07f370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d2402ae70 tx=0x7f6d240095c0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 -- 172.21.15.143:0/3312017156 shutdown_connections 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea067680 0x5579ea07f370 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea05cdc0 0x5579ea07ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 --2- 172.21.15.143:0/3312017156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05c4a0 0x5579ea0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 -- 172.21.15.143:0/3312017156 >> 172.21.15.143:0/3312017156 conn(0x5579ea058a30 msgr2=0x5579ea05f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 -- 172.21.15.143:0/3312017156 shutdown_connections 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.322+0000 7f6d332dc740 1 -- 172.21.15.143:0/3312017156 wait complete. 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 -- start start 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 Processor -- start 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 0x5579ea06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05cdc0 0x5579ea06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea0dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1add30 con 0x5579ea05c4a0 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1ad660 con 0x5579ea067680 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c1aa8c0 con 0x5579ea05cdc0 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d332dc740 10 client.0 _ll_register_callbacks cb 0x7f6d2009ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 0x5579ea06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 0x5579ea06abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60894/0 (socket says 172.21.15.143:60894) 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 -- 172.21.15.143:0/1495471363 learned_addr learned my addr 172.21.15.143:0/1495471363 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05cdc0 0x5579ea06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2ad9d640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea0dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 -- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05cdc0 msgr2=0x5579ea06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05cdc0 0x5579ea06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 -- 172.21.15.143:0/1495471363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 msgr2=0x5579ea0dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea0dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.323+0000 7f6d2a59c640 1 -- 172.21.15.143:0/1495471363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579ea067360 con 0x5579ea05c4a0 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d2a59c640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 0x5579ea06abd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c017010 tx=0x7f6d1c004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc0a4aecdf8ed99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d11ffb640 1 -- 172.21.15.143:0/1495471363 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c003d30 con 0x5579ea05c4a0 2024-09-10T10:52:11.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d11ffb640 1 -- 172.21.15.143:0/1495471363 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6d1c013ae0 con 0x5579ea05c4a0 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d24004800 con 0x5579ea05c4a0 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d11ffb640 1 -- 172.21.15.143:0/1495471363 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d1c013ce0 con 0x5579ea05c4a0 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 10 client.17037 fetch_fsmap learned FSMap version 35 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5579ea0e1440 con 0x5579ea05c4a0 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d11ffb640 1 -- 172.21.15.143:0/1495471363 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d24004800 con 0x5579ea05c4a0 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d11ffb640 1 -- 172.21.15.143:0/1495471363 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f6d1c029440 con 0x5579ea05c4a0 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 10 client.17037 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 20 client.17037 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea0e1c10 0x5579ea0e4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 4 client.17037 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5579e9fc9d60 con 0x5579ea0e1c10 2024-09-10T10:52:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.324+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea0e1c10 0x5579ea0e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d11ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d29d9b640 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea0e1c10 0x5579ea0e4030 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc739fe824764af server_cookie=6e97f5253a608561 in_seq=0 out_seq=0 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d11ffb640 1 -- 172.21.15.143:0/1495471363 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5579e9fc9d60 con 0x5579ea0e1c10 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d11ffb640 10 client.17037 handle_command_reply: tid=0 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 client.17037 shutdown 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 msgr2=0x5579ea06abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 0x5579ea06abd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c017010 tx=0x7f6d1c004260 comp rx=0 tx=0).stop 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 shutdown_connections 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579ea067680 0x5579ea0dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5579ea0e1c10 0x5579ea0e4030 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579ea05cdc0 0x5579ea06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 --2- 172.21.15.143:0/1495471363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5579ea05c4a0 0x5579ea06abd0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d11ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 >> 172.21.15.143:0/1495471363 conn(0x5579ea058a30 msgr2=0x5579ea05efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 shutdown_connections 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.325+0000 7f6d332dc740 1 -- 172.21.15.143:0/1495471363 wait complete. 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d332dc740 20 client.17037 trim_cache size 0 max 16384 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:52:11.409 INFO:teuthology.orchestra.run.smithi143.stdout: "1ee854bf-7351-4a9d-90b6-c52613701ba7": { 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "1ee854bf-7351-4a9d-90b6-c52613701ba7", 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d140a2b80 msgr2=0x7f6d140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d140a2b80 0x7f6d140a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d2404d000 tx=0x7f6d2404a000 comp rx=0 tx=0).stop 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 msgr2=0x7f6d2c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c198840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000e9f0 tx=0x7f6d2000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 shutdown_connections 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6d140a2b80 0x7f6d140a5040 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d2c10c170 0x7f6d2c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2c103660 0x7f6d2c198840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 --2- 172.21.15.143:0/2420797778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d2c102d40 0x7f6d2c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 >> 172.21.15.143:0/2420797778 conn(0x7f6d2c0fc7a0 msgr2=0x7f6d2c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 shutdown_connections 2024-09-10T10:52:11.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.326+0000 7f6d30f03640 1 -- 172.21.15.143:0/2420797778 wait complete. 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb887fff640 1 -- 172.21.15.143:0/206742326 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fb89c0c3ed0 con 0x560325e76af0 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb887fff640 10 client.26428 handle_command_reply: tid=0 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 client.26428 shutdown 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325e132a0 msgr2=0x560325e001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325e132a0 0x560325e001b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0010800 tx=0x7fb8a0004ab0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 shutdown_connections 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x560325e76af0 0x560325e78fb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560325e132a0 0x560325e001b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560325dfb240 0x560325e03720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 --2- 172.21.15.143:0/206742326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560325df0980 0x560325e031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 >> 172.21.15.143:0/206742326 conn(0x560325dec5f0 msgr2=0x560325df1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.411+0000 7fb887fff640 0 client.26428 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.412+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 shutdown_connections 2024-09-10T10:52:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.412+0000 7fb8b1307740 1 -- 172.21.15.143:0/206742326 wait complete. 2024-09-10T10:52:11.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.412+0000 7fb8b1307740 20 client.26428 trim_cache size 0 max 16384 2024-09-10T10:52:11.414 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 35, 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 20, 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "ddc3abbd": { 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 20, 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:52:11.415 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 140.1, 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:11.416 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.414+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8900a2c00 msgr2=0x7fb8900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.414+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8900a2c00 0x7fb8900a50c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0011880 tx=0x7fb8a0004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.414+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 msgr2=0x7fb8a81a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.414+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 0x7fb8a81a4be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb89c02a860 tx=0x7fb89c030bd0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.417+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 shutdown_connections 2024-09-10T10:52:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.417+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fb8900a2c00 0x7fb8900a50c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.417+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a810d520 0x7fb8a819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.417+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a8108da0 0x7fb8a81a4be0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.417+0000 7fb8aef2e640 1 --2- 172.21.15.143:0/3518620442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8a81083f0 0x7fb8a81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.417+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 >> 172.21.15.143:0/3518620442 conn(0x7fb8a8077af0 msgr2=0x7fb8a8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.419+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 shutdown_connections 2024-09-10T10:52:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:11.419+0000 7fb8aef2e640 1 -- 172.21.15.143:0/3518620442 wait complete. 2024-09-10T10:52:11.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:11 smithi178 ceph-mon[34813]: pgmap v565: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-10T10:52:11.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:11 smithi144 ceph-mon[34230]: pgmap v565: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-10T10:52:12.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:11 smithi143 ceph-mon[31057]: pgmap v565: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-10T10:52:12.569 INFO:tasks.cephfs.filesystem:scrub status for tag:1ee854bf-7351-4a9d-90b6-c52613701ba7 - {'path': '/', 'tag': '1ee854bf-7351-4a9d-90b6-c52613701ba7', 'options': 'recursive,force'} 2024-09-10T10:52:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:12 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:52:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:12 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:12.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:12 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:12.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:12 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:52:12.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:12 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:12.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:12 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:12.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:12 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:52:12.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:12 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:12.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:12 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:13.154 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ddc3abbd', total seconds elapsed: 32.6 2024-09-10T10:52:13.154 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T10:52:13.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:52:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:13.592 INFO:tasks.workunit.client.0.smithi143.stdout: 96 9229 11 6775 2 3538 6 2024-09-10T10:52:13.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:13 smithi178 ceph-mon[34813]: pgmap v566: 129 pgs: 129 active+clean; 422 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 704 op/s 2024-09-10T10:52:13.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:13 smithi144 ceph-mon[34230]: pgmap v566: 129 pgs: 129 active+clean; 422 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 704 op/s 2024-09-10T10:52:13.980 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:13 smithi143 ceph-mon[31057]: pgmap v566: 129 pgs: 129 active+clean; 422 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 704 op/s 2024-09-10T10:52:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 -- 172.21.15.143:0/3861102498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4101de0 msgr2=0x7fbcb41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/3861102498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4101de0 0x7fbcb41021c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009900 tx=0x7fbca802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 -- 172.21.15.143:0/3861102498 shutdown_connections 2024-09-10T10:52:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/3861102498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4106f10 0x7fbcb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/3861102498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4102790 0x7fbcb41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/3861102498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4101de0 0x7fbcb41021c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 -- 172.21.15.143:0/3861102498 >> 172.21.15.143:0/3861102498 conn(0x7fbcb40fd880 msgr2=0x7fbcb40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 -- 172.21.15.143:0/3861102498 shutdown_connections 2024-09-10T10:52:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.092+0000 7fbcbac2f640 1 -- 172.21.15.143:0/3861102498 wait complete. 2024-09-10T10:52:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 Processor -- start 2024-09-10T10:52:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 -- start start 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4101de0 0x7fbcb40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4102790 0x7fbcb4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4106f10 0x7fbcb4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4111e20 con 0x7fbcb4106f10 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4111ca0 con 0x7fbcb4102790 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.093+0000 7fbcbac2f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4111fa0 con 0x7fbcb4101de0 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb89cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4101de0 0x7fbcb40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb89cb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4101de0 0x7fbcb40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:41970/0 (socket says 172.21.15.143:41970) 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb91cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4106f10 0x7fbcb4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb89cb640 1 -- 172.21.15.143:0/1637502831 learned_addr learned my addr 172.21.15.143:0/1637502831 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4102790 0x7fbcb4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 -- 172.21.15.143:0/1637502831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4101de0 msgr2=0x7fbcb40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4101de0 0x7fbcb40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 -- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4106f10 msgr2=0x7fbcb4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4106f10 0x7fbcb4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 -- 172.21.15.143:0/1637502831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb41aac90 con 0x7fbcb4102790 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb91cc640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4106f10 0x7fbcb4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb3fff640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4102790 0x7fbcb4074a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbca000e9f0 tx=0x7fbca000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca000ccc0 con 0x7fbcb4102790 2024-09-10T10:52:14.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbca00108e0 con 0x7fbcb4102790 2024-09-10T10:52:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb41aaec0 con 0x7fbcb4102790 2024-09-10T10:52:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.094+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0018880 con 0x7fbcb4102790 2024-09-10T10:52:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.095+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb41ab310 con 0x7fbcb4102790 2024-09-10T10:52:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.096+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb41032c0 con 0x7fbcb4102790 2024-09-10T10:52:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.097+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbca0010a90 con 0x7fbcb4102790 2024-09-10T10:52:14.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.098+0000 7fbcb1ffb640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2c00 0x7fbca40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.099+0000 7fbcb89cb640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2c00 0x7fbca40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.099+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbca00cf6c0 con 0x7fbcb4102790 2024-09-10T10:52:14.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.099+0000 7fbcb89cb640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2c00 0x7fbca40a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbca8031440 tx=0x7fbca80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:14.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.102+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbca009bc10 con 0x7fbcb4102790 2024-09-10T10:52:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.426+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcb410a120 con 0x7fbcb4102790 2024-09-10T10:52:14.435 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:52:14.435 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"btime":"2024-09-10T10:52:11:648994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:52:11.410829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:52:14.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.429+0000 7fbcb1ffb640 1 -- 172.21.15.143:0/1637502831 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12001 (secure 0 0 0) 0x7fbca009ff10 con 0x7fbcb4102790 2024-09-10T10:52:14.436 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 36 2024-09-10T10:52:14.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2c00 msgr2=0x7fbca40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2c00 0x7fbca40a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbca8031440 tx=0x7fbca80058d0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4102790 msgr2=0x7fbcb4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4102790 0x7fbcb4074a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbca000e9f0 tx=0x7fbca000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 shutdown_connections 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbca40a2c00 0x7fbca40a50c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcb4106f10 0x7fbcb4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb4102790 0x7fbcb4074a30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 --2- 172.21.15.143:0/1637502831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcb4101de0 0x7fbcb40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.436+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 >> 172.21.15.143:0/1637502831 conn(0x7fbcb40fd880 msgr2=0x7fbcb40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:14.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.437+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 shutdown_connections 2024-09-10T10:52:14.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:14.437+0000 7fbcbac2f640 1 -- 172.21.15.143:0/1637502831 wait complete. 2024-09-10T10:52:14.770 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:14 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:14.770 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:14 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1637502831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:14.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:14 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:14.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:14 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1637502831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:14 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:14.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:14 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1637502831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:15.326 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T10:52:15.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T10:52:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:15.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:15 smithi178 ceph-mon[34813]: pgmap v567: 129 pgs: 129 active+clean; 418 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 798 op/s 2024-09-10T10:52:15.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:15 smithi144 ceph-mon[34230]: pgmap v567: 129 pgs: 129 active+clean; 418 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 798 op/s 2024-09-10T10:52:15.992 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:15 smithi143 ceph-mon[31057]: pgmap v567: 129 pgs: 129 active+clean; 418 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 798 op/s 2024-09-10T10:52:16.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/3042065254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7810e230 msgr2=0x7f3e7810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/3042065254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7810e230 0x7f3e7810e610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009900 tx=0x7f3e6802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/3042065254 shutdown_connections 2024-09-10T10:52:16.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/3042065254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e78108b50 0x7f3e78115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/3042065254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e781081f0 0x7f3e78108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/3042065254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e7810e230 0x7f3e7810e610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/3042065254 >> 172.21.15.143:0/3042065254 conn(0x7f3e78077bb0 msgr2=0x7f3e78079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:16.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/3042065254 shutdown_connections 2024-09-10T10:52:16.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.273+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/3042065254 wait complete. 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 Processor -- start 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 -- start start 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 0x7f3e781a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78108b50 0x7f3e781a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e7810e230 0x7f3e7819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781183e0 con 0x7f3e781081f0 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78118260 con 0x7f3e78108b50 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7e9ca640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78118560 con 0x7f3e7810e230 2024-09-10T10:52:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 0x7f3e781a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e7cf67640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e7810e230 0x7f3e7819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 0x7f3e781a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60932/0 (socket says 172.21.15.143:60932) 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e77fff640 1 -- 172.21.15.143:0/1832563042 learned_addr learned my addr 172.21.15.143:0/1832563042 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.274+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78108b50 0x7f3e781a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e77fff640 1 -- 172.21.15.143:0/1832563042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e7810e230 msgr2=0x7f3e7819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e77fff640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e7810e230 0x7f3e7819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e77fff640 1 -- 172.21.15.143:0/1832563042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78108b50 msgr2=0x7f3e781a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e77fff640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78108b50 0x7f3e781a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e77fff640 1 -- 172.21.15.143:0/1832563042 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7819ef30 con 0x7f3e781081f0 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e7810e230 0x7f3e7819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e77fff640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 0x7f3e781a4610 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006010 tx=0x7f3e68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e75ffb640 1 -- 172.21.15.143:0/1832563042 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6803f030 con 0x7f3e781081f0 2024-09-10T10:52:16.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e75ffb640 1 -- 172.21.15.143:0/1832563042 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e68003aa0 con 0x7f3e781081f0 2024-09-10T10:52:16.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7819f1c0 con 0x7f3e781081f0 2024-09-10T10:52:16.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e75ffb640 1 -- 172.21.15.143:0/1832563042 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68035070 con 0x7f3e781081f0 2024-09-10T10:52:16.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.275+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781ab470 con 0x7f3e781081f0 2024-09-10T10:52:16.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.278+0000 7f3e75ffb640 1 -- 172.21.15.143:0/1832563042 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3e68042030 con 0x7f3e781081f0 2024-09-10T10:52:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 Processor -- start 2024-09-10T10:52:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 -- start start 2024-09-10T10:52:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0887b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0888080 0x5619c0892400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0892940 0x5619c0894d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5619c07ed2a0 con 0x5619c0888080 2024-09-10T10:52:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5619c0794b10 con 0x5619c0887760 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e80da3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5619c07e9260 con 0x5619c0892940 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0888080 0x5619c0892400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e77fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0888080 0x5619c0892400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60934/0 (socket says 172.21.15.143:60934) 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e77fff640 1 -- 172.21.15.143:0/3994080251 learned_addr learned my addr 172.21.15.143:0/3994080251 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0887b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.279+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0892940 0x5619c0894d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e7cf67640 1 -- 172.21.15.143:0/3994080251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0892940 msgr2=0x5619c0894d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0892940 0x5619c0894d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e7cf67640 1 -- 172.21.15.143:0/3994080251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0888080 msgr2=0x5619c0892400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0888080 0x5619c0892400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e7cf67640 1 -- 172.21.15.143:0/3994080251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c055eb90 con 0x5619c0887760 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0887b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e70011760 tx=0x7f3e70011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf002331e8102746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e75ffb640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3e640a2c00 0x7f3e640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e6e7fc640 1 -- 172.21.15.143:0/3994080251 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70004d20 con 0x5619c0887760 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e6e7fc640 1 -- 172.21.15.143:0/3994080251 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7000f9a0 con 0x5619c0887760 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e6e7fc640 1 -- 172.21.15.143:0/3994080251 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700043d0 con 0x5619c0887760 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e77fff640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3e640a2c00 0x7f3e640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.280+0000 7f3e75ffb640 1 -- 172.21.15.143:0/1832563042 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3e680ef6b0 con 0x7f3e781081f0 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 -- 172.21.15.143:0/3994080251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 msgr2=0x5619c0887b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e77fff640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3e640a2c00 0x7f3e640a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3e680ecdf0 tx=0x7f3e680f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0887b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e70011760 tx=0x7f3e70011c20 comp rx=0 tx=0).stop 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 -- 172.21.15.143:0/3994080251 shutdown_connections 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0892940 0x5619c0894d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0888080 0x5619c0892400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 --2- 172.21.15.143:0/3994080251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0887b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 -- 172.21.15.143:0/3994080251 >> 172.21.15.143:0/3994080251 conn(0x5619c0883cf0 msgr2=0x5619c08840d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 -- 172.21.15.143:0/3994080251 shutdown_connections 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.281+0000 7f3e80da3740 1 -- 172.21.15.143:0/3994080251 wait complete. 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 -- start start 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 Processor -- start 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c088f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c088fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c0890240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5619c0895510 con 0x5619c0892940 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5619c07756e0 con 0x5619c0888080 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5619c0895690 con 0x5619c0887760 2024-09-10T10:52:16.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.282+0000 7f3e80da3740 10 client.0 _ll_register_callbacks cb 0x5619c0799a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e6ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c088fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c088f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e6ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c088fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38356/0 (socket says 172.21.15.143:38356) 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e6ffff640 1 -- 172.21.15.143:0/4095794412 learned_addr learned my addr 172.21.15.143:0/4095794412 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c088f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:41994/0 (socket says 172.21.15.143:41994) 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c0890240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 -- 172.21.15.143:0/4095794412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 msgr2=0x5619c088fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c088fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 -- 172.21.15.143:0/4095794412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 msgr2=0x5619c0890240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c0890240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.283+0000 7f3e7cf67640 1 -- 172.21.15.143:0/4095794412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c089e9e0 con 0x5619c0887760 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c088f7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e70011760 tx=0x7f3e70004050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f38ac14f670b5b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e577fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:52:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e577fe640 1 -- 172.21.15.143:0/4095794412 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7002d060 con 0x5619c0887760 2024-09-10T10:52:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7000f9a0 con 0x5619c0887760 2024-09-10T10:52:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e577fe640 1 -- 172.21.15.143:0/4095794412 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3e70011250 con 0x5619c0887760 2024-09-10T10:52:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e577fe640 1 -- 172.21.15.143:0/4095794412 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3e70034050 con 0x5619c0887760 2024-09-10T10:52:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e80da3740 10 client.26464 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5619c08a1350 con 0x5619c0887760 2024-09-10T10:52:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e577fe640 1 -- 172.21.15.143:0/4095794412 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7000f9a0 con 0x5619c0887760 2024-09-10T10:52:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.284+0000 7f3e577fe640 1 -- 172.21.15.143:0/4095794412 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f3e70024070 con 0x5619c0887760 2024-09-10T10:52:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e80da3740 10 client.26464 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e80da3740 10 client.26464 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e80da3740 20 client.26464 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e80da3740 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c08a2810 0x5619c08a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e80da3740 4 client.26464 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619c07fa9d0 con 0x5619c08a2810 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c08a2810 0x5619c08a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c08a2810 0x5619c08a4c00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f150e54426cca0e server_cookie=93dc7e7392740b19 in_seq=0 out_seq=0 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.285+0000 7f3e577fe640 10 client.26464 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e577fe640 1 -- 172.21.15.143:0/4095794412 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619c07fa9d0 con 0x5619c08a2810 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e577fe640 10 client.26464 handle_command_reply: tid=0 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 client.26464 shutdown 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 msgr2=0x5619c088f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c088f7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e70011760 tx=0x7f3e70004050 comp rx=0 tx=0).stop 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 shutdown_connections 2024-09-10T10:52:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c0890240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c088fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c08a2810 0x5619c08a4c00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 --2- 172.21.15.143:0/4095794412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c088f7c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e577fe640 0 client.26464 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.286+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 >> 172.21.15.143:0/4095794412 conn(0x5619c0883cf0 msgr2=0x5619c0889a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.287+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 shutdown_connections 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.287+0000 7f3e80da3740 1 -- 172.21.15.143:0/4095794412 wait complete. 2024-09-10T10:52:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.287+0000 7f3e80da3740 20 client.26464 trim_cache size 0 max 16384 2024-09-10T10:52:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 Processor -- start 2024-09-10T10:52:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 -- start start 2024-09-10T10:52:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0890090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0888080 0x5619c0882d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b51e0 con 0x5619c0892940 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b6d20 con 0x5619c0887760 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e80da3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b58b0 con 0x5619c0888080 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e7cf67640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0890090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e7cf67640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0890090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38372/0 (socket says 172.21.15.143:38372) 2024-09-10T10:52:16.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e7cf67640 1 -- 172.21.15.143:0/62277940 learned_addr learned my addr 172.21.15.143:0/62277940 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.312+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.313+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0888080 0x5619c0882d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.313+0000 7f3e7cf67640 1 -- 172.21.15.143:0/62277940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0888080 msgr2=0x5619c0882d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.313+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0888080 0x5619c0882d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.313+0000 7f3e7cf67640 1 -- 172.21.15.143:0/62277940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 msgr2=0x5619c08aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.313+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.313+0000 7f3e7cf67640 1 -- 172.21.15.143:0/62277940 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c0883b20 con 0x5619c0887760 2024-09-10T10:52:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.314+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0890090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e70004120 tx=0x7f3e70011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce15ea2ec9699a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.314+0000 7f3e6dffb640 1 -- 172.21.15.143:0/62277940 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7001e040 con 0x5619c0887760 2024-09-10T10:52:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.314+0000 7f3e6dffb640 1 -- 172.21.15.143:0/62277940 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7000f790 con 0x5619c0887760 2024-09-10T10:52:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.314+0000 7f3e6dffb640 1 -- 172.21.15.143:0/62277940 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700048c0 con 0x5619c0887760 2024-09-10T10:52:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 -- 172.21.15.143:0/62277940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 msgr2=0x5619c0890090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0890090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e70004120 tx=0x7f3e70011c20 comp rx=0 tx=0).stop 2024-09-10T10:52:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 -- 172.21.15.143:0/62277940 shutdown_connections 2024-09-10T10:52:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0888080 0x5619c0882d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 --2- 172.21.15.143:0/62277940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0887760 0x5619c0890090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 -- 172.21.15.143:0/62277940 >> 172.21.15.143:0/62277940 conn(0x5619c0883cf0 msgr2=0x5619c0882650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 -- 172.21.15.143:0/62277940 shutdown_connections 2024-09-10T10:52:16.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.315+0000 7f3e80da3740 1 -- 172.21.15.143:0/62277940 wait complete. 2024-09-10T10:52:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.316+0000 7f3e80da3740 1 -- start start 2024-09-10T10:52:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.316+0000 7f3e80da3740 1 Processor -- start 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.316+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c089a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c089ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e80da3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e80da3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b6650 con 0x5619c0892940 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e80da3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b5f80 con 0x5619c0888080 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e80da3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b31c0 con 0x5619c0887760 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e80da3740 10 client.0 _ll_register_callbacks cb 0x7f3e7000ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c089a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c089a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42018/0 (socket says 172.21.15.143:42018) 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 -- 172.21.15.143:0/2975791746 learned_addr learned my addr 172.21.15.143:0/2975791746 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c089ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 -- 172.21.15.143:0/2975791746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 msgr2=0x5619c089ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c089ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 -- 172.21.15.143:0/2975791746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 msgr2=0x5619c08978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.317+0000 7f3e7cf67640 1 -- 172.21.15.143:0/2975791746 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c087f260 con 0x5619c0887760 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e7cf67640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c089a8e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e70000f80 tx=0x7f3e700047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ff481e11b5a0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e577fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e577fe640 1 -- 172.21.15.143:0/2975791746 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7002d050 con 0x5619c0887760 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e577fe640 1 -- 172.21.15.143:0/2975791746 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3e700058a0 con 0x5619c0887760 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7000f790 con 0x5619c0887760 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e577fe640 1 -- 172.21.15.143:0/2975791746 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3e70048050 con 0x5619c0887760 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e80da3740 10 client.26470 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5619c090d810 con 0x5619c0887760 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e577fe640 1 -- 172.21.15.143:0/2975791746 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7000f790 con 0x5619c0887760 2024-09-10T10:52:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e577fe640 1 -- 172.21.15.143:0/2975791746 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f3e70024070 con 0x5619c0887760 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.318+0000 7f3e80da3740 10 client.26470 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e80da3740 10 client.26470 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e80da3740 20 client.26470 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e80da3740 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c090df10 0x5619c0910300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e80da3740 4 client.26470 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5619c07f4d20 con 0x5619c090df10 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c090df10 0x5619c0910300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e6ffff640 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c090df10 0x5619c0910300 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff4ff3afb705ddb server_cookie=b1a9b29bb175f6e0 in_seq=0 out_seq=0 2024-09-10T10:52:16.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:16.319+0000 7f3e577fe640 10 client.26470 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:18.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:18 smithi178 ceph-mon[34813]: pgmap v568: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 1.09k op/s 2024-09-10T10:52:18.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:18 smithi144 ceph-mon[34230]: pgmap v568: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 1.09k op/s 2024-09-10T10:52:19.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:18 smithi143 ceph-mon[31057]: pgmap v568: 129 pgs: 129 active+clean; 424 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 1.09k op/s 2024-09-10T10:52:20.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:20 smithi178 ceph-mon[34813]: pgmap v569: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.19k op/s 2024-09-10T10:52:20.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:20 smithi144 ceph-mon[34230]: pgmap v569: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.19k op/s 2024-09-10T10:52:21.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:20 smithi143 ceph-mon[31057]: pgmap v569: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.19k op/s 2024-09-10T10:52:21.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:21 smithi178 ceph-mon[34813]: pgmap v570: 129 pgs: 129 active+clean; 429 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.17k op/s 2024-09-10T10:52:21.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:21 smithi144 ceph-mon[34230]: pgmap v570: 129 pgs: 129 active+clean; 429 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.17k op/s 2024-09-10T10:52:22.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.029+0000 7f3e577fe640 1 -- 172.21.15.143:0/2975791746 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5619c07f4d20 con 0x5619c090df10 2024-09-10T10:52:22.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.029+0000 7f3e577fe640 10 client.26470 handle_command_reply: tid=0 2024-09-10T10:52:22.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.029+0000 7f3e80da3740 1 client.26470 shutdown 2024-09-10T10:52:22.032 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:21 smithi143 ceph-mon[31057]: pgmap v570: 129 pgs: 129 active+clean; 429 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.17k op/s 2024-09-10T10:52:22.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.029+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 msgr2=0x5619c089a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:22.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.029+0000 7f3e80da3740 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c089a8e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e70000f80 tx=0x7f3e700047d0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 shutdown_connections 2024-09-10T10:52:22.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5619c0892940 0x5619c08978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619c0888080 0x5619c089ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x5619c090df10 0x5619c0910300 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e577fe640 0 client.26470 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 --2- 172.21.15.143:0/2975791746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5619c0887760 0x5619c089a8e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 >> 172.21.15.143:0/2975791746 conn(0x5619c0883cf0 msgr2=0x5619c0889980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 shutdown_connections 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 1 -- 172.21.15.143:0/2975791746 wait complete. 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.030+0000 7f3e80da3740 20 client.26470 trim_cache size 0 max 16384 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:52:22.034 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 22, 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "768305f7": { 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 22, 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 140.1, 2024-09-10T10:52:22.035 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "QUIESCED", 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:22.036 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3e640a2c00 msgr2=0x7f3e640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3e640a2c00 0x7f3e640a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3e680ecdf0 tx=0x7f3e680f5530 comp rx=0 tx=0).stop 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 msgr2=0x7f3e781a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 0x7f3e781a4610 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006010 tx=0x7f3e68004270 comp rx=0 tx=0).stop 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 shutdown_connections 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3e640a2c00 0x7f3e640a50c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e7810e230 0x7f3e7819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e78108b50 0x7f3e781a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 --2- 172.21.15.143:0/1832563042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e781081f0 0x7f3e781a4610 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.031+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 >> 172.21.15.143:0/1832563042 conn(0x7f3e78077bb0 msgr2=0x7f3e780794d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.032+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 shutdown_connections 2024-09-10T10:52:22.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:22.032+0000 7f3e7e9ca640 1 -- 172.21.15.143:0/1832563042 wait complete. 2024-09-10T10:52:22.969 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '768305f7', quiesce took 9.8 seconds. Will release after: 10.3 2024-09-10T10:52:23.592 INFO:tasks.workunit.client.0.smithi143.stdout: 96 4371 30 2999 7 1678 23 2024-09-10T10:52:24.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:24 smithi178 ceph-mon[34813]: pgmap v571: 129 pgs: 129 active+clean; 429 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 1.09k op/s 2024-09-10T10:52:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:24 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:52:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:24 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:52:24.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:24 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:52:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:24 smithi144 ceph-mon[34230]: pgmap v571: 129 pgs: 129 active+clean; 429 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 1.09k op/s 2024-09-10T10:52:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:24 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:52:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:24 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:52:24.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:24 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:52:25.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:24 smithi143 ceph-mon[31057]: pgmap v571: 129 pgs: 129 active+clean; 429 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 1.09k op/s 2024-09-10T10:52:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:24 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:52:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:24 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:52:25.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:24 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:52:26.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:25 smithi143 ceph-mon[31057]: pgmap v572: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 582 op/s 2024-09-10T10:52:26.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:25 smithi178 ceph-mon[34813]: pgmap v572: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 582 op/s 2024-09-10T10:52:26.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:25 smithi144 ceph-mon[34230]: pgmap v572: 129 pgs: 129 active+clean; 428 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 582 op/s 2024-09-10T10:52:28.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:28 smithi178 ceph-mon[34813]: pgmap v573: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-10T10:52:28.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:28 smithi144 ceph-mon[34230]: pgmap v573: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-10T10:52:29.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:28 smithi143 ceph-mon[31057]: pgmap v573: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-10T10:52:29.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:29 smithi178 ceph-mon[34813]: pgmap v574: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 742 KiB/s wr, 151 op/s 2024-09-10T10:52:29.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:29 smithi144 ceph-mon[34230]: pgmap v574: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 742 KiB/s wr, 151 op/s 2024-09-10T10:52:30.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:29 smithi143 ceph-mon[31057]: pgmap v574: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 742 KiB/s wr, 151 op/s 2024-09-10T10:52:32.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:31 smithi143 ceph-mon[31057]: pgmap v575: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-09-10T10:52:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:31 smithi178 ceph-mon[34813]: pgmap v575: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-09-10T10:52:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:31 smithi144 ceph-mon[34230]: pgmap v575: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-09-10T10:52:33.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:52:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:34.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:33 smithi143 ceph-mon[31057]: pgmap v576: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 22 op/s 2024-09-10T10:52:34.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:33 smithi178 ceph-mon[34813]: pgmap v576: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 22 op/s 2024-09-10T10:52:34.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:33 smithi144 ceph-mon[34230]: pgmap v576: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 22 op/s 2024-09-10T10:52:36.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:35 smithi143 ceph-mon[31057]: pgmap v577: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 18 op/s 2024-09-10T10:52:36.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:35 smithi178 ceph-mon[34813]: pgmap v577: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 18 op/s 2024-09-10T10:52:36.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:35 smithi144 ceph-mon[34230]: pgmap v577: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 18 op/s 2024-09-10T10:52:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 -- 172.21.15.143:0/2844070074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 msgr2=0x7f2bd01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 --2- 172.21.15.143:0/2844070074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01029b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009900 tx=0x7f2bc002a480 comp rx=0 tx=0).stop 2024-09-10T10:52:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 -- 172.21.15.143:0/2844070074 shutdown_connections 2024-09-10T10:52:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 --2- 172.21.15.143:0/2844070074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd0102ef0 0x7f2bd010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 --2- 172.21.15.143:0/2844070074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01029b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 --2- 172.21.15.143:0/2844070074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 0x7f2bd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 -- 172.21.15.143:0/2844070074 >> 172.21.15.143:0/2844070074 conn(0x7f2bd00fc7d0 msgr2=0x7f2bd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 -- 172.21.15.143:0/2844070074 shutdown_connections 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.918+0000 7f2bd8682640 1 -- 172.21.15.143:0/2844070074 wait complete. 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 Processor -- start 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 -- start start 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 0x7f2bd01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd0102ef0 0x7f2bd019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0110e30 con 0x7f2bd0102570 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0110cb0 con 0x7f2bd0102ef0 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd8682640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0110fb0 con 0x7f2bd0100e20 2024-09-10T10:52:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd5c1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd5c1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40136/0 (socket says 172.21.15.143:40136) 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.919+0000 7f2bd5c1d640 1 -- 172.21.15.143:0/688879979 learned_addr learned my addr 172.21.15.143:0/688879979 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd6c1f640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd0102ef0 0x7f2bd019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd641e640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 0x7f2bd01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd5c1d640 1 -- 172.21.15.143:0/688879979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 msgr2=0x7f2bd01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd5c1d640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 0x7f2bd01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd5c1d640 1 -- 172.21.15.143:0/688879979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd0102ef0 msgr2=0x7f2bd019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd5c1d640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd0102ef0 0x7f2bd019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd5c1d640 1 -- 172.21.15.143:0/688879979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd019d460 con 0x7f2bd0102570 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd641e640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 0x7f2bd01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd5c1d640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01989f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0006010 tx=0x7f2bc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc003f070 con 0x7f2bd0102570 2024-09-10T10:52:36.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc0003aa0 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0035030 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd019d6f0 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.920+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a5030 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.921+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd01021b0 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.922+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2bc0042070 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.923+0000 7f2bc77fe640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bbc0a2b80 0x7f2bbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.923+0000 7f2bd641e640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bbc0a2b80 0x7f2bbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.923+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2bc00efff0 con 0x7f2bd0102570 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.923+0000 7f2bd641e640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bbc0a2b80 0x7f2bbc0a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0058c0 tx=0x7f2bcc005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:36.926+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bc00bcbf0 con 0x7f2bd0102570 2024-09-10T10:52:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.132+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bd010a0a0 con 0x7f2bd0102570 2024-09-10T10:52:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.133+0000 7f2bc77fe640 1 -- 172.21.15.143:0/688879979 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12001 (secure 0 0 0) 0x7f2bc00bc4f0 con 0x7f2bd0102570 2024-09-10T10:52:37.135 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:52:37.135 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"btime":"2024-09-10T10:52:11:648994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:52:11.410829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:52:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 36 2024-09-10T10:52:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.135+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bbc0a2b80 msgr2=0x7f2bbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.135+0000 7f2bd8682640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bbc0a2b80 0x7f2bbc0a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0058c0 tx=0x7f2bcc005d30 comp rx=0 tx=0).stop 2024-09-10T10:52:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 msgr2=0x7f2bd01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01989f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0006010 tx=0x7f2bc00042f0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 shutdown_connections 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2bbc0a2b80 0x7f2bbc0a5040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd0102ef0 0x7f2bd019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bd0102570 0x7f2bd01989f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 --2- 172.21.15.143:0/688879979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd0100e20 0x7f2bd01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 >> 172.21.15.143:0/688879979 conn(0x7f2bd00fc7d0 msgr2=0x7f2bd006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 shutdown_connections 2024-09-10T10:52:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:37.136+0000 7f2bd8682640 1 -- 172.21.15.143:0/688879979 wait complete. 2024-09-10T10:52:37.462 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:37 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/688879979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:37.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:37 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/688879979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:37.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:37 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/688879979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:37.883 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 768305f7 --release --await' 2024-09-10T10:52:38.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 768305f7 --release --await 2024-09-10T10:52:38.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:38 smithi143 ceph-mon[31057]: pgmap v578: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-09-10T10:52:38.593 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:38.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:38 smithi178 ceph-mon[34813]: pgmap v578: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-09-10T10:52:38.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:38 smithi144 ceph-mon[34230]: pgmap v578: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-09-10T10:52:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 -- 172.21.15.143:0/1769772246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554102790 msgr2=0x7f65541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 --2- 172.21.15.143:0/1769772246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554102790 0x7f65541067e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f654402a410 comp rx=0 tx=0).stop 2024-09-10T10:52:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 -- 172.21.15.143:0/1769772246 shutdown_connections 2024-09-10T10:52:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 --2- 172.21.15.143:0/1769772246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554106f10 0x7f655410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 --2- 172.21.15.143:0/1769772246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554102790 0x7f65541067e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 --2- 172.21.15.143:0/1769772246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6554101de0 0x7f65541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 -- 172.21.15.143:0/1769772246 >> 172.21.15.143:0/1769772246 conn(0x7f65540fd8c0 msgr2=0x7f65540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 -- 172.21.15.143:0/1769772246 shutdown_connections 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.104+0000 7f655b7bf640 1 -- 172.21.15.143:0/1769772246 wait complete. 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 Processor -- start 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 -- start start 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 0x7f6554198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554102790 0x7f65541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6554106f10 0x7f655419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6554111e20 con 0x7f6554101de0 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6554111ca0 con 0x7f6554102790 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655b7bf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6554111fa0 con 0x7f6554106f10 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655955b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 0x7f6554198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655955b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 0x7f6554198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40158/0 (socket says 172.21.15.143:40158) 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f655955b640 1 -- 172.21.15.143:0/3949220417 learned_addr learned my addr 172.21.15.143:0/3949220417 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f6559d5c640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6554106f10 0x7f655419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.105+0000 7f6558d5a640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554102790 0x7f65541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655955b640 1 -- 172.21.15.143:0/3949220417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6554106f10 msgr2=0x7f655419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655955b640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6554106f10 0x7f655419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655955b640 1 -- 172.21.15.143:0/3949220417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554102790 msgr2=0x7f65541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655955b640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554102790 0x7f65541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655955b640 1 -- 172.21.15.143:0/3949220417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655419d430 con 0x7f6554101de0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f6558d5a640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554102790 0x7f65541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655955b640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 0x7f6554198480 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f655000d770 tx=0x7f655000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f654a7fc640 1 -- 172.21.15.143:0/3949220417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550017070 con 0x7f6554101de0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f654a7fc640 1 -- 172.21.15.143:0/3949220417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65500054d0 con 0x7f6554101de0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655419d720 con 0x7f6554101de0 2024-09-10T10:52:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f654a7fc640 1 -- 172.21.15.143:0/3949220417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550004cf0 con 0x7f6554101de0 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.106+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541a5000 con 0x7f6554101de0 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f654a7fc640 1 -- 172.21.15.143:0/3949220417 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6550004e90 con 0x7f6554101de0 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 Processor -- start 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 -- start start 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159e47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e159dac620 con 0x55e159e47980 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e159d53e50 con 0x55e159e47060 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655db98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e159da85e0 con 0x55e159e52240 2024-09-10T10:52:39.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.109+0000 7f655955b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e51d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40166/0 (socket says 172.21.15.143:40166) 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 -- 172.21.15.143:0/1101544501 learned_addr learned my addr 172.21.15.143:0/1101544501 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f654a7fc640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65400a2b80 0x7f65400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f6558d5a640 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f6559d5c640 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159e47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f6559d5c640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65400a2b80 0x7f65400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f654a7fc640 1 -- 172.21.15.143:0/3949220417 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f65500cee10 con 0x7f6554101de0 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 -- 172.21.15.143:0/1101544501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 msgr2=0x55e159e54630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e54630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 -- 172.21.15.143:0/1101544501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 msgr2=0x55e159e47440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159e47440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 -- 172.21.15.143:0/1101544501 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e159d44d40 con 0x55e159e47980 2024-09-10T10:52:39.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f6558d5a640 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f655955b640 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e51d00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f655009b010 tx=0x7f6550005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccc96f00463e459 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f6559d5c640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65400a2b80 0x7f65400a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f654c005fd0 tx=0x7f654c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f653f7fe640 1 -- 172.21.15.143:0/1101544501 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550020070 con 0x55e159e47980 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f653f7fe640 1 -- 172.21.15.143:0/1101544501 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6550009c30 con 0x55e159e47980 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.110+0000 7f653f7fe640 1 -- 172.21.15.143:0/1101544501 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500a3f80 con 0x55e159e47980 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 -- 172.21.15.143:0/1101544501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 msgr2=0x55e159e51d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e51d00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f655009b010 tx=0x7f6550005740 comp rx=0 tx=0).stop 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 -- 172.21.15.143:0/1101544501 shutdown_connections 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e51d00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 --2- 172.21.15.143:0/1101544501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159e47440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 -- 172.21.15.143:0/1101544501 >> 172.21.15.143:0/1101544501 conn(0x55e159e435f0 msgr2=0x55e159e439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 -- 172.21.15.143:0/1101544501 shutdown_connections 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 -- 172.21.15.143:0/1101544501 wait complete. 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.111+0000 7f655db98740 1 -- start start 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 Processor -- start 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e4f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47980 0x55e159e4f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 0x55e159e4fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e159e54db0 con 0x55e159e52240 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e159d34f10 con 0x55e159e47980 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e159e54f30 con 0x55e159e47060 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655db98740 10 client.0 _ll_register_callbacks cb 0x55e159d58db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 0x55e159e4fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 0x55e159e4fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40176/0 (socket says 172.21.15.143:40176) 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 -- 172.21.15.143:0/2745801707 learned_addr learned my addr 172.21.15.143:0/2745801707 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6558d5a640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47980 0x55e159e4f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f655955b640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e4f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 -- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 msgr2=0x55e159e4f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e4f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 -- 172.21.15.143:0/2745801707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47980 msgr2=0x55e159e4f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47980 0x55e159e4f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.112+0000 7f6559d5c640 1 -- 172.21.15.143:0/2745801707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e159abb580 con 0x55e159e52240 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f6559d5c640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 0x55e159e4fbd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f654c00ff20 tx=0x7f654c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef750b9a1ccf598 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f653cff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f653cff9640 1 -- 172.21.15.143:0/2745801707 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c02e060 con 0x55e159e52240 2024-09-10T10:52:39.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f653cff9640 1 -- 172.21.15.143:0/2745801707 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f654c021cc0 con 0x55e159e52240 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e159db9b90 con 0x55e159e52240 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f653cff9640 1 -- 172.21.15.143:0/2745801707 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f654c035050 con 0x55e159e52240 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 10 client.17079 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e159b85960 con 0x55e159e52240 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f653cff9640 1 -- 172.21.15.143:0/2745801707 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e159db9b90 con 0x55e159e52240 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f653cff9640 1 -- 172.21.15.143:0/2745801707 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f654c01d070 con 0x55e159e52240 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 10 client.17079 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 10 client.17079 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 20 client.17079 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159e61e30 0x55e159e64260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 4 client.17079 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.113+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e159db40a0 con 0x55e159e61e30 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.114+0000 7f655955b640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159e61e30 0x55e159e64260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.114+0000 7f653cff9640 10 client.17079 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.114+0000 7f655955b640 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159e61e30 0x55e159e64260 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed1d98caafd1f8f server_cookie=631e49afa677c925 in_seq=0 out_seq=0 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f653cff9640 1 -- 172.21.15.143:0/2745801707 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6550080e80 con 0x55e159e61e30 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f653cff9640 10 client.17079 handle_command_reply: tid=0 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 client.17079 shutdown 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 msgr2=0x55e159e4fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 0x55e159e4fbd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f654c00ff20 tx=0x7f654c004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 shutdown_connections 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e52240 0x55e159e4fbd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47980 0x55e159e4f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159e61e30 0x55e159e64260 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 --2- 172.21.15.143:0/2745801707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e4f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 >> 172.21.15.143:0/2745801707 conn(0x55e159e435f0 msgr2=0x55e159e49320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f653cff9640 0 client.17079 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 shutdown_connections 2024-09-10T10:52:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 1 -- 172.21.15.143:0/2745801707 wait complete. 2024-09-10T10:52:39.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.115+0000 7f655db98740 20 client.17079 trim_cache size 0 max 16384 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.132+0000 7f655db98740 1 Processor -- start 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 -- start start 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e69b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e6a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e52240 0x55e159e4c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65541aeaf0 con 0x55e159e47980 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65541b0630 con 0x55e159e52240 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655db98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65541af1c0 con 0x55e159e47060 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e6a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e6a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40190/0 (socket says 172.21.15.143:40190) 2024-09-10T10:52:39.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 -- 172.21.15.143:0/2651264543 learned_addr learned my addr 172.21.15.143:0/2651264543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f655955b640 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e69b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6559d5c640 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e52240 0x55e159e4c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 -- 172.21.15.143:0/2651264543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 msgr2=0x55e159e69b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e69b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 -- 172.21.15.143:0/2651264543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e52240 msgr2=0x55e159e4c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e52240 0x55e159e4c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 -- 172.21.15.143:0/2651264543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e159e648e0 con 0x55e159e47980 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6559d5c640 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e52240 0x55e159e4c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.133+0000 7f6558d5a640 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e6a070 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f6544032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230fa87925bd453d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f653effd640 1 -- 172.21.15.143:0/2651264543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544035040 con 0x55e159e47980 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f653effd640 1 -- 172.21.15.143:0/2651264543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6544041070 con 0x55e159e47980 2024-09-10T10:52:39.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f653effd640 1 -- 172.21.15.143:0/2651264543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440046c0 con 0x55e159e47980 2024-09-10T10:52:39.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 -- 172.21.15.143:0/2651264543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 msgr2=0x55e159e6a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e6a070 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6544009900 tx=0x7f6544032e80 comp rx=0 tx=0).stop 2024-09-10T10:52:39.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 -- 172.21.15.143:0/2651264543 shutdown_connections 2024-09-10T10:52:39.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e52240 0x55e159e4c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159e6a070 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 --2- 172.21.15.143:0/2651264543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e47060 0x55e159e69b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.134+0000 7f655db98740 1 -- 172.21.15.143:0/2651264543 >> 172.21.15.143:0/2651264543 conn(0x55e159e435f0 msgr2=0x55e159e509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.135+0000 7f655db98740 1 -- 172.21.15.143:0/2651264543 shutdown_connections 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.135+0000 7f655db98740 1 -- 172.21.15.143:0/2651264543 wait complete. 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.135+0000 7f655db98740 1 -- start start 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.135+0000 7f655db98740 1 Processor -- start 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159ec6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159ec6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e5a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65541aff60 con 0x55e159e47980 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65541af890 con 0x55e159e47060 2024-09-10T10:52:39.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65541acad0 con 0x55e159e52240 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 10 client.0 _ll_register_callbacks cb 0x7f654c009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159ec6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159ec6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40194/0 (socket says 172.21.15.143:40194) 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 -- 172.21.15.143:0/4136319761 learned_addr learned my addr 172.21.15.143:0/4136319761 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655955b640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159ec6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6559d5c640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e5a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 -- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 msgr2=0x55e159e5a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e5a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 -- 172.21.15.143:0/4136319761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 msgr2=0x55e159ec6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159ec6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 -- 172.21.15.143:0/4136319761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e159e5a6c0 con 0x55e159e47980 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655955b640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159ec6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:39.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f6558d5a640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159ec6660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6544005ab0 tx=0x7f65440056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df64b4f71bba6bae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f653cff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f653cff9640 1 -- 172.21.15.143:0/4136319761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654404d060 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f653cff9640 1 -- 172.21.15.143:0/4136319761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6544004030 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.136+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6544041070 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f653cff9640 1 -- 172.21.15.143:0/4136319761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6544035030 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 10 client.17085 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e159ed1930 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f653cff9640 1 -- 172.21.15.143:0/4136319761 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6544041070 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f653cff9640 1 -- 172.21.15.143:0/4136319761 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f65440046c0 con 0x55e159e47980 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 10 client.17085 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 10 client.17085 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:52:39.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 20 client.17085 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:39.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159ed2080 0x55e159ed44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:39.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 4 client.17085 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "768305f7", "release": true, "await": true}] 2024-09-10T10:52:39.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "set_id": "768305f7", "release": true, "await": true}) -- 0x55e159d305a0 con 0x55e159ed2080 2024-09-10T10:52:39.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.137+0000 7f655955b640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159ed2080 0x55e159ed44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:39.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.138+0000 7f653cff9640 10 client.17085 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:39.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.138+0000 7f655955b640 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159ed2080 0x55e159ed44a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eab57734b482578 server_cookie=13f0413c34d52718 in_seq=0 out_seq=0 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f653cff9640 1 -- 172.21.15.143:0/4136319761 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e159d305a0 con 0x55e159ed2080 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f653cff9640 10 client.17085 handle_command_reply: tid=0 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 client.17085 shutdown 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 msgr2=0x55e159ec6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159ec6660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6544005ab0 tx=0x7f65440056b0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 shutdown_connections 2024-09-10T10:52:39.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55e159ed2080 0x55e159ed44a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f653cff9640 0 client.17085 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e159e52240 0x55e159e5a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e159e47980 0x55e159ec6660 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 --2- 172.21.15.143:0/4136319761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e159e47060 0x55e159ec6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 >> 172.21.15.143:0/4136319761 conn(0x55e159e435f0 msgr2=0x55e159e48b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 shutdown_connections 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.919+0000 7f655db98740 1 -- 172.21.15.143:0/4136319761 wait complete. 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655db98740 20 client.17085 trim_cache size 0 max 16384 2024-09-10T10:52:39.922 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 24, 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": { 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "768305f7": { 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "version": 24, 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "age_ref": 0.0, 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:52:39.923 INFO:teuthology.orchestra.run.smithi143.stdout: "timeout": 90.0, 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "expiration": 140.1, 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "members": { 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "file:/": { 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "excluded": false, 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "state": { 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RELEASED", 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: "age": 0.0 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:39.924 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:52:39.925 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65400a2b80 msgr2=0x7f65400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65400a2b80 0x7f65400a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f654c005fd0 tx=0x7f654c009500 comp rx=0 tx=0).stop 2024-09-10T10:52:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 msgr2=0x7f6554198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 0x7f6554198480 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f655000d770 tx=0x7f655000dc30 comp rx=0 tx=0).stop 2024-09-10T10:52:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 shutdown_connections 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65400a2b80 0x7f65400a5040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6554106f10 0x7f655419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6554102790 0x7f65541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 --2- 172.21.15.143:0/3949220417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6554101de0 0x7f6554198480 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 >> 172.21.15.143:0/3949220417 conn(0x7f65540fd8c0 msgr2=0x7f65540fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 shutdown_connections 2024-09-10T10:52:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:39.920+0000 7f655b7bf640 1 -- 172.21.15.143:0/3949220417 wait complete. 2024-09-10T10:52:40.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:39 smithi143 ceph-mon[31057]: pgmap v579: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T10:52:40.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:39 smithi178 ceph-mon[34813]: pgmap v579: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T10:52:40.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:39 smithi144 ceph-mon[34230]: pgmap v579: 129 pgs: 129 active+clean; 420 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T10:52:40.739 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '768305f7', total seconds elapsed: 27.6 2024-09-10T10:52:42.033 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:41 smithi143 ceph-mon[31057]: pgmap v580: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 47 KiB/s wr, 13 op/s 2024-09-10T10:52:42.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:41 smithi178 ceph-mon[34813]: pgmap v580: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 47 KiB/s wr, 13 op/s 2024-09-10T10:52:42.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:41 smithi144 ceph-mon[34230]: pgmap v580: 129 pgs: 129 active+clean; 421 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 47 KiB/s wr, 13 op/s 2024-09-10T10:52:42.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:52:42.852 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:43.117 INFO:tasks.workunit.client.0.smithi143.stdout: 2024-09-10T10:52:43.117 INFO:tasks.workunit.client.0.smithi143.stdout:Final score for writes: 96 2024-09-10T10:52:43.117 INFO:tasks.workunit.client.0.smithi143.stdout:Final score for reads : 4342 2024-09-10T10:52:43.117 INFO:tasks.workunit.client.0.smithi143.stdout: 2024-09-10T10:52:43.117 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T10:52:43.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 -- 172.21.15.143:0/1452507491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102570 msgr2=0x7efd541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/1452507491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102570 0x7efd541029b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efd44009900 tx=0x7efd4402a480 comp rx=0 tx=0).stop 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 -- 172.21.15.143:0/1452507491 shutdown_connections 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/1452507491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd54102ef0 0x7efd5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/1452507491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102570 0x7efd541029b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/1452507491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 -- 172.21.15.143:0/1452507491 >> 172.21.15.143:0/1452507491 conn(0x7efd540fc7d0 msgr2=0x7efd540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 -- 172.21.15.143:0/1452507491 shutdown_connections 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 -- 172.21.15.143:0/1452507491 wait complete. 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.359+0000 7efd5b7d0640 1 Processor -- start 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 -- start start 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd54102570 0x7efd54198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102ef0 0x7efd5419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd54111f50 con 0x7efd54102ef0 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd54111dd0 con 0x7efd54102570 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5b7d0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd541120d0 con 0x7efd54100e20 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5956c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102ef0 0x7efd5419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5956c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:55896/0 (socket says 172.21.15.143:55896) 2024-09-10T10:52:43.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5956c640 1 -- 172.21.15.143:0/804398330 learned_addr learned my addr 172.21.15.143:0/804398330 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd58d6b640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd54102570 0x7efd54198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 -- 172.21.15.143:0/804398330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 msgr2=0x7efd54198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 -- 172.21.15.143:0/804398330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd54102570 msgr2=0x7efd54198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd54102570 0x7efd54198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 -- 172.21.15.143:0/804398330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5419d3c0 con 0x7efd54102ef0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd5956c640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.360+0000 7efd59d6d640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102ef0 0x7efd5419cce0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009fd0 tx=0x7efd4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.361+0000 7efd4a7fc640 1 -- 172.21.15.143:0/804398330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00ce20 con 0x7efd54102ef0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.361+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5419d6b0 con 0x7efd54102ef0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.361+0000 7efd4a7fc640 1 -- 172.21.15.143:0/804398330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd4c005020 con 0x7efd54102ef0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.361+0000 7efd4a7fc640 1 -- 172.21.15.143:0/804398330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c010690 con 0x7efd54102ef0 2024-09-10T10:52:43.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.361+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd541a4f90 con 0x7efd54102ef0 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.363+0000 7efd4a7fc640 1 -- 172.21.15.143:0/804398330 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efd4c010830 con 0x7efd54102ef0 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 Processor -- start 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 -- start start 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c530f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c53102550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53104e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561c530592e0 con 0x561c530f3360 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c53000ce0 con 0x561c530f3c80 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5dba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c530552a0 con 0x561c53102a90 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c530f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c530f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40224/0 (socket says 172.21.15.143:40224) 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd58d6b640 1 -- 172.21.15.143:0/87449098 learned_addr learned my addr 172.21.15.143:0/87449098 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:43.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd5956c640 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c53102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd58d6b640 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.364+0000 7efd4a7fc640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efd400a2b80 0x7efd400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd5956c640 1 -- 172.21.15.143:0/87449098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 msgr2=0x561c53104e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd5956c640 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53104e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd5956c640 1 -- 172.21.15.143:0/87449098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 msgr2=0x561c530f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd5956c640 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c530f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd5956c640 1 -- 172.21.15.143:0/87449098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c52fe5440 con 0x561c530f3c80 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd4a7fc640 1 -- 172.21.15.143:0/804398330 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd4c0cfa70 con 0x7efd54102ef0 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd59d6d640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efd400a2b80 0x7efd400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd5956c640 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c53102550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd50011510 tx=0x7efd500119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62c2cb543a89505 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd3f7fe640 1 -- 172.21.15.143:0/87449098 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5001e070 con 0x561c530f3c80 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd3f7fe640 1 -- 172.21.15.143:0/87449098 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd5000fa10 con 0x561c530f3c80 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.365+0000 7efd3f7fe640 1 -- 172.21.15.143:0/87449098 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50019a80 con 0x561c530f3c80 2024-09-10T10:52:43.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 -- 172.21.15.143:0/87449098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 msgr2=0x561c53102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c53102550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd50011510 tx=0x7efd500119d0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd59d6d640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efd400a2b80 0x7efd400a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd4c09bf00 tx=0x7efd4c005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 -- 172.21.15.143:0/87449098 shutdown_connections 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c53102550 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 --2- 172.21.15.143:0/87449098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c530f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 -- 172.21.15.143:0/87449098 >> 172.21.15.143:0/87449098 conn(0x561c530efa30 msgr2=0x561c530efe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 -- 172.21.15.143:0/87449098 shutdown_connections 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 -- 172.21.15.143:0/87449098 wait complete. 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 -- start start 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.366+0000 7efd5dba9740 1 Processor -- start 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 0x561c53102430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c530ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c53102a90 0x561c530ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561c53105660 con 0x561c53102a90 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c53014050 con 0x561c530f3c80 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c531057e0 con 0x561c530f3360 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 0x561c53102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 0x561c53102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:55914/0 (socket says 172.21.15.143:55914) 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 -- 172.21.15.143:0/342789402 learned_addr learned my addr 172.21.15.143:0/342789402 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5956c640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c530ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c53102a90 0x561c530ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd5dba9740 10 client.0 _ll_register_callbacks cb 0x561c53054160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 -- 172.21.15.143:0/342789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 msgr2=0x561c530ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c530ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 -- 172.21.15.143:0/342789402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c53102a90 msgr2=0x561c530ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c53102a90 0x561c530ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.367+0000 7efd58d6b640 1 -- 172.21.15.143:0/342789402 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c52dca3c0 con 0x561c530f3360 2024-09-10T10:52:43.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd58d6b640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 0x561c53102430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd44006010 tx=0x7efd440058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a019d20d7f3ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd3cff9640 1 -- 172.21.15.143:0/342789402 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44035070 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5000fa10 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd3cff9640 1 -- 172.21.15.143:0/342789402 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd4403e910 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd3cff9640 1 -- 172.21.15.143:0/342789402 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd4403eb10 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 10 client.26500 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561c53111090 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd3cff9640 1 -- 172.21.15.143:0/342789402 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd5000fa10 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd3cff9640 1 -- 172.21.15.143:0/342789402 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7efd44004850 con 0x561c530f3360 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 10 client.26500 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 20 client.26500 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.368+0000 7efd5dba9740 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c53112550 0x561c53114960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.369+0000 7efd5dba9740 4 client.26500 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:43.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.369+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c53066850 con 0x561c53112550 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.369+0000 7efd5956c640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c53112550 0x561c53114960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.369+0000 7efd3cff9640 10 client.26500 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.369+0000 7efd5956c640 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c53112550 0x561c53114960 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed55323b336ecb1 server_cookie=6428160976deea3a in_seq=0 out_seq=0 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd3cff9640 1 -- 172.21.15.143:0/342789402 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c53066850 con 0x561c53112550 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd3cff9640 10 client.26500 handle_command_reply: tid=0 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 client.26500 shutdown 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 msgr2=0x561c53102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 0x561c53102430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd44006010 tx=0x7efd440058d0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 shutdown_connections 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c53102a90 0x561c530ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3c80 0x561c530ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c53112550 0x561c53114960 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 --2- 172.21.15.143:0/342789402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3360 0x561c53102430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd3cff9640 0 client.26500 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:43.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 >> 172.21.15.143:0/342789402 conn(0x561c530efa30 msgr2=0x561c530f5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 shutdown_connections 2024-09-10T10:52:43.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 1 -- 172.21.15.143:0/342789402 wait complete. 2024-09-10T10:52:43.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.370+0000 7efd5dba9740 20 client.26500 trim_cache size 0 max 16384 2024-09-10T10:52:43.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.389+0000 7efd5dba9740 1 Processor -- start 2024-09-10T10:52:43.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.389+0000 7efd5dba9740 1 -- start start 2024-09-10T10:52:43.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.389+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c5310e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.389+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 0x561c5311a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.389+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53102a90 0x561c5311a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5dba9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ac820 con 0x561c530f3360 2024-09-10T10:52:43.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5dba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ae360 con 0x561c53102a90 2024-09-10T10:52:43.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5dba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd541acef0 con 0x561c530f3c80 2024-09-10T10:52:43.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 0x561c5311a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c5310e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 0x561c5311a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:55924/0 (socket says 172.21.15.143:55924) 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 -- 172.21.15.143:0/1691460023 learned_addr learned my addr 172.21.15.143:0/1691460023 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd58d6b640 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53102a90 0x561c5311a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 -- 172.21.15.143:0/1691460023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53102a90 msgr2=0x561c5311a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53102a90 0x561c5311a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 -- 172.21.15.143:0/1691460023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 msgr2=0x561c5310e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c5310e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 -- 172.21.15.143:0/1691460023 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c530fe690 con 0x561c530f3c80 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd58d6b640 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c5310e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd5956c640 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 0x561c5311a230 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd500194f0 tx=0x7efd5000fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=329feb4f77a46ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.390+0000 7efd3effd640 1 -- 172.21.15.143:0/1691460023 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5001e040 con 0x561c530f3c80 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd3effd640 1 -- 172.21.15.143:0/1691460023 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd5000fa10 con 0x561c530f3c80 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd3effd640 1 -- 172.21.15.143:0/1691460023 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd500055a0 con 0x561c530f3c80 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 -- 172.21.15.143:0/1691460023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 msgr2=0x561c5311a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 0x561c5311a230 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd500194f0 tx=0x7efd5000fef0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 -- 172.21.15.143:0/1691460023 shutdown_connections 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c53102a90 0x561c5311a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c530f3c80 0x561c5311a230 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 --2- 172.21.15.143:0/1691460023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3360 0x561c5310e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 -- 172.21.15.143:0/1691460023 >> 172.21.15.143:0/1691460023 conn(0x561c530efa30 msgr2=0x561c530fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 -- 172.21.15.143:0/1691460023 shutdown_connections 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.391+0000 7efd5dba9740 1 -- 172.21.15.143:0/1691460023 wait complete. 2024-09-10T10:52:43.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 -- start start 2024-09-10T10:52:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 Processor -- start 2024-09-10T10:52:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 0x561c5310aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3c80 0x561c5310afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53107a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd541adc90 con 0x561c530f3c80 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd541ad5c0 con 0x561c530f3360 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd541aa800 con 0x561c53102a90 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5dba9740 10 client.0 _ll_register_callbacks cb 0x7efd44031510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 0x561c5310aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd5956c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3c80 0x561c5310afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd58d6b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 0x561c5310aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:54210/0 (socket says 172.21.15.143:54210) 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.392+0000 7efd58d6b640 1 -- 172.21.15.143:0/697617542 learned_addr learned my addr 172.21.15.143:0/697617542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 -- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 msgr2=0x561c53107a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53107a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 -- 172.21.15.143:0/697617542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3c80 msgr2=0x561c5310afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3c80 0x561c5310afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 -- 172.21.15.143:0/697617542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c531027a0 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd58d6b640 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 0x561c5310aa70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd44009900 tx=0x7efd44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49923c5b195c1bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd3cff9640 1 -- 172.21.15.143:0/697617542 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44061060 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd3cff9640 1 -- 172.21.15.143:0/697617542 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7efd4403e030 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5000fa10 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd3cff9640 1 -- 172.21.15.143:0/697617542 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd44068050 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd5dba9740 10 client.26417 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.393+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561c5317d710 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd3cff9640 1 -- 172.21.15.143:0/697617542 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd5000fa10 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd3cff9640 1 -- 172.21.15.143:0/697617542 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7efd4403d450 con 0x561c530f3360 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5dba9740 10 client.26417 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5dba9740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5dba9740 20 client.26417 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5dba9740 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c5317de30 0x561c53180250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5dba9740 4 client.26417 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c53060d60 con 0x561c5317de30 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5956c640 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c5317de30 0x561c53180250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd5956c640 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c5317de30 0x561c53180250 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584b156171a673d9 server_cookie=a30c893f8bdddf4b in_seq=0 out_seq=0 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.394+0000 7efd3cff9640 10 client.26417 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd3cff9640 1 -- 172.21.15.143:0/697617542 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561c53060d60 con 0x561c5317de30 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd3cff9640 10 client.26417 handle_command_reply: tid=0 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 client.26417 shutdown 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 msgr2=0x561c5310aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 0x561c5310aa70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd44009900 tx=0x7efd44000bc0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 shutdown_connections 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x561c5317de30 0x561c53180250 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c53102a90 0x561c53107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c530f3c80 0x561c5310afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 --2- 172.21.15.143:0/697617542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c530f3360 0x561c5310aa70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 >> 172.21.15.143:0/697617542 conn(0x561c530efa30 msgr2=0x561c530ee350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd3cff9640 0 client.26417 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 shutdown_connections 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 1 -- 172.21.15.143:0/697617542 wait complete. 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5dba9740 20 client.26417 trim_cache size 0 max 16384 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:52:43.401 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:43.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.395+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efd400a2b80 msgr2=0x7efd400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efd400a2b80 0x7efd400a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd4c09bf00 tx=0x7efd4c005f70 comp rx=0 tx=0).stop 2024-09-10T10:52:43.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102ef0 msgr2=0x7efd5419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102ef0 0x7efd5419cce0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009fd0 tx=0x7efd4c00ef10 comp rx=0 tx=0).stop 2024-09-10T10:52:43.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 shutdown_connections 2024-09-10T10:52:43.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7efd400a2b80 0x7efd400a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd54102ef0 0x7efd5419cce0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd54102570 0x7efd54198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 --2- 172.21.15.143:0/804398330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd54100e20 0x7efd54198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 >> 172.21.15.143:0/804398330 conn(0x7efd540fc7d0 msgr2=0x7efd540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 shutdown_connections 2024-09-10T10:52:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:43.396+0000 7efd5b7d0640 1 -- 172.21.15.143:0/804398330 wait complete. 2024-09-10T10:52:44.002 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:43 smithi143 ceph-mon[31057]: pgmap v581: 129 pgs: 129 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 435 KiB/s wr, 78 op/s 2024-09-10T10:52:44.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:43 smithi178 ceph-mon[34813]: pgmap v581: 129 pgs: 129 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 435 KiB/s wr, 78 op/s 2024-09-10T10:52:44.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:43 smithi144 ceph-mon[34230]: pgmap v581: 129 pgs: 129 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 435 KiB/s wr, 78 op/s 2024-09-10T10:52:44.352 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:52:44.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:52:44.624 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:45.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 -- 172.21.15.143:0/119833798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f201082a0 msgr2=0x7f3f2010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 --2- 172.21.15.143:0/119833798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f201082a0 0x7f3f2010ec40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009930 tx=0x7f3f1802a420 comp rx=0 tx=0).stop 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 -- 172.21.15.143:0/119833798 shutdown_connections 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 --2- 172.21.15.143:0/119833798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f201082a0 0x7f3f2010ec40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 --2- 172.21.15.143:0/119833798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f201036e0 0x7f3f20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 --2- 172.21.15.143:0/119833798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f20102d30 0x7f3f20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 -- 172.21.15.143:0/119833798 >> 172.21.15.143:0/119833798 conn(0x7f3f200fc7d0 msgr2=0x7f3f200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 -- 172.21.15.143:0/119833798 shutdown_connections 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.144+0000 7f3f26e97640 1 -- 172.21.15.143:0/119833798 wait complete. 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 Processor -- start 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 -- start start 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 0x7f3f2019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f201036e0 0x7f3f2019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f201082a0 0x7f3f2019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20111f50 con 0x7f3f20102d30 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20111dd0 con 0x7f3f201036e0 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f26e97640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201120d0 con 0x7f3f201082a0 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f24c33640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 0x7f3f2019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f24c33640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 0x7f3f2019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40294/0 (socket says 172.21.15.143:40294) 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f24c33640 1 -- 172.21.15.143:0/2081276778 learned_addr learned my addr 172.21.15.143:0/2081276778 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f1ffff640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f201036e0 0x7f3f2019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f25434640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f201082a0 0x7f3f2019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f24c33640 1 -- 172.21.15.143:0/2081276778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f201082a0 msgr2=0x7f3f2019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.145+0000 7f3f24c33640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f201082a0 0x7f3f2019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f24c33640 1 -- 172.21.15.143:0/2081276778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f201036e0 msgr2=0x7f3f2019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f24c33640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f201036e0 0x7f3f2019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f24c33640 1 -- 172.21.15.143:0/2081276778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2019f620 con 0x7f3f20102d30 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f25434640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f201082a0 0x7f3f2019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f24c33640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 0x7f3f2019a670 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800d950 tx=0x7f3f0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08017030 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2019f910 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f08004020 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080047a0 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.146+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201a71f0 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.147+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f20103110 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.148+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3f080041d0 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.149+0000 7f3f1dffb640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3f0c0a2b80 0x7f3f0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.150+0000 7f3f1ffff640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3f0c0a2b80 0x7f3f0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.150+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3f080cef40 con 0x7f3f20102d30 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.150+0000 7f3f1ffff640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3f0c0a2b80 0x7f3f0c0a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3f20102820 tx=0x7f3f10009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:45.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.154+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f0809bae0 con 0x7f3f20102d30 2024-09-10T10:52:45.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.379+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f2010c2c0 con 0x7f3f20102d30 2024-09-10T10:52:45.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.379+0000 7f3f1dffb640 1 -- 172.21.15.143:0/2081276778 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12001 (secure 0 0 0) 0x7f3f0809b3e0 con 0x7f3f20102d30 2024-09-10T10:52:45.381 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:52:45.381 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"btime":"2024-09-10T10:52:11:648994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:52:11.410829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:52:45.381 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 36 2024-09-10T10:52:45.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3f0c0a2b80 msgr2=0x7f3f0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3f0c0a2b80 0x7f3f0c0a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3f20102820 tx=0x7f3f10009500 comp rx=0 tx=0).stop 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 msgr2=0x7f3f2019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 0x7f3f2019a670 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800d950 tx=0x7f3f0800de10 comp rx=0 tx=0).stop 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 shutdown_connections 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3f0c0a2b80 0x7f3f0c0a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f201082a0 0x7f3f2019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f201036e0 0x7f3f2019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 --2- 172.21.15.143:0/2081276778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f20102d30 0x7f3f2019a670 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 >> 172.21.15.143:0/2081276778 conn(0x7f3f200fc7d0 msgr2=0x7f3f200fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 shutdown_connections 2024-09-10T10:52:45.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:45.382+0000 7f3f26e97640 1 -- 172.21.15.143:0/2081276778 wait complete. 2024-09-10T10:52:45.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:45 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2081276778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:45.750 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:45 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2081276778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:45.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:45 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2081276778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:52:46.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:52:46.469 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:46.752 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:46 smithi143 ceph-mon[31057]: pgmap v582: 129 pgs: 129 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 475 KiB/s wr, 83 op/s 2024-09-10T10:52:46.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:46 smithi178 ceph-mon[34813]: pgmap v582: 129 pgs: 129 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 475 KiB/s wr, 83 op/s 2024-09-10T10:52:46.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:46 smithi144 ceph-mon[34230]: pgmap v582: 129 pgs: 129 active+clean; 425 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 475 KiB/s wr, 83 op/s 2024-09-10T10:52:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 -- 172.21.15.143:0/4265943494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 msgr2=0x7f0de81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 --2- 172.21.15.143:0/4265943494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de81031b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009930 tx=0x7f0dd802a420 comp rx=0 tx=0).stop 2024-09-10T10:52:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 -- 172.21.15.143:0/4265943494 shutdown_connections 2024-09-10T10:52:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 --2- 172.21.15.143:0/4265943494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 0x7f0de810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 --2- 172.21.15.143:0/4265943494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 0x7f0de8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 --2- 172.21.15.143:0/4265943494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de81031b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 -- 172.21.15.143:0/4265943494 >> 172.21.15.143:0/4265943494 conn(0x7f0de80fc7d0 msgr2=0x7f0de80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 -- 172.21.15.143:0/4265943494 shutdown_connections 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:46.999+0000 7f0dece88640 1 -- 172.21.15.143:0/4265943494 wait complete. 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 Processor -- start 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 -- start start 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 0x7f0de8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 0x7f0de819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8111f60 con 0x7f0de8102dd0 2024-09-10T10:52:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8111de0 con 0x7f0de8108300 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0dece88640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0de81120e0 con 0x7f0de8103780 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de8198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60078/0 (socket says 172.21.15.143:60078) 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 -- 172.21.15.143:0/4210009716 learned_addr learned my addr 172.21.15.143:0/4210009716 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 0x7f0de8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 0x7f0de819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 -- 172.21.15.143:0/4210009716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 msgr2=0x7f0de8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 0x7f0de8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 -- 172.21.15.143:0/4210009716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 msgr2=0x7f0de819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 0x7f0de819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de659c640 1 -- 172.21.15.143:0/4210009716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de819d4d0 con 0x7f0de8102dd0 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 0x7f0de819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.000+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 0x7f0de8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.001+0000 7f0de659c640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de8198520 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005940 tx=0x7f0dd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.001+0000 7f0dd77fe640 1 -- 172.21.15.143:0/4210009716 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd803f030 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.001+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de819d760 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.001+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de81a50a0 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.001+0000 7f0dd77fe640 1 -- 172.21.15.143:0/4210009716 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dd80318a0 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.001+0000 7f0dd77fe640 1 -- 172.21.15.143:0/4210009716 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8035070 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.003+0000 7f0dd77fe640 1 -- 172.21.15.143:0/4210009716 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0dd8042030 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 Processor -- start 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0dd77fe640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dd00a6f90 0x7f0dd00a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dd00a6f90 0x7f0dd00a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0dd77fe640 1 -- 172.21.15.143:0/4210009716 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0dd80ef730 con 0x7f0de8102dd0 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 -- start start 2024-09-10T10:52:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e271b40 0x559a1e270090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e270a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e27d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559a1e1d62f0 con 0x559a1e2705d0 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a1e17dcf0 con 0x559a1e27b370 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0def261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a1e1d22b0 con 0x559a1e271b40 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dd00a6f90 0x7f0dd00a9450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005fd0 tx=0x7f0de00093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0de5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e270a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e27d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.004+0000 7f0de659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e271b40 0x559a1e270090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e270a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60090/0 (socket says 172.21.15.143:60090) 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 -- 172.21.15.143:0/787623846 learned_addr learned my addr 172.21.15.143:0/787623846 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 -- 172.21.15.143:0/787623846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e271b40 msgr2=0x559a1e270090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e271b40 0x559a1e270090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 -- 172.21.15.143:0/787623846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 msgr2=0x559a1e27d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e27d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 -- 172.21.15.143:0/787623846 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1e1623a0 con 0x559a1e2705d0 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e270a10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011470 tx=0x7f0ddc011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8074245b102f629c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0dd4ff9640 1 -- 172.21.15.143:0/787623846 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc01e070 con 0x559a1e2705d0 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0dd4ff9640 1 -- 172.21.15.143:0/787623846 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ddc004310 con 0x559a1e2705d0 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.005+0000 7f0dd4ff9640 1 -- 172.21.15.143:0/787623846 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00f9a0 con 0x559a1e2705d0 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 -- 172.21.15.143:0/787623846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 msgr2=0x559a1e270a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e270a10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011470 tx=0x7f0ddc011930 comp rx=0 tx=0).stop 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 -- 172.21.15.143:0/787623846 shutdown_connections 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e27d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e270a10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 --2- 172.21.15.143:0/787623846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e271b40 0x559a1e270090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 -- 172.21.15.143:0/787623846 >> 172.21.15.143:0/787623846 conn(0x559a1e239310 msgr2=0x559a1e26cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 -- 172.21.15.143:0/787623846 shutdown_connections 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 -- 172.21.15.143:0/787623846 wait complete. 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 -- start start 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.006+0000 7f0def261740 1 Processor -- start 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e2705d0 0x559a1e27acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 0x559a1e277f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e2784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559a1e27e070 con 0x559a1e271b40 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a1e191060 con 0x559a1e27b370 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559a1e27e1f0 con 0x559a1e2705d0 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0def261740 10 client.0 _ll_register_callbacks cb 0x559a1dee48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 0x559a1e277f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 0x559a1e277f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60094/0 (socket says 172.21.15.143:60094) 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 -- 172.21.15.143:0/2415647102 learned_addr learned my addr 172.21.15.143:0/2415647102 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de659c640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e2705d0 0x559a1e27acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e2784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 -- 172.21.15.143:0/2415647102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e2705d0 msgr2=0x559a1e27acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e2705d0 0x559a1e27acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 -- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 msgr2=0x559a1e2784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e2784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 -- 172.21.15.143:0/2415647102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1df473c0 con 0x559a1e271b40 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 0x559a1e277f70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0096e0 tx=0x7f0ddc0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0c2bc649ec9189 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.007+0000 7f0dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0dcdffb640 1 -- 172.21.15.143:0/2415647102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc01e050 con 0x559a1e271b40 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0dcdffb640 1 -- 172.21.15.143:0/2415647102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0ddc003800 con 0x559a1e271b40 2024-09-10T10:52:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ddc004310 con 0x559a1e271b40 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0dcdffb640 1 -- 172.21.15.143:0/2415647102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ddc033050 con 0x559a1e271b40 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 10 client.17121 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559a1e289b10 con 0x559a1e271b40 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0dcdffb640 1 -- 172.21.15.143:0/2415647102 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ddc004310 con 0x559a1e271b40 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0dcdffb640 1 -- 172.21.15.143:0/2415647102 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0ddc00f6e0 con 0x559a1e271b40 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 10 client.17121 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 10 client.17121 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 20 client.17121 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e28af10 0x559a1e28d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 4 client.17121 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.008+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a1e1e3860 con 0x559a1e28af10 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.009+0000 7f0de659c640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e28af10 0x559a1e28d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.009+0000 7f0de659c640 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e28af10 0x559a1e28d300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96b5709a691508f server_cookie=a1a952306005122 in_seq=0 out_seq=0 2024-09-10T10:52:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.009+0000 7f0dcdffb640 10 client.17121 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.009+0000 7f0dcdffb640 1 -- 172.21.15.143:0/2415647102 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a1e1e3860 con 0x559a1e28af10 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.009+0000 7f0dcdffb640 10 client.17121 handle_command_reply: tid=0 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 client.17121 shutdown 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 msgr2=0x559a1e277f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 0x559a1e277f70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0096e0 tx=0x7f0ddc0056e0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 shutdown_connections 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e28af10 0x559a1e28d300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e27b370 0x559a1e2784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e271b40 0x559a1e277f70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0dcdffb640 0 client.17121 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 --2- 172.21.15.143:0/2415647102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e2705d0 0x559a1e27acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 >> 172.21.15.143:0/2415647102 conn(0x559a1e239310 msgr2=0x559a1e272930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 shutdown_connections 2024-09-10T10:52:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 1 -- 172.21.15.143:0/2415647102 wait complete. 2024-09-10T10:52:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.010+0000 7f0def261740 20 client.17121 trim_cache size 0 max 16384 2024-09-10T10:52:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 Processor -- start 2024-09-10T10:52:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 -- start start 2024-09-10T10:52:47.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e2873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 0x559a1e277830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e27c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc800fa50 con 0x559a1e2705d0 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8011590 con 0x559a1e271b40 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.031+0000 7f0def261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8010120 con 0x559a1e27b370 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e27c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e27c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42954/0 (socket says 172.21.15.143:42954) 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 -- 172.21.15.143:0/3182846645 learned_addr learned my addr 172.21.15.143:0/3182846645 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de659c640 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e2873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 -- 172.21.15.143:0/3182846645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 msgr2=0x559a1e277830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:52:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 0x559a1e277830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 -- 172.21.15.143:0/3182846645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 msgr2=0x559a1e2873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e2873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.032+0000 7f0de6d9d640 1 -- 172.21.15.143:0/3182846645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1e278840 con 0x559a1e27b370 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.033+0000 7f0de6d9d640 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e27c760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0de000fbc0 tx=0x7f0de0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622ba579b64cae90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.033+0000 7f0dcffff640 1 -- 172.21.15.143:0/3182846645 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de001f620 con 0x559a1e27b370 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.033+0000 7f0dcffff640 1 -- 172.21.15.143:0/3182846645 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0de001fc70 con 0x559a1e27b370 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.033+0000 7f0dcffff640 1 -- 172.21.15.143:0/3182846645 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de001ee30 con 0x559a1e27b370 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 -- 172.21.15.143:0/3182846645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 msgr2=0x559a1e27c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e27c760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0de000fbc0 tx=0x7f0de0012000 comp rx=0 tx=0).stop 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 -- 172.21.15.143:0/3182846645 shutdown_connections 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e27c760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 0x559a1e277830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 --2- 172.21.15.143:0/3182846645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e2873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.034+0000 7f0def261740 1 -- 172.21.15.143:0/3182846645 >> 172.21.15.143:0/3182846645 conn(0x559a1e239310 msgr2=0x559a1e276840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.035+0000 7f0def261740 1 -- 172.21.15.143:0/3182846645 shutdown_connections 2024-09-10T10:52:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.035+0000 7f0def261740 1 -- 172.21.15.143:0/3182846645 wait complete. 2024-09-10T10:52:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 -- start start 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 Processor -- start 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e3043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 0x559a1e283500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e283a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8010ec0 con 0x559a1e2705d0 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc80107f0 con 0x559a1e271b40 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc800da30 con 0x559a1e27b370 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.036+0000 7f0def261740 10 client.0 _ll_register_callbacks cb 0x7f0ddc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e3043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e3043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60108/0 (socket says 172.21.15.143:60108) 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 -- 172.21.15.143:0/3490394681 learned_addr learned my addr 172.21.15.143:0/3490394681 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 -- 172.21.15.143:0/3490394681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 msgr2=0x559a1e283a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e283a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 -- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 msgr2=0x559a1e283500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 0x559a1e283500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.037+0000 7f0de659c640 1 -- 172.21.15.143:0/3490394681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a1e2925a0 con 0x559a1e2705d0 2024-09-10T10:52:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0de659c640 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e3043d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80ecf20 tx=0x7f0dd80b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf6a15533dbe02d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0dcdffb640 1 -- 172.21.15.143:0/3490394681 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80ef410 con 0x559a1e2705d0 2024-09-10T10:52:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0dcdffb640 1 -- 172.21.15.143:0/3490394681 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0dd80b82a0 con 0x559a1e2705d0 2024-09-10T10:52:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0de001fc70 con 0x559a1e2705d0 2024-09-10T10:52:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0dcdffb640 1 -- 172.21.15.143:0/3490394681 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0dd80bbe90 con 0x559a1e2705d0 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0def261740 10 client.17127 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.038+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559a1e30f500 con 0x559a1e2705d0 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0dcdffb640 1 -- 172.21.15.143:0/3490394681 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0de001fc70 con 0x559a1e2705d0 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0dcdffb640 1 -- 172.21.15.143:0/3490394681 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0dd80b7980 con 0x559a1e2705d0 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0def261740 10 client.17127 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0def261740 10 client.17127 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0def261740 20 client.17127 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0def261740 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e30fbf0 0x559a1e312080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0def261740 4 client.17127 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.039+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a1e1ddd70 con 0x559a1e30fbf0 2024-09-10T10:52:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.040+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e30fbf0 0x559a1e312080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.041+0000 7f0de5d9b640 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e30fbf0 0x559a1e312080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd0872901714306d server_cookie=e0bf34914f2f1d95 in_seq=0 out_seq=0 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.041+0000 7f0dcdffb640 10 client.17127 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.041+0000 7f0dcdffb640 1 -- 172.21.15.143:0/3490394681 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a1e1ddd70 con 0x559a1e30fbf0 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.041+0000 7f0dcdffb640 10 client.17127 handle_command_reply: tid=0 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.041+0000 7f0def261740 1 client.17127 shutdown 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 msgr2=0x559a1e3043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e3043d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80ecf20 tx=0x7f0dd80b8000 comp rx=0 tx=0).stop 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 shutdown_connections 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x559a1e30fbf0 0x559a1e312080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559a1e27b370 0x559a1e283a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a1e271b40 0x559a1e283500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 --2- 172.21.15.143:0/3490394681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a1e2705d0 0x559a1e3043d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 >> 172.21.15.143:0/3490394681 conn(0x559a1e239310 msgr2=0x559a1e26bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0dcdffb640 0 client.17127 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:52:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 shutdown_connections 2024-09-10T10:52:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 1 -- 172.21.15.143:0/3490394681 wait complete. 2024-09-10T10:52:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.042+0000 7f0def261740 20 client.17127 trim_cache size 0 max 16384 2024-09-10T10:52:47.045 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dd00a6f90 msgr2=0x7f0dd00a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dd00a6f90 0x7f0dd00a9450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005fd0 tx=0x7f0de00093c0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 msgr2=0x7f0de8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de8198520 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005940 tx=0x7f0dd80042f0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 shutdown_connections 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dd00a6f90 0x7f0dd00a9450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0de8108300 0x7f0de819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de8103780 0x7f0de8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 --2- 172.21.15.143:0/4210009716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0de8102dd0 0x7f0de8198520 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.043+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 >> 172.21.15.143:0/4210009716 conn(0x7f0de80fc7d0 msgr2=0x7f0de8105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.044+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 shutdown_connections 2024-09-10T10:52:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:47.044+0000 7f0dece88640 1 -- 172.21.15.143:0/4210009716 wait complete. 2024-09-10T10:52:47.768 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:52:48.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:47 smithi143 ceph-mon[31057]: pgmap v583: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 982 KiB/s wr, 162 op/s 2024-09-10T10:52:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:48.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:47 smithi178 ceph-mon[34813]: pgmap v583: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 982 KiB/s wr, 162 op/s 2024-09-10T10:52:48.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:47 smithi144 ceph-mon[34230]: pgmap v583: 129 pgs: 129 active+clean; 426 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 982 KiB/s wr, 162 op/s 2024-09-10T10:52:48.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 -- 172.21.15.143:0/1572874102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c10c170 msgr2=0x7fc11c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 --2- 172.21.15.143:0/1572874102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c10c170 0x7fc11c10e610 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc114009ca0 tx=0x7fc11402a950 comp rx=0 tx=0).stop 2024-09-10T10:52:48.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 -- 172.21.15.143:0/1572874102 shutdown_connections 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 --2- 172.21.15.143:0/1572874102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c10c170 0x7fc11c10e610 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 --2- 172.21.15.143:0/1572874102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c103660 0x7fc11c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 --2- 172.21.15.143:0/1572874102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c102d40 0x7fc11c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 -- 172.21.15.143:0/1572874102 >> 172.21.15.143:0/1572874102 conn(0x7fc11c0fc7a0 msgr2=0x7fc11c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 -- 172.21.15.143:0/1572874102 shutdown_connections 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.623+0000 7fc12174c640 1 -- 172.21.15.143:0/1572874102 wait complete. 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 Processor -- start 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 -- start start 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c102d40 0x7fc11c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c103660 0x7fc11c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c10c170 0x7fc11c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c111de0 con 0x7fc11c102d40 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c111c60 con 0x7fc11c103660 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc12174c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c111f60 con 0x7fc11c10c170 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc11affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c102d40 0x7fc11c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc11affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c102d40 0x7fc11c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60122/0 (socket says 172.21.15.143:60122) 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc11affd640 1 -- 172.21.15.143:0/649130651 learned_addr learned my addr 172.21.15.143:0/649130651 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc11b7fe640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c10c170 0x7fc11c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.624+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c103660 0x7fc11c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc11b7fe640 1 -- 172.21.15.143:0/649130651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c103660 msgr2=0x7fc11c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc11b7fe640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c103660 0x7fc11c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc11b7fe640 1 -- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c102d40 msgr2=0x7fc11c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc11b7fe640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c102d40 0x7fc11c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc11b7fe640 1 -- 172.21.15.143:0/649130651 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c1a67c0 con 0x7fc11c10c170 2024-09-10T10:52:48.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc11b7fe640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c10c170 0x7fc11c101420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc114005cc0 tx=0x7fc114031f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc103fff640 1 -- 172.21.15.143:0/649130651 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114034040 con 0x7fc11c10c170 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc103fff640 1 -- 172.21.15.143:0/649130651 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1140308f0 con 0x7fc11c10c170 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c1a6a50 con 0x7fc11c10c170 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc103fff640 1 -- 172.21.15.143:0/649130651 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140044d0 con 0x7fc11c10c170 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.625+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c1a6f30 con 0x7fc11c10c170 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc103fff640 1 -- 172.21.15.143:0/649130651 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc114037020 con 0x7fc11c10c170 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 Processor -- start 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 -- start start 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55925298b2f0 con 0x559252a255d0 2024-09-10T10:52:48.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559252932cf0 con 0x559252a26b40 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc123b25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5592529872b0 con 0x559252a30370 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc11affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a32760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:42992/0 (socket says 172.21.15.143:42992) 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc11a7fc640 1 -- 172.21.15.143:0/1950918601 learned_addr learned my addr 172.21.15.143:0/1950918601 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc11affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a25a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60126/0 (socket says 172.21.15.143:60126) 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.628+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc103fff640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1040a2b80 0x7fc1040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11a7fc640 1 -- 172.21.15.143:0/1950918601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 msgr2=0x559252a25090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a25090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11a7fc640 1 -- 172.21.15.143:0/1950918601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 msgr2=0x559252a25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11a7fc640 1 -- 172.21.15.143:0/1950918601 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592529173a0 con 0x559252a30370 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11b7fe640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1040a2b80 0x7fc1040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a32760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc1100126f0 tx=0x7fc110009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c808de235c3f7cca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc103fff640 1 -- 172.21.15.143:0/649130651 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1140f00c0 con 0x7fc11c10c170 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc1017fa640 1 -- 172.21.15.143:0/1950918601 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110012c70 con 0x559252a30370 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc1017fa640 1 -- 172.21.15.143:0/1950918601 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc110012e10 con 0x559252a30370 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.629+0000 7fc1017fa640 1 -- 172.21.15.143:0/1950918601 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100035f0 con 0x559252a30370 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 -- 172.21.15.143:0/1950918601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 msgr2=0x559252a32760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc11b7fe640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1040a2b80 0x7fc1040a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc1140c4d00 tx=0x7fc1140b1aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a32760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc1100126f0 tx=0x7fc110009700 comp rx=0 tx=0).stop 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 -- 172.21.15.143:0/1950918601 shutdown_connections 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a32760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 --2- 172.21.15.143:0/1950918601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a25090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 -- 172.21.15.143:0/1950918601 >> 172.21.15.143:0/1950918601 conn(0x5592529ee310 msgr2=0x559252a21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 -- 172.21.15.143:0/1950918601 shutdown_connections 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 -- 172.21.15.143:0/1950918601 wait complete. 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 -- start start 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.630+0000 7fc123b25740 1 Processor -- start 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a2d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559252a33070 con 0x559252a255d0 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559252946060 con 0x559252a26b40 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559252a331f0 con 0x559252a30370 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc123b25740 10 client.0 _ll_register_callbacks cb 0x5592526998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a2d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a2d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:52250/0 (socket says 172.21.15.143:52250) 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 -- 172.21.15.143:0/3458188329 learned_addr learned my addr 172.21.15.143:0/3458188329 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 -- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 msgr2=0x559252a2d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a2d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 -- 172.21.15.143:0/3458188329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 msgr2=0x559252a2fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a2fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11affd640 1 -- 172.21.15.143:0/3458188329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592526fc3c0 con 0x559252a26b40 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.631+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc11affd640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a2d060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc10c012860 tx=0x7fc10c012d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dbc662180b90a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc0feff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc0feff5640 1 -- 172.21.15.143:0/3458188329 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00f920 con 0x559252a26b40 2024-09-10T10:52:48.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc110012e10 con 0x559252a26b40 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc0feff5640 1 -- 172.21.15.143:0/3458188329 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc10c00fac0 con 0x559252a26b40 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc0feff5640 1 -- 172.21.15.143:0/3458188329 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc10c00fcc0 con 0x559252a26b40 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 10 client.26444 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559252a3ec40 con 0x559252a26b40 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc0feff5640 1 -- 172.21.15.143:0/3458188329 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc110012e10 con 0x559252a26b40 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc0feff5640 1 -- 172.21.15.143:0/3458188329 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc10c004a40 con 0x559252a26b40 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 10 client.26444 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 10 client.26444 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 20 client.26444 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252a400a0 0x559252a42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 4 client.26444 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:48.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559252998860 con 0x559252a400a0 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.632+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252a400a0 0x559252a42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.633+0000 7fc0feff5640 10 client.26444 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.633+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252a400a0 0x559252a42520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5525cc34f4a4651 server_cookie=b6d302cfd37208d9 in_seq=0 out_seq=0 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc0feff5640 1 -- 172.21.15.143:0/3458188329 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559252998860 con 0x559252a400a0 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc0feff5640 10 client.26444 handle_command_reply: tid=0 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 client.26444 shutdown 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 msgr2=0x559252a2d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a2d060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc10c012860 tx=0x7fc10c012d20 comp rx=0 tx=0).stop 2024-09-10T10:52:48.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 shutdown_connections 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252a400a0 0x559252a42520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a30370 0x559252a2d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc0feff5640 0 client.26444 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a26b40 0x559252a2d060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 --2- 172.21.15.143:0/3458188329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 >> 172.21.15.143:0/3458188329 conn(0x5592529ee310 msgr2=0x559252a27930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 shutdown_connections 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 1 -- 172.21.15.143:0/3458188329 wait complete. 2024-09-10T10:52:48.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.634+0000 7fc123b25740 20 client.26444 trim_cache size 0 max 16384 2024-09-10T10:52:48.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.648+0000 7fc123b25740 1 Processor -- start 2024-09-10T10:52:48.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.648+0000 7fc123b25740 1 -- start start 2024-09-10T10:52:48.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.648+0000 7fc123b25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a255d0 0x559252a42490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.648+0000 7fc123b25740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 0x559252a47df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc123b25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252a48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc123b25740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1b0c80 con 0x559252a26b40 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc123b25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1b27c0 con 0x559252a30370 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc123b25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1b1350 con 0x559252a255d0 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 0x559252a47df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 0x559252a47df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60136/0 (socket says 172.21.15.143:60136) 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a255d0 0x559252a42490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11affd640 1 -- 172.21.15.143:0/2648899663 learned_addr learned my addr 172.21.15.143:0/2648899663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252a48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2648899663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 msgr2=0x559252a48330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252a48330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2648899663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 msgr2=0x559252a47df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 0x559252a47df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2648899663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559252a2db60 con 0x559252a255d0 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11affd640 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 0x559252a47df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a255d0 0x559252a42490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1100126c0 tx=0x7fc110009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c3aa63a0c395fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc100ff9640 1 -- 172.21.15.143:0/2648899663 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110010e50 con 0x559252a255d0 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc100ff9640 1 -- 172.21.15.143:0/2648899663 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc110031020 con 0x559252a255d0 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.649+0000 7fc100ff9640 1 -- 172.21.15.143:0/2648899663 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110003e00 con 0x559252a255d0 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 -- 172.21.15.143:0/2648899663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a255d0 msgr2=0x559252a42490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a255d0 0x559252a42490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1100126c0 tx=0x7fc110009ab0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 -- 172.21.15.143:0/2648899663 shutdown_connections 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252a48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a26b40 0x559252a47df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 --2- 172.21.15.143:0/2648899663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a255d0 0x559252a42490 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 -- 172.21.15.143:0/2648899663 >> 172.21.15.143:0/2648899663 conn(0x5592529ee310 msgr2=0x559252a1ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 -- 172.21.15.143:0/2648899663 shutdown_connections 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 -- 172.21.15.143:0/2648899663 wait complete. 2024-09-10T10:52:48.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.650+0000 7fc123b25740 1 -- start start 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 Processor -- start 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a26b40 0x559252a33a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252aa8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1b20f0 con 0x559252a255d0 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1b1a20 con 0x559252a30370 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1aec60 con 0x559252a26b40 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc123b25740 10 client.0 _ll_register_callbacks cb 0x7fc10c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a26b40 0x559252a33a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60140/0 (socket says 172.21.15.143:60140) 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2745745450 learned_addr learned my addr 172.21.15.143:0/2745745450 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a26b40 0x559252a33a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43020/0 (socket says 172.21.15.143:43020) 2024-09-10T10:52:48.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a26b40 msgr2=0x559252a33a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a26b40 0x559252a33a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2745745450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 msgr2=0x559252aa8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:48.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252aa8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 -- 172.21.15.143:0/2745745450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559252a34330 con 0x559252a255d0 2024-09-10T10:52:48.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc11a7fc640 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a334f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc110009bf0 tx=0x7fc110010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960b6d5438423d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.651+0000 7fc0feff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc0feff5640 1 -- 172.21.15.143:0/2745745450 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100053b0 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc0feff5640 1 -- 172.21.15.143:0/2745745450 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc11002de80 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc110031020 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc0feff5640 1 -- 172.21.15.143:0/2745745450 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc110043050 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 10 client.17145 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559252aaa3f0 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc0feff5640 1 -- 172.21.15.143:0/2745745450 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc110031020 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc0feff5640 1 -- 172.21.15.143:0/2745745450 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc11002d6d0 con 0x559252a255d0 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 10 client.17145 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 10 client.17145 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 20 client.17145 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:48.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252aaa8e0 0x559252aacd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 4 client.17145 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559252992d70 con 0x559252aaa8e0 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.652+0000 7fc11affd640 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252aaa8e0 0x559252aacd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc0feff5640 10 client.17145 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc11affd640 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252aaa8e0 0x559252aacd00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4945e25af1095110 server_cookie=8e74705f432b7620 in_seq=0 out_seq=0 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc0feff5640 1 -- 172.21.15.143:0/2745745450 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559252992d70 con 0x559252aaa8e0 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc0feff5640 10 client.17145 handle_command_reply: tid=0 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 client.17145 shutdown 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 msgr2=0x559252a334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a334f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc110009bf0 tx=0x7fc110010e50 comp rx=0 tx=0).stop 2024-09-10T10:52:48.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 shutdown_connections 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x559252aaa8e0 0x559252aacd00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559252a30370 0x559252aa8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559252a26b40 0x559252a33a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc0feff5640 0 client.17145 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 --2- 172.21.15.143:0/2745745450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559252a255d0 0x559252a334f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 >> 172.21.15.143:0/2745745450 conn(0x5592529ee310 msgr2=0x559252a2b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 shutdown_connections 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 1 -- 172.21.15.143:0/2745745450 wait complete. 2024-09-10T10:52:48.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.653+0000 7fc123b25740 20 client.17145 trim_cache size 0 max 16384 2024-09-10T10:52:48.668 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:52:48.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1040a2b80 msgr2=0x7fc1040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1040a2b80 0x7fc1040a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc1140c4d00 tx=0x7fc1140b1aa0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c10c170 msgr2=0x7fc11c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:48.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c10c170 0x7fc11c101420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc114005cc0 tx=0x7fc114031f40 comp rx=0 tx=0).stop 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 shutdown_connections 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1040a2b80 0x7fc1040a5040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc11c10c170 0x7fc11c101420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc11c103660 0x7fc11c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 --2- 172.21.15.143:0/649130651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c102d40 0x7fc11c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 >> 172.21.15.143:0/649130651 conn(0x7fc11c0fc7a0 msgr2=0x7fc11c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 shutdown_connections 2024-09-10T10:52:48.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:48.654+0000 7fc12174c640 1 -- 172.21.15.143:0/649130651 wait complete. 2024-09-10T10:52:49.337 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:52:49.599 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:49.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:49 smithi178 ceph-mon[34813]: pgmap v584: 129 pgs: 129 active+clean; 418 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 986 KiB/s wr, 211 op/s 2024-09-10T10:52:49.987 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:49 smithi144 ceph-mon[34230]: pgmap v584: 129 pgs: 129 active+clean; 418 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 986 KiB/s wr, 211 op/s 2024-09-10T10:52:50.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:49 smithi143 ceph-mon[31057]: pgmap v584: 129 pgs: 129 active+clean; 418 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 986 KiB/s wr, 211 op/s 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 -- 172.21.15.143:0/796851333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 msgr2=0x7f09f410ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 --2- 172.21.15.143:0/796851333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 0x7f09f410ce00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009900 tx=0x7f09e002a410 comp rx=0 tx=0).stop 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 -- 172.21.15.143:0/796851333 shutdown_connections 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 --2- 172.21.15.143:0/796851333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 0x7f09f41150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 --2- 172.21.15.143:0/796851333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 0x7f09f410ce00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 --2- 172.21.15.143:0/796851333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 0x7f09f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 -- 172.21.15.143:0/796851333 >> 172.21.15.143:0/796851333 conn(0x7f09f4077ba0 msgr2=0x7f09f4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 -- 172.21.15.143:0/796851333 shutdown_connections 2024-09-10T10:52:50.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.202+0000 7f09fadb1640 1 -- 172.21.15.143:0/796851333 wait complete. 2024-09-10T10:52:50.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.203+0000 7f09fadb1640 1 Processor -- start 2024-09-10T10:52:50.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.203+0000 7f09fadb1640 1 -- start start 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09fadb1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 0x7f09f4073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09fadb1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 0x7f09f406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09fadb1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 0x7f09f406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09fadb1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09f406c510 con 0x7f09f4108450 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09fadb1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09f406c390 con 0x7f09f4108e00 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09fadb1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09f406c690 con 0x7f09f410d530 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09f8b4d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 0x7f09f4073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09f8b4d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 0x7f09f4073ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60152/0 (socket says 172.21.15.143:60152) 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09f8b4d640 1 -- 172.21.15.143:0/3558280542 learned_addr learned my addr 172.21.15.143:0/3558280542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09f934e640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 0x7f09f406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.204+0000 7f09f3fff640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 0x7f09f406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f934e640 1 -- 172.21.15.143:0/3558280542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 msgr2=0x7f09f406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f934e640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 0x7f09f406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f934e640 1 -- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 msgr2=0x7f09f4073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f934e640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 0x7f09f4073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f934e640 1 -- 172.21.15.143:0/3558280542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f406e8e0 con 0x7f09f410d530 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f934e640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 0x7f09f406e160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d9c0 tx=0x7f09ec00de80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f1ffb640 1 -- 172.21.15.143:0/3558280542 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00bd10 con 0x7f09f410d530 2024-09-10T10:52:50.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.205+0000 7f09f1ffb640 1 -- 172.21.15.143:0/3558280542 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09ec003e20 con 0x7f09f410d530 2024-09-10T10:52:50.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.206+0000 7f09f1ffb640 1 -- 172.21.15.143:0/3558280542 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0106c0 con 0x7f09f410d530 2024-09-10T10:52:50.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.206+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f41b3880 con 0x7f09f410d530 2024-09-10T10:52:50.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.206+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f41b3c80 con 0x7f09f410d530 2024-09-10T10:52:50.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.209+0000 7f09fd18a740 1 Processor -- start 2024-09-10T10:52:50.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.210+0000 7f09f1ffb640 1 -- 172.21.15.143:0/3558280542 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f09ec003970 con 0x7f09f410d530 2024-09-10T10:52:50.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.210+0000 7f09fd18a740 1 -- start start 2024-09-10T10:52:50.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 0x55632379d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55632379d5d0 0x55632379da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a8370 0x5563237aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09fd18a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5563237032f0 con 0x5563237a8370 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09fd18a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5563236aacf0 con 0x55632379eb40 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09fd18a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5563236ff2b0 con 0x55632379d5d0 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 0x55632379d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 0x55632379d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:52272/0 (socket says 172.21.15.143:52272) 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 -- 172.21.15.143:0/3122116145 learned_addr learned my addr 172.21.15.143:0/3122116145 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:50.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a8370 0x5563237aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 -- 172.21.15.143:0/3122116145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55632379d5d0 msgr2=0x55632379da10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55632379d5d0 0x55632379da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 -- 172.21.15.143:0/3122116145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a8370 msgr2=0x5563237aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a8370 0x5563237aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.211+0000 7f09f3fff640 1 -- 172.21.15.143:0/3122116145 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632368f3a0 con 0x55632379eb40 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.212+0000 7f09f1ffb640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f09e40a2d90 0x7f09e40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.212+0000 7f09f3fff640 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 0x55632379d090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09e0005ab0 tx=0x7f09e0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8d4cb44f1a51ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.212+0000 7f09f1ffb640 1 -- 172.21.15.143:0/3558280542 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f09ec0cf5c0 con 0x7f09f410d530 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.212+0000 7f09deffd640 1 -- 172.21.15.143:0/3122116145 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0035040 con 0x55632379eb40 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.212+0000 7f09deffd640 1 -- 172.21.15.143:0/3122116145 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09e00046c0 con 0x55632379eb40 2024-09-10T10:52:50.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.212+0000 7f09deffd640 1 -- 172.21.15.143:0/3122116145 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0031700 con 0x55632379eb40 2024-09-10T10:52:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.213+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f09e40a2d90 0x7f09e40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 -- 172.21.15.143:0/3122116145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 msgr2=0x55632379d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 0x55632379d090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09e0005ab0 tx=0x7f09e0005d50 comp rx=0 tx=0).stop 2024-09-10T10:52:50.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f09e40a2d90 0x7f09e40a5250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55632379aac0 tx=0x7f09e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 -- 172.21.15.143:0/3122116145 shutdown_connections 2024-09-10T10:52:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a8370 0x5563237aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55632379d5d0 0x55632379da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3122116145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379eb40 0x55632379d090 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09e0005ab0 tx=0x7f09e0005d50 comp rx=0 tx=0).stop 2024-09-10T10:52:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 -- 172.21.15.143:0/3122116145 >> 172.21.15.143:0/3122116145 conn(0x556323766310 msgr2=0x556323799c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 -- 172.21.15.143:0/3122116145 shutdown_connections 2024-09-10T10:52:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 -- 172.21.15.143:0/3122116145 wait complete. 2024-09-10T10:52:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.214+0000 7f09fd18a740 1 -- start start 2024-09-10T10:52:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 Processor -- start 2024-09-10T10:52:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5563237ab060 con 0x5563237a5d90 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5563236be060 con 0x55632379d5d0 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5563237ab1e0 con 0x5563237a8370 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.215+0000 7f09fd18a740 10 client.0 _ll_register_callbacks cb 0x5563234118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.216+0000 7f09f8b4d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.216+0000 7f09f934e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.216+0000 7f09f934e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60176/0 (socket says 172.21.15.143:60176) 2024-09-10T10:52:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.216+0000 7f09f934e640 1 -- 172.21.15.143:0/1050685597 learned_addr learned my addr 172.21.15.143:0/1050685597 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.216+0000 7f09f3fff640 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09f934e640 1 -- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 msgr2=0x5563237a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09f934e640 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09f934e640 1 -- 172.21.15.143:0/1050685597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 msgr2=0x5563237a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09f934e640 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09f934e640 1 -- 172.21.15.143:0/1050685597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563234743c0 con 0x5563237a5d90 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09f934e640 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237b4510 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0a41a0 tx=0x7f09ec09b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7496ed096eee2115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09d3fff640 1 -- 172.21.15.143:0/1050685597 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec004250 con 0x5563237a5d90 2024-09-10T10:52:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e00046c0 con 0x5563237a5d90 2024-09-10T10:52:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.217+0000 7f09d3fff640 1 -- 172.21.15.143:0/1050685597 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f09ec0cce10 con 0x5563237a5d90 2024-09-10T10:52:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.218+0000 7f09d3fff640 1 -- 172.21.15.143:0/1050685597 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f09ec09c160 con 0x5563237a5d90 2024-09-10T10:52:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.218+0000 7f09fd18a740 10 client.17157 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.218+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5563237b7220 con 0x5563237a5d90 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09d3fff640 1 -- 172.21.15.143:0/1050685597 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09e00046c0 con 0x5563237a5d90 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09d3fff640 1 -- 172.21.15.143:0/1050685597 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f09ec024060 con 0x5563237a5d90 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09fd18a740 10 client.17157 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09fd18a740 10 client.17157 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09fd18a740 20 client.17157 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563237b8b10 0x5563237baf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09fd18a740 4 client.17157 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556323710860 con 0x5563237b8b10 2024-09-10T10:52:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.219+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563237b8b10 0x5563237baf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.221+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563237b8b10 0x5563237baf60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=378c6c7cea1c0f69 server_cookie=4367b4778d549cc8 in_seq=0 out_seq=0 2024-09-10T10:52:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.221+0000 7f09d3fff640 10 client.17157 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:52:50.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.223+0000 7f09d3fff640 1 -- 172.21.15.143:0/1050685597 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556323710860 con 0x5563237b8b10 2024-09-10T10:52:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.223+0000 7f09d3fff640 10 client.17157 handle_command_reply: tid=0 2024-09-10T10:52:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 client.17157 shutdown 2024-09-10T10:52:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 msgr2=0x5563237b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237b4510 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0a41a0 tx=0x7f09ec09b660 comp rx=0 tx=0).stop 2024-09-10T10:52:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 shutdown_connections 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563237b8b10 0x5563237baf60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237b4510 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1050685597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 >> 172.21.15.143:0/1050685597 conn(0x556323766310 msgr2=0x55632379f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.224+0000 7f09d3fff640 0 client.17157 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:52:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.225+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 shutdown_connections 2024-09-10T10:52:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.225+0000 7f09fd18a740 1 -- 172.21.15.143:0/1050685597 wait complete. 2024-09-10T10:52:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.226+0000 7f09fd18a740 20 client.17157 trim_cache size 0 max 16384 2024-09-10T10:52:50.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.249+0000 7f09fd18a740 1 Processor -- start 2024-09-10T10:52:50.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.249+0000 7f09fd18a740 1 -- start start 2024-09-10T10:52:50.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09fd18a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41bd9c0 con 0x5563237a5d90 2024-09-10T10:52:50.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09fd18a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41bf500 con 0x55632379d5d0 2024-09-10T10:52:50.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09fd18a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41be090 con 0x5563237a8370 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09f934e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09f934e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43068/0 (socket says 172.21.15.143:43068) 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09f934e640 1 -- 172.21.15.143:0/3201495721 learned_addr learned my addr 172.21.15.143:0/3201495721 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.250+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60190/0 (socket says 172.21.15.143:60190) 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f934e640 1 -- 172.21.15.143:0/3201495721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 msgr2=0x5563237a7280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f934e640 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a7280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f934e640 1 -- 172.21.15.143:0/3201495721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 msgr2=0x5563237c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f934e640 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237c0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f934e640 1 -- 172.21.15.143:0/3201495721 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563237a42c0 con 0x5563237a8370 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f3fff640 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.251+0000 7f09f934e640 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237c0db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09ec09aeb0 tx=0x7f09ec099f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9394ab903cc6ec32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.252+0000 7f09de7fc640 1 -- 172.21.15.143:0/3201495721 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec09b830 con 0x5563237a8370 2024-09-10T10:52:50.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.253+0000 7f09de7fc640 1 -- 172.21.15.143:0/3201495721 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09ec0b4070 con 0x5563237a8370 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.253+0000 7f09de7fc640 1 -- 172.21.15.143:0/3201495721 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec097d00 con 0x5563237a8370 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.253+0000 7f09fd18a740 1 -- 172.21.15.143:0/3201495721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 msgr2=0x5563237c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.253+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237c0db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09ec09aeb0 tx=0x7f09ec099f70 comp rx=0 tx=0).stop 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 -- 172.21.15.143:0/3201495721 shutdown_connections 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237c0db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5563237a5d90 0x5563237c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 --2- 172.21.15.143:0/3201495721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55632379d5d0 0x5563237a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 -- 172.21.15.143:0/3201495721 >> 172.21.15.143:0/3201495721 conn(0x556323766310 msgr2=0x5563237a6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 -- 172.21.15.143:0/3201495721 shutdown_connections 2024-09-10T10:52:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.254+0000 7f09fd18a740 1 -- 172.21.15.143:0/3201495721 wait complete. 2024-09-10T10:52:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.255+0000 7f09fd18a740 1 -- start start 2024-09-10T10:52:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.255+0000 7f09fd18a740 1 Processor -- start 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.255+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 0x5563237b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.255+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563237a5d90 0x5563237a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.255+0000 7f09fd18a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.255+0000 7f09fd18a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41bee30 con 0x55632379d5d0 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09fd18a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41be760 con 0x5563237a5d90 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09fd18a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09f41bb9a0 con 0x5563237a8370 2024-09-10T10:52:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09fd18a740 10 client.0 _ll_register_callbacks cb 0x7f09ec005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09f8b4d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 0x5563237b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09f8b4d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 0x5563237b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60200/0 (socket says 172.21.15.143:60200) 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09f8b4d640 1 -- 172.21.15.143:0/1151864057 learned_addr learned my addr 172.21.15.143:0/1151864057 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09f934e640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.256+0000 7f09f3fff640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563237a5d90 0x5563237a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.257+0000 7f09f8b4d640 1 -- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 msgr2=0x5563237a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.257+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.257+0000 7f09f8b4d640 1 -- 172.21.15.143:0/1151864057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563237a5d90 msgr2=0x5563237a91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.257+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563237a5d90 0x5563237a91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.257+0000 7f09f8b4d640 1 -- 172.21.15.143:0/1151864057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563237ae760 con 0x55632379d5d0 2024-09-10T10:52:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.257+0000 7f09f3fff640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563237a5d90 0x5563237a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.258+0000 7f09f8b4d640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 0x5563237b0c80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x5563237bb7e0 tx=0x7f09e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538fd1763fc38c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.258+0000 7f09d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.259+0000 7f09d3fff640 1 -- 172.21.15.143:0/1151864057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8003b40 con 0x55632379d5d0 2024-09-10T10:52:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.259+0000 7f09d3fff640 1 -- 172.21.15.143:0/1151864057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f09e8003ce0 con 0x55632379d5d0 2024-09-10T10:52:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.259+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09ec0b4070 con 0x55632379d5d0 2024-09-10T10:52:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.259+0000 7f09d3fff640 1 -- 172.21.15.143:0/1151864057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f09e8005020 con 0x55632379d5d0 2024-09-10T10:52:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.259+0000 7f09fd18a740 10 client.17163 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.259+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556323823930 con 0x55632379d5d0 2024-09-10T10:52:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09d3fff640 1 -- 172.21.15.143:0/1151864057 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09ec0b4070 con 0x55632379d5d0 2024-09-10T10:52:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09d3fff640 1 -- 172.21.15.143:0/1151864057 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f09e801a040 con 0x55632379d5d0 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09fd18a740 10 client.17163 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09fd18a740 10 client.17163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09fd18a740 20 client.17163 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563238240b0 0x5563238264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09fd18a740 4 client.17163 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.260+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55632370ad70 con 0x5563238240b0 2024-09-10T10:52:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.261+0000 7f09f3fff640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563238240b0 0x5563238264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.263+0000 7f09f3fff640 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563238240b0 0x5563238264d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67f048c800b514ac server_cookie=d5f29d9bfecbcf0a in_seq=0 out_seq=0 2024-09-10T10:52:50.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.263+0000 7f09d3fff640 10 client.17163 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:52:50.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.263+0000 7f09d3fff640 1 -- 172.21.15.143:0/1151864057 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55632370ad70 con 0x5563238240b0 2024-09-10T10:52:50.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.263+0000 7f09d3fff640 10 client.17163 handle_command_reply: tid=0 2024-09-10T10:52:50.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.263+0000 7f09fd18a740 1 client.17163 shutdown 2024-09-10T10:52:50.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 msgr2=0x5563237b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 0x5563237b0c80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x5563237bb7e0 tx=0x7f09e8004730 comp rx=0 tx=0).stop 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 shutdown_connections 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5563238240b0 0x5563238264d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563237a8370 0x5563237a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563237a5d90 0x5563237a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 --2- 172.21.15.143:0/1151864057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55632379d5d0 0x5563237b0c80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 >> 172.21.15.143:0/1151864057 conn(0x556323766310 msgr2=0x5563237a7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09d3fff640 0 client.17163 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 shutdown_connections 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 1 -- 172.21.15.143:0/1151864057 wait complete. 2024-09-10T10:52:50.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.264+0000 7f09fd18a740 20 client.17163 trim_cache size 0 max 16384 2024-09-10T10:52:50.267 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.265+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f09e40a2d90 msgr2=0x7f09e40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.265+0000 7f09fadb1640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f09e40a2d90 0x7f09e40a5250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55632379aac0 tx=0x7f09e8005f90 comp rx=0 tx=0).stop 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.265+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 msgr2=0x7f09f406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.265+0000 7f09fadb1640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 0x7f09f406e160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d9c0 tx=0x7f09ec00de80 comp rx=0 tx=0).stop 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 shutdown_connections 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f09e40a2d90 0x7f09e40a5250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09f410d530 0x7f09f406e160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f4108e00 0x7f09f406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 --2- 172.21.15.143:0/3558280542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f4108450 0x7f09f4073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 >> 172.21.15.143:0/3558280542 conn(0x7f09f4077ba0 msgr2=0x7f09f4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.266+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 shutdown_connections 2024-09-10T10:52:50.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:50.267+0000 7f09fadb1640 1 -- 172.21.15.143:0/3558280542 wait complete. 2024-09-10T10:52:50.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:52:51.237 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:51.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 -- 172.21.15.143:0/748586599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102570 msgr2=0x7f55cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 --2- 172.21.15.143:0/748586599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102570 0x7f55cc1029b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b402a480 comp rx=0 tx=0).stop 2024-09-10T10:52:51.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 -- 172.21.15.143:0/748586599 shutdown_connections 2024-09-10T10:52:51.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 --2- 172.21.15.143:0/748586599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102ef0 0x7f55cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 --2- 172.21.15.143:0/748586599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102570 0x7f55cc1029b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 --2- 172.21.15.143:0/748586599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55cc100e20 0x7f55cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 -- 172.21.15.143:0/748586599 >> 172.21.15.143:0/748586599 conn(0x7f55cc0fc7d0 msgr2=0x7f55cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 -- 172.21.15.143:0/748586599 shutdown_connections 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.729+0000 7f55d0c79640 1 -- 172.21.15.143:0/748586599 wait complete. 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.730+0000 7f55d0c79640 1 Processor -- start 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.730+0000 7f55d0c79640 1 -- start start 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55d0c79640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55cc100e20 0x7f55cc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55d0c79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 0x7f55cc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55d0c79640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102ef0 0x7f55cc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55d0c79640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc110e00 con 0x7f55cc102ef0 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55d0c79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc110c80 con 0x7f55cc102570 2024-09-10T10:52:51.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55d0c79640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc110f80 con 0x7f55cc100e20 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 0x7f55cc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 0x7f55cc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:52320/0 (socket says 172.21.15.143:52320) 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 -- 172.21.15.143:0/3805826177 learned_addr learned my addr 172.21.15.143:0/3805826177 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55ca59c640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55cc100e20 0x7f55cc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102ef0 0x7f55cc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 -- 172.21.15.143:0/3805826177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55cc100e20 msgr2=0x7f55cc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55cc100e20 0x7f55cc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 -- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102ef0 msgr2=0x7f55cc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102ef0 0x7f55cc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 -- 172.21.15.143:0/3805826177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc1a6990 con 0x7f55cc102570 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.731+0000 7f55c3fff640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 0x7f55cc0fd1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55b4006010 tx=0x7f55b40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.732+0000 7f55c37fe640 1 -- 172.21.15.143:0/3805826177 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b403f030 con 0x7f55cc102570 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.732+0000 7f55c37fe640 1 -- 172.21.15.143:0/3805826177 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b40044c0 con 0x7f55cc102570 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.732+0000 7f55c37fe640 1 -- 172.21.15.143:0/3805826177 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4035070 con 0x7f55cc102570 2024-09-10T10:52:51.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.732+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc1a6c20 con 0x7f55cc102570 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.732+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a7150 con 0x7f55cc102570 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.734+0000 7f55c37fe640 1 -- 172.21.15.143:0/3805826177 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f55b4042030 con 0x7f55cc102570 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 Processor -- start 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55c37fe640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f55b80a2c00 0x7f55b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f55b80a2c00 0x7f55b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55c37fe640 1 -- 172.21.15.143:0/3805826177 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55b40ef620 con 0x7f55cc102570 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 -- start start 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7fb40 0x557031a7e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a7ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a8b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5570319e42f0 con 0x557031a89370 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55703198bcf0 con 0x557031a7fb40 2024-09-10T10:52:51.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55d3052740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5570319e02b0 con 0x557031a7e5d0 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f55b80a2c00 0x7f55b80a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005fd0 tx=0x7f55c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a7ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55cad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a7ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43102/0 (socket says 172.21.15.143:43102) 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55c3fff640 1 -- 172.21.15.143:0/2942478668 learned_addr learned my addr 172.21.15.143:0/2942478668 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.735+0000 7f55ca59c640 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7fb40 0x557031a7e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c3fff640 1 -- 172.21.15.143:0/2942478668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7fb40 msgr2=0x557031a7e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c3fff640 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7fb40 0x557031a7e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c3fff640 1 -- 172.21.15.143:0/2942478668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 msgr2=0x557031a8b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c3fff640 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a8b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c3fff640 1 -- 172.21.15.143:0/2942478668 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570319703a0 con 0x557031a7e5d0 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55cad9d640 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a8b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:51.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c3fff640 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a7ea10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55b40c4200 tx=0x7f55b40bb590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9820a777af253847 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c0ff9640 1 -- 172.21.15.143:0/2942478668 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4049040 con 0x557031a7e5d0 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c0ff9640 1 -- 172.21.15.143:0/2942478668 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b40c4730 con 0x557031a7e5d0 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.736+0000 7f55c0ff9640 1 -- 172.21.15.143:0/2942478668 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b40c42a0 con 0x557031a7e5d0 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 -- 172.21.15.143:0/2942478668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 msgr2=0x557031a7ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a7ea10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55b40c4200 tx=0x7f55b40bb590 comp rx=0 tx=0).stop 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 -- 172.21.15.143:0/2942478668 shutdown_connections 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a8b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a7ea10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 --2- 172.21.15.143:0/2942478668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7fb40 0x557031a7e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 -- 172.21.15.143:0/2942478668 >> 172.21.15.143:0/2942478668 conn(0x557031a47310 msgr2=0x557031a7ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 -- 172.21.15.143:0/2942478668 shutdown_connections 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 -- 172.21.15.143:0/2942478668 wait complete. 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 -- start start 2024-09-10T10:52:51.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.737+0000 7f55d3052740 1 Processor -- start 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a88d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7fb40 0x557031a86030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a86640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5570319e02b0 con 0x557031a89370 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55703199f060 con 0x557031a7e5d0 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557031a8be30 con 0x557031a7fb40 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55d3052740 10 client.0 _ll_register_callbacks cb 0x5570316f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7fb40 0x557031a86030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a86640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7fb40 0x557031a86030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43104/0 (socket says 172.21.15.143:43104) 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55c3fff640 1 -- 172.21.15.143:0/3547798410 learned_addr learned my addr 172.21.15.143:0/3547798410 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a86640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60236/0 (socket says 172.21.15.143:60236) 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55ca59c640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a88d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 -- 172.21.15.143:0/3547798410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7fb40 msgr2=0x557031a86030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7fb40 0x557031a86030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 -- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 msgr2=0x557031a88d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a88d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 -- 172.21.15.143:0/3547798410 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570317553c0 con 0x557031a89370 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55ca59c640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a88d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55cad9d640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a86640 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f55c4011ca0 tx=0x7f55c4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8f085f9ca4ad72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.738+0000 7f55b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55b1ffb640 1 -- 172.21.15.143:0/3547798410 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c401be10 con 0x557031a89370 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55b1ffb640 1 -- 172.21.15.143:0/3547798410 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55c4004b20 con 0x557031a89370 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b40c4730 con 0x557031a89370 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55b1ffb640 1 -- 172.21.15.143:0/3547798410 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f55c4031050 con 0x557031a89370 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 10 client.17175 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:51.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557031a97810 con 0x557031a89370 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55b1ffb640 1 -- 172.21.15.143:0/3547798410 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b40c4730 con 0x557031a89370 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55b1ffb640 1 -- 172.21.15.143:0/3547798410 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f55c4018040 con 0x557031a89370 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 10 client.17175 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 10 client.17175 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 20 client.17175 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031a98ce0 0x557031a9b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 4 client.17175 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570319f1860 con 0x557031a98ce0 2024-09-10T10:52:51.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.739+0000 7f55ca59c640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031a98ce0 0x557031a9b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.740+0000 7f55ca59c640 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031a98ce0 0x557031a9b0d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=318e1881899e1010 server_cookie=546d95987b3ba4c8 in_seq=0 out_seq=0 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.740+0000 7f55b1ffb640 10 client.17175 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.740+0000 7f55b1ffb640 1 -- 172.21.15.143:0/3547798410 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570319f1860 con 0x557031a98ce0 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.740+0000 7f55b1ffb640 10 client.17175 handle_command_reply: tid=0 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.740+0000 7f55d3052740 1 client.17175 shutdown 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 msgr2=0x557031a86640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a86640 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f55c4011ca0 tx=0x7f55c4013040 comp rx=0 tx=0).stop 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 shutdown_connections 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a89370 0x557031a86640 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7fb40 0x557031a86030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031a98ce0 0x557031a9b0d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 --2- 172.21.15.143:0/3547798410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a88d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55b1ffb640 0 client.17175 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 >> 172.21.15.143:0/3547798410 conn(0x557031a47310 msgr2=0x557031a80960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 shutdown_connections 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 1 -- 172.21.15.143:0/3547798410 wait complete. 2024-09-10T10:52:51.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.741+0000 7f55d3052740 20 client.17175 trim_cache size 0 max 16384 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.754+0000 7f55d3052740 1 Processor -- start 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.754+0000 7f55d3052740 1 -- start start 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55d3052740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a87fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55d3052740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a81120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55d3052740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a89370 0x557031a81660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55d3052740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1b0d70 con 0x557031a7fb40 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55d3052740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1b28b0 con 0x557031a89370 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55d3052740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1b1440 con 0x557031a7e5d0 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a81120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55ca59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a87fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a81120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60244/0 (socket says 172.21.15.143:60244) 2024-09-10T10:52:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 -- 172.21.15.143:0/1417162106 learned_addr learned my addr 172.21.15.143:0/1417162106 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55cad9d640 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a89370 0x557031a81660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 -- 172.21.15.143:0/1417162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 msgr2=0x557031a87fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a87fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 -- 172.21.15.143:0/1417162106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a89370 msgr2=0x557031a81660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a89370 0x557031a81660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 -- 172.21.15.143:0/1417162106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557031a84fe0 con 0x557031a7fb40 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55ca59c640 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a87fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.755+0000 7f55c3fff640 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a81120 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f55b40bb870 tx=0x7f55b40bbcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddcf882550b4f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55b3fff640 1 -- 172.21.15.143:0/1417162106 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b40da070 con 0x557031a7fb40 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55b3fff640 1 -- 172.21.15.143:0/1417162106 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b40ef360 con 0x557031a7fb40 2024-09-10T10:52:51.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55b3fff640 1 -- 172.21.15.143:0/1417162106 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4049040 con 0x557031a7fb40 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 -- 172.21.15.143:0/1417162106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 msgr2=0x557031a81120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a81120 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f55b40bb870 tx=0x7f55b40bbcb0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 -- 172.21.15.143:0/1417162106 shutdown_connections 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a89370 0x557031a81660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a81120 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 --2- 172.21.15.143:0/1417162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a7e5d0 0x557031a87fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 -- 172.21.15.143:0/1417162106 >> 172.21.15.143:0/1417162106 conn(0x557031a47310 msgr2=0x557031a87890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 -- 172.21.15.143:0/1417162106 shutdown_connections 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.756+0000 7f55d3052740 1 -- 172.21.15.143:0/1417162106 wait complete. 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 -- start start 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 Processor -- start 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a91550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a82ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a89370 0x557031a83010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1b21e0 con 0x557031a7fb40 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1b1b10 con 0x557031a7e5d0 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc1aed50 con 0x557031a89370 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55d3052740 10 client.0 _ll_register_callbacks cb 0x7f55c4003a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a82ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55c3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a82ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60260/0 (socket says 172.21.15.143:60260) 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55c3fff640 1 -- 172.21.15.143:0/2995875610 learned_addr learned my addr 172.21.15.143:0/2995875610 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55cad9d640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a89370 0x557031a83010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.757+0000 7f55ca59c640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a91550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55c3fff640 1 -- 172.21.15.143:0/2995875610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a89370 msgr2=0x557031a83010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55c3fff640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a89370 0x557031a83010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55c3fff640 1 -- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 msgr2=0x557031a91550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55c3fff640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a91550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55c3fff640 1 -- 172.21.15.143:0/2995875610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557031a83550 con 0x557031a7fb40 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55c3fff640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a82ad0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f55b40bb780 tx=0x7f55b40bfff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2321c97b71ded5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55b1ffb640 1 -- 172.21.15.143:0/2995875610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4049070 con 0x557031a7fb40 2024-09-10T10:52:51.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55b1ffb640 1 -- 172.21.15.143:0/2995875610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f55b4003d30 con 0x557031a7fb40 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b40ef360 con 0x557031a7fb40 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55b1ffb640 1 -- 172.21.15.143:0/2995875610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f55b40da040 con 0x557031a7fb40 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55d3052740 10 client.17181 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557031b036c0 con 0x557031a7fb40 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.758+0000 7f55b1ffb640 1 -- 172.21.15.143:0/2995875610 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55b40ef360 con 0x557031a7fb40 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55b1ffb640 1 -- 172.21.15.143:0/2995875610 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f55b40da270 con 0x557031a7fb40 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 10 client.17181 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 10 client.17181 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 20 client.17181 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031b039b0 0x557031b05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 4 client.17181 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570319ebd70 con 0x557031b039b0 2024-09-10T10:52:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55ca59c640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031b039b0 0x557031b05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55ca59c640 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031b039b0 0x557031b05da0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbcb03a66b90b392 server_cookie=52a474d68b54e4df in_seq=0 out_seq=0 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55b1ffb640 10 client.17181 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55b1ffb640 1 -- 172.21.15.143:0/2995875610 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570319ebd70 con 0x557031b039b0 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55b1ffb640 10 client.17181 handle_command_reply: tid=0 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 client.17181 shutdown 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 msgr2=0x557031a82ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a82ad0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f55b40bb780 tx=0x7f55b40bfff0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 shutdown_connections 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557031a89370 0x557031a83010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557031a7fb40 0x557031a82ad0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x557031b039b0 0x557031b05da0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 --2- 172.21.15.143:0/2995875610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557031a7e5d0 0x557031a91550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 >> 172.21.15.143:0/2995875610 conn(0x557031a47310 msgr2=0x557031a880b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.759+0000 7f55b1ffb640 0 client.17181 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:52:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 shutdown_connections 2024-09-10T10:52:51.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d3052740 1 -- 172.21.15.143:0/2995875610 wait complete. 2024-09-10T10:52:51.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d3052740 20 client.17181 trim_cache size 0 max 16384 2024-09-10T10:52:51.767 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:52:51.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f55b80a2c00 msgr2=0x7f55b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f55b80a2c00 0x7f55b80a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005fd0 tx=0x7f55c4009500 comp rx=0 tx=0).stop 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 msgr2=0x7f55cc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 0x7f55cc0fd1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55b4006010 tx=0x7f55b40040c0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 shutdown_connections 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f55b80a2c00 0x7f55b80a50c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc102ef0 0x7f55cc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55cc102570 0x7f55cc0fd1c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 --2- 172.21.15.143:0/3805826177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55cc100e20 0x7f55cc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 >> 172.21.15.143:0/3805826177 conn(0x7f55cc0fc7d0 msgr2=0x7f55cc06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 shutdown_connections 2024-09-10T10:52:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:51.760+0000 7f55d0c79640 1 -- 172.21.15.143:0/3805826177 wait complete. 2024-09-10T10:52:52.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:51 smithi143 ceph-mon[31057]: pgmap v585: 129 pgs: 129 active+clean; 411 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 992 KiB/s wr, 285 op/s 2024-09-10T10:52:52.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:51 smithi178 ceph-mon[34813]: pgmap v585: 129 pgs: 129 active+clean; 411 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 992 KiB/s wr, 285 op/s 2024-09-10T10:52:52.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:51 smithi144 ceph-mon[34230]: pgmap v585: 129 pgs: 129 active+clean; 411 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 992 KiB/s wr, 285 op/s 2024-09-10T10:52:53.457 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:52:53.728 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:54.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:53 smithi143 ceph-mon[31057]: pgmap v586: 129 pgs: 129 active+clean; 394 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-09-10T10:52:54.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:53 smithi178 ceph-mon[34813]: pgmap v586: 129 pgs: 129 active+clean; 394 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-09-10T10:52:54.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:53 smithi144 ceph-mon[34230]: pgmap v586: 129 pgs: 129 active+clean; 394 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 -- 172.21.15.143:0/3962391958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 msgr2=0x7fcbb8074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/3962391958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb8074bd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009900 tx=0x7fcbb402a410 comp rx=0 tx=0).stop 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 -- 172.21.15.143:0/3962391958 shutdown_connections 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/3962391958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb810e480 0x7fcbb81108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/3962391958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb8075110 0x7fcbb810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/3962391958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb8074bd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 -- 172.21.15.143:0/3962391958 >> 172.21.15.143:0/3962391958 conn(0x7fcbb80fd8d0 msgr2=0x7fcbb80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 -- 172.21.15.143:0/3962391958 shutdown_connections 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.427+0000 7fcbbf80a640 1 -- 172.21.15.143:0/3962391958 wait complete. 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 Processor -- start 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 -- start start 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb8075110 0x7fcbb819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb810e480 0x7fcbb819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81140c0 con 0x7fcbb80747f0 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb8113f40 con 0x7fcbb810e480 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbf80a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb8114240 con 0x7fcbb8075110 2024-09-10T10:52:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60284/0 (socket says 172.21.15.143:60284) 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/1618169173 learned_addr learned my addr 172.21.15.143:0/1618169173 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb810e480 0x7fcbb819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb8075110 0x7fcbb819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/1618169173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb8075110 msgr2=0x7fcbb819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.428+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb8075110 0x7fcbb819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/1618169173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb810e480 msgr2=0x7fcbb819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb810e480 0x7fcbb819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/1618169173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb819f540 con 0x7fcbb80747f0 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb810e480 0x7fcbb819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb819a590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4005940 tx=0x7fcbb40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbae7fc640 1 -- 172.21.15.143:0/1618169173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb403f030 con 0x7fcbb80747f0 2024-09-10T10:52:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb819f7d0 con 0x7fcbb80747f0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbae7fc640 1 -- 172.21.15.143:0/1618169173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb40044c0 con 0x7fcbb80747f0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbae7fc640 1 -- 172.21.15.143:0/1618169173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4035070 con 0x7fcbb80747f0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.429+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb81a7110 con 0x7fcbb80747f0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.431+0000 7fcbae7fc640 1 -- 172.21.15.143:0/1618169173 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcbb4042030 con 0x7fcbb80747f0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.431+0000 7fcbc1be3740 1 Processor -- start 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 -- start start 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f123c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f123ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1249760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f0f11a22f0 con 0x55f0f1247370 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f0f1149cf0 con 0x55f0f123db40 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f0f119e2b0 con 0x55f0f123c5d0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbae7fc640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcba40a2b80 0x7fcba40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f123c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbcda5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1249760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f123c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:52386/0 (socket says 172.21.15.143:52386) 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 -- 172.21.15.143:0/893718917 learned_addr learned my addr 172.21.15.143:0/893718917 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcba40a2b80 0x7fcba40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbae7fc640 1 -- 172.21.15.143:0/1618169173 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcbb40ef690 con 0x7fcbb80747f0 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f123ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 -- 172.21.15.143:0/893718917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 msgr2=0x55f0f123ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f123ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 -- 172.21.15.143:0/893718917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 msgr2=0x55f0f1249760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1249760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.432+0000 7fcbbdda7640 1 -- 172.21.15.143:0/893718917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f112e3a0 con 0x55f0f123db40 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.433+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f123c090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000f5f0 tx=0x7fcbb000fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd466bb23f86979 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.433+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcba40a2b80 0x7fcba40a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000ffb0 tx=0x7fcbb0018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.433+0000 7fcba37fe640 1 -- 172.21.15.143:0/893718917 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0022070 con 0x55f0f123db40 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.433+0000 7fcba37fe640 1 -- 172.21.15.143:0/893718917 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb0009570 con 0x55f0f123db40 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.433+0000 7fcba37fe640 1 -- 172.21.15.143:0/893718917 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000ed00 con 0x55f0f123db40 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 -- 172.21.15.143:0/893718917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 msgr2=0x55f0f123c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f123c090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000f5f0 tx=0x7fcbb000fab0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 -- 172.21.15.143:0/893718917 shutdown_connections 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f123ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/893718917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f123c090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 -- 172.21.15.143:0/893718917 >> 172.21.15.143:0/893718917 conn(0x55f0f1205310 msgr2=0x55f0f1238c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 -- 172.21.15.143:0/893718917 shutdown_connections 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 -- 172.21.15.143:0/893718917 wait complete. 2024-09-10T10:52:54.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 -- start start 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.434+0000 7fcbc1be3740 1 Processor -- start 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f1246d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1244020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1244630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f0f1249fb0 con 0x55f0f1247370 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f0f115d060 con 0x55f0f123db40 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f0f124a130 con 0x55f0f123c5d0 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbc1be3740 10 client.0 _ll_register_callbacks cb 0x55f0f0eb08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1244020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbdda7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1244630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1244020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:52400/0 (socket says 172.21.15.143:52400) 2024-09-10T10:52:54.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 -- 172.21.15.143:0/2071817487 learned_addr learned my addr 172.21.15.143:0/2071817487 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f1246d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 -- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 msgr2=0x55f0f1246d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f1246d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 -- 172.21.15.143:0/2071817487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 msgr2=0x55f0f1244630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1244630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 -- 172.21.15.143:0/2071817487 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f0f133c0 con 0x55f0f123db40 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1244630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1244020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcba8012720 tx=0x7fcba8012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248aa2108101f30d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcba0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.435+0000 7fcba0ff9640 1 -- 172.21.15.143:0/2071817487 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba800f920 con 0x55f0f123db40 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcba0ff9640 1 -- 172.21.15.143:0/2071817487 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcba800fac0 con 0x55f0f123db40 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb0009570 con 0x55f0f123db40 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcba0ff9640 1 -- 172.21.15.143:0/2071817487 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcba800fd00 con 0x55f0f123db40 2024-09-10T10:52:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 10 client.26486 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f0f1255aa0 con 0x55f0f123db40 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcba0ff9640 1 -- 172.21.15.143:0/2071817487 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb0009570 con 0x55f0f123db40 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcba0ff9640 1 -- 172.21.15.143:0/2071817487 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fcba8010430 con 0x55f0f123db40 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 10 client.26486 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 10 client.26486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 20 client.26486 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f1256fb0 0x55f0f12593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 4 client.26486 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.436+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0f11af860 con 0x55f0f1256fb0 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.437+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f1256fb0 0x55f0f12593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.437+0000 7fcba0ff9640 10 client.26486 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.437+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f1256fb0 0x55f0f12593d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249b6bf81d2dc689 server_cookie=8c4d934040c8abf5 in_seq=0 out_seq=0 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcba0ff9640 1 -- 172.21.15.143:0/2071817487 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0f11af860 con 0x55f0f1256fb0 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcba0ff9640 10 client.26486 handle_command_reply: tid=0 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 client.26486 shutdown 2024-09-10T10:52:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 msgr2=0x55f0f1244020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1244020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcba8012720 tx=0x7fcba8012be0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 shutdown_connections 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1244630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1244020 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f1256fb0 0x55f0f12593d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcba0ff9640 0 client.26486 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/2071817487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f1246d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 >> 172.21.15.143:0/2071817487 conn(0x55f0f1205310 msgr2=0x55f0f123e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 shutdown_connections 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 1 -- 172.21.15.143:0/2071817487 wait complete. 2024-09-10T10:52:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.438+0000 7fcbc1be3740 20 client.26486 trim_cache size 0 max 16384 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.451+0000 7fcbc1be3740 1 Processor -- start 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 -- start start 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 0x55f0f1238d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1253250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f1247370 0x55f0f1243080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81ac860 con 0x55f0f123c5d0 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81ae3a0 con 0x55f0f123db40 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81acf30 con 0x55f0f1247370 2024-09-10T10:52:54.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 0x55f0f1238d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 0x55f0f1238d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60304/0 (socket says 172.21.15.143:60304) 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/462706617 learned_addr learned my addr 172.21.15.143:0/462706617 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f1247370 0x55f0f1243080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1253250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/462706617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f1247370 msgr2=0x55f0f1243080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f1247370 0x55f0f1243080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/462706617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 msgr2=0x55f0f1253250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1253250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.452+0000 7fcbbd5a6640 1 -- 172.21.15.143:0/462706617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f12425f0 con 0x55f0f123c5d0 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.453+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 0x55f0f1238d70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40096c0 tx=0x7fcbb40b7fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3859fd9c58c82a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.453+0000 7fcba2ffd640 1 -- 172.21.15.143:0/462706617 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4049040 con 0x55f0f123c5d0 2024-09-10T10:52:54.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.453+0000 7fcba2ffd640 1 -- 172.21.15.143:0/462706617 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb40c4800 con 0x55f0f123c5d0 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.453+0000 7fcba2ffd640 1 -- 172.21.15.143:0/462706617 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40ef3d0 con 0x55f0f123c5d0 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.453+0000 7fcbc1be3740 1 -- 172.21.15.143:0/462706617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 msgr2=0x55f0f1238d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 0x55f0f1238d70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40096c0 tx=0x7fcbb40b7fa0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 -- 172.21.15.143:0/462706617 shutdown_connections 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f1247370 0x55f0f1243080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f1253250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/462706617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f123c5d0 0x55f0f1238d70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 -- 172.21.15.143:0/462706617 >> 172.21.15.143:0/462706617 conn(0x55f0f1205310 msgr2=0x55f0f1242970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 -- 172.21.15.143:0/462706617 shutdown_connections 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 -- 172.21.15.143:0/462706617 wait complete. 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 -- start start 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.454+0000 7fcbc1be3740 1 Processor -- start 2024-09-10T10:52:54.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f124f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f124fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1240930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81adcd0 con 0x55f0f1247370 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81ad600 con 0x55f0f123db40 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81aa840 con 0x55f0f123c5d0 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbc1be3740 10 client.0 _ll_register_callbacks cb 0x7fcba8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f124f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f124fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f124fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:52424/0 (socket says 172.21.15.143:52424) 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 -- 172.21.15.143:0/232787874 learned_addr learned my addr 172.21.15.143:0/232787874 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbd5a6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f124f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:43160/0 (socket says 172.21.15.143:43160) 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbdda7640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1240930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 -- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 msgr2=0x55f0f124f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f124f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 -- 172.21.15.143:0/232787874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 msgr2=0x55f0f1240930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1240930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 -- 172.21.15.143:0/232787874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f1241160 con 0x55f0f123db40 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f124f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcbbcda5640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f124fb90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55f0f12455d0 tx=0x7fcba8010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bacd0dbcf0ded9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcba0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.455+0000 7fcba0ff9640 1 -- 172.21.15.143:0/232787874 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba800fea0 con 0x55f0f123db40 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb40c4800 con 0x55f0f123db40 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcba0ff9640 1 -- 172.21.15.143:0/232787874 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fcba8010640 con 0x55f0f123db40 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcba0ff9640 1 -- 172.21.15.143:0/232787874 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcba8036050 con 0x55f0f123db40 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 10 client.26492 fetch_fsmap learned FSMap version 36 2024-09-10T10:52:54.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f0f12c17b0 con 0x55f0f123db40 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcba0ff9640 1 -- 172.21.15.143:0/232787874 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb40c4800 con 0x55f0f123db40 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcba0ff9640 1 -- 172.21.15.143:0/232787874 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fcba802e020 con 0x55f0f123db40 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 10 client.26492 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 20 client.26492 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f12c1f20 0x55f0f12c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 4 client.26492 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0f11a9d70 con 0x55f0f12c1f20 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.456+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f12c1f20 0x55f0f12c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcba0ff9640 10 client.26492 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbbd5a6640 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f12c1f20 0x55f0f12c4340 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2688e12a2eca3ef7 server_cookie=4bf73106f38c158b in_seq=0 out_seq=0 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcba0ff9640 1 -- 172.21.15.143:0/232787874 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0f11a9d70 con 0x55f0f12c1f20 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcba0ff9640 10 client.26492 handle_command_reply: tid=0 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 client.26492 shutdown 2024-09-10T10:52:54.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 msgr2=0x55f0f124fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f124fb90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55f0f12455d0 tx=0x7fcba8010cf0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 shutdown_connections 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f0f1247370 0x55f0f1240930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0f123db40 0x55f0f124fb90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f0f12c1f20 0x55f0f12c4340 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 --2- 172.21.15.143:0/232787874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f0f123c5d0 0x55f0f124f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 >> 172.21.15.143:0/232787874 conn(0x55f0f1205310 msgr2=0x55f0f1243190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcba0ff9640 0 client.26492 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 shutdown_connections 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 1 -- 172.21.15.143:0/232787874 wait complete. 2024-09-10T10:52:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.457+0000 7fcbc1be3740 20 client.26492 trim_cache size 0 max 16384 2024-09-10T10:52:54.472 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:52:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcba40a2b80 msgr2=0x7fcba40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcba40a2b80 0x7fcba40a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000ffb0 tx=0x7fcbb0018000 comp rx=0 tx=0).stop 2024-09-10T10:52:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 msgr2=0x7fcbb819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb819a590 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4005940 tx=0x7fcbb40040c0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 shutdown_connections 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fcba40a2b80 0x7fcba40a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbb810e480 0x7fcbb819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb8075110 0x7fcbb819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 --2- 172.21.15.143:0/1618169173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbb80747f0 0x7fcbb819a590 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 >> 172.21.15.143:0/1618169173 conn(0x7fcbb80fd8d0 msgr2=0x7fcbb80fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 shutdown_connections 2024-09-10T10:52:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:54.458+0000 7fcbbf80a640 1 -- 172.21.15.143:0/1618169173 wait complete. 2024-09-10T10:52:56.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:55 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:52:56.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:55 smithi143 ceph-mon[31057]: pgmap v587: 129 pgs: 129 active+clean; 394 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-09-10T10:52:56.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:55 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:52:56.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:55 smithi178 ceph-mon[34813]: pgmap v587: 129 pgs: 129 active+clean; 394 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-09-10T10:52:56.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:55 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:52:56.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:55 smithi144 ceph-mon[34230]: pgmap v587: 129 pgs: 129 active+clean; 394 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-09-10T10:52:56.210 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:52:56.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:52:56.473 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 -- 172.21.15.143:0/4122279010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f731010c170 msgr2=0x7f731010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 --2- 172.21.15.143:0/4122279010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f731010c170 0x7f731010e610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7308009ca0 tx=0x7f730802a950 comp rx=0 tx=0).stop 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 -- 172.21.15.143:0/4122279010 shutdown_connections 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 --2- 172.21.15.143:0/4122279010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f731010c170 0x7f731010e610 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 --2- 172.21.15.143:0/4122279010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7310103660 0x7f731010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 --2- 172.21.15.143:0/4122279010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7310102d40 0x7f7310103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 -- 172.21.15.143:0/4122279010 >> 172.21.15.143:0/4122279010 conn(0x7f73100fc7a0 msgr2=0x7f73100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 -- 172.21.15.143:0/4122279010 shutdown_connections 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 -- 172.21.15.143:0/4122279010 wait complete. 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.980+0000 7f731878d640 1 Processor -- start 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 -- start start 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7310102d40 0x7f7310198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 0x7f7310198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f731010c170 0x7f731019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7310111e20 con 0x7f7310103660 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7310111ca0 con 0x7f731010c170 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7310111fa0 con 0x7f7310102d40 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 0x7f7310198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 0x7f7310198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52178/0 (socket says 172.21.15.143:52178) 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 -- 172.21.15.143:0/943147854 learned_addr learned my addr 172.21.15.143:0/943147854 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7316529640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7310102d40 0x7f7310198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7316d2a640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f731010c170 0x7f731019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 -- 172.21.15.143:0/943147854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7310102d40 msgr2=0x7f7310198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7310102d40 0x7f7310198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 -- 172.21.15.143:0/943147854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f731010c170 msgr2=0x7f731019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f731010c170 0x7f731019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 -- 172.21.15.143:0/943147854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731019d2b0 con 0x7f7310103660 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f7315d28640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 0x7f7310198840 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f730000e9f0 tx=0x7f730000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f73077fe640 1 -- 172.21.15.143:0/943147854 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730000ccc0 con 0x7f7310103660 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731019d5a0 con 0x7f7310103660 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f73077fe640 1 -- 172.21.15.143:0/943147854 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f730000ce60 con 0x7f7310103660 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.981+0000 7f73077fe640 1 -- 172.21.15.143:0/943147854 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300010430 con 0x7f7310103660 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.982+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73101a4e80 con 0x7f7310103660 2024-09-10T10:52:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.984+0000 7f73077fe640 1 -- 172.21.15.143:0/943147854 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7300005020 con 0x7f7310103660 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.984+0000 7f731ab66740 1 Processor -- start 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.984+0000 7f731ab66740 1 -- start start 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f731ab66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f731ab66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d39c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f731ab66740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f731ab66740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b773c90280 con 0x55b773d3a150 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f731ab66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b773c37b10 con 0x55b773d2ef70 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f731ab66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b773c8c240 con 0x55b773d2f890 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316d2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52194/0 (socket says 172.21.15.143:52194) 2024-09-10T10:52:57.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 -- 172.21.15.143:0/3744128964 learned_addr learned my addr 172.21.15.143:0/3744128964 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316d2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d2f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:49466/0 (socket says 172.21.15.143:49466) 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f73077fe640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72fc0a2b80 0x7f72fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316529640 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d39c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316d2a640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72fc0a2b80 0x7f72fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 -- 172.21.15.143:0/3744128964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 msgr2=0x55b773d39c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d39c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 -- 172.21.15.143:0/3744128964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 msgr2=0x55b773d2f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d2f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f73077fe640 1 -- 172.21.15.143:0/943147854 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f73000cf8d0 con 0x7f7310103660 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 -- 172.21.15.143:0/3744128964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b773c441f0 con 0x55b773d3a150 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316529640 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d39c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316d2a640 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7315d28640 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3c540 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f730009bad0 tx=0x7f7300005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23f7193fd7854cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.985+0000 7f7316d2a640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72fc0a2b80 0x7f72fc0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7308009a60 tx=0x7f730802a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f7304ff9640 1 -- 172.21.15.143:0/3744128964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730009c4d0 con 0x55b773d3a150 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f7304ff9640 1 -- 172.21.15.143:0/3744128964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73000d1e50 con 0x55b773d3a150 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f7304ff9640 1 -- 172.21.15.143:0/3744128964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000984c0 con 0x55b773d3a150 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 -- 172.21.15.143:0/3744128964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 msgr2=0x55b773d3c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3c540 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f730009bad0 tx=0x7f7300005f10 comp rx=0 tx=0).stop 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 -- 172.21.15.143:0/3744128964 shutdown_connections 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3c540 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d39c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 --2- 172.21.15.143:0/3744128964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 -- 172.21.15.143:0/3744128964 >> 172.21.15.143:0/3744128964 conn(0x55b773d2b500 msgr2=0x55b773d2b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 -- 172.21.15.143:0/3744128964 shutdown_connections 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.986+0000 7f731ab66740 1 -- 172.21.15.143:0/3744128964 wait complete. 2024-09-10T10:52:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 -- start start 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 Processor -- start 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 0x55b773d36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2f890 0x55b773d37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d3a150 0x55b773d37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b773d3cd20 con 0x55b773d2ef70 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b773c4af10 con 0x55b773d2f890 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b773d3cea0 con 0x55b773d3a150 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f731ab66740 10 client.0 _ll_register_callbacks cb 0x55b773c13a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 0x55b773d36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 0x55b773d36ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52200/0 (socket says 172.21.15.143:52200) 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 -- 172.21.15.143:0/2872248238 learned_addr learned my addr 172.21.15.143:0/2872248238 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316d2a640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d3a150 0x55b773d37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7315d28640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2f890 0x55b773d37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 -- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d3a150 msgr2=0x55b773d37a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d3a150 0x55b773d37a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 -- 172.21.15.143:0/2872248238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2f890 msgr2=0x55b773d37530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2f890 0x55b773d37530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7316529640 1 -- 172.21.15.143:0/2872248238 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b773a016a0 con 0x55b773d2ef70 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.987+0000 7f7315d28640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2f890 0x55b773d37530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f7316529640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 0x55b773d36ff0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f730c012770 tx=0x7f730c012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4e814bd0dd0ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f72f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f72f9ffb640 1 -- 172.21.15.143:0/2872248238 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c004030 con 0x55b773d2ef70 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f72f9ffb640 1 -- 172.21.15.143:0/2872248238 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f730c0041d0 con 0x55b773d2ef70 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73000d1e50 con 0x55b773d2ef70 2024-09-10T10:52:57.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f72f9ffb640 1 -- 172.21.15.143:0/2872248238 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f730c025050 con 0x55b773d2ef70 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 10 client.17211 fetch_fsmap learned FSMap version 37 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b773d48950 con 0x55b773d2ef70 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f72f9ffb640 1 -- 172.21.15.143:0/2872248238 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73000d1e50 con 0x55b773d2ef70 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f72f9ffb640 1 -- 172.21.15.143:0/2872248238 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f730c0039c0 con 0x55b773d2ef70 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 10 client.17211 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 20 client.17211 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773d49e10 0x55b773d4c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 4 client.17211 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.988+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b773c9d7f0 con 0x55b773d49e10 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.989+0000 7f7315d28640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773d49e10 0x55b773d4c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.989+0000 7f7315d28640 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773d49e10 0x55b773d4c280 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b7a475e03de1902 server_cookie=a050036048ddcb61 in_seq=0 out_seq=0 2024-09-10T10:52:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.989+0000 7f72f9ffb640 10 client.17211 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.989+0000 7f72f9ffb640 1 -- 172.21.15.143:0/2872248238 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b773c9d7f0 con 0x55b773d49e10 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.989+0000 7f72f9ffb640 10 client.17211 handle_command_reply: tid=0 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 client.17211 shutdown 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 msgr2=0x55b773d36ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 0x55b773d36ff0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f730c012770 tx=0x7f730c012c30 comp rx=0 tx=0).stop 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 shutdown_connections 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773d49e10 0x55b773d4c280 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d3a150 0x55b773d37a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2f890 0x55b773d37530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f72f9ffb640 0 client.17211 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 --2- 172.21.15.143:0/2872248238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d2ef70 0x55b773d36ff0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 >> 172.21.15.143:0/2872248238 conn(0x55b773d2b500 msgr2=0x55b773d26270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 shutdown_connections 2024-09-10T10:52:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 1 -- 172.21.15.143:0/2872248238 wait complete. 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:56.990+0000 7f731ab66740 20 client.17211 trim_cache size 0 max 16384 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 Processor -- start 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 -- start start 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d51b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d35790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ac900 con 0x55b773d3a150 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ae470 con 0x55b773d2ef70 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f731ab66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73101acfd0 con 0x55b773d2f890 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d35790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d35790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52206/0 (socket says 172.21.15.143:52206) 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 -- 172.21.15.143:0/2739513101 learned_addr learned my addr 172.21.15.143:0/2739513101 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316529640 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d51b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7315d28640 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 -- 172.21.15.143:0/2739513101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 msgr2=0x55b773d52090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d52090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 -- 172.21.15.143:0/2739513101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 msgr2=0x55b773d51b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d51b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 -- 172.21.15.143:0/2739513101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b773d2c690 con 0x55b773d3a150 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.007+0000 7f7316d2a640 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d35790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f730802b0a0 tx=0x7f7308005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12112e5803aec19f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f72fbfff640 1 -- 172.21.15.143:0/2739513101 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308034040 con 0x55b773d3a150 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f72fbfff640 1 -- 172.21.15.143:0/2739513101 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7308003c30 con 0x55b773d3a150 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f72fbfff640 1 -- 172.21.15.143:0/2739513101 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730804c7e0 con 0x55b773d3a150 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 -- 172.21.15.143:0/2739513101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 msgr2=0x55b773d35790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d35790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f730802b0a0 tx=0x7f7308005df0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 -- 172.21.15.143:0/2739513101 shutdown_connections 2024-09-10T10:52:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d35790 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 --2- 172.21.15.143:0/2739513101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d51b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 -- 172.21.15.143:0/2739513101 >> 172.21.15.143:0/2739513101 conn(0x55b773d2b500 msgr2=0x55b773d26270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 -- 172.21.15.143:0/2739513101 shutdown_connections 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.008+0000 7f731ab66740 1 -- 172.21.15.143:0/2739513101 wait complete. 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 -- start start 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 Processor -- start 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d3dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f73101adda0 con 0x55b773d3a150 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ad6d0 con 0x55b773d2ef70 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73101aa8e0 con 0x55b773d2f890 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f731ab66740 10 client.0 _ll_register_callbacks cb 0x7f730c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52220/0 (socket says 172.21.15.143:52220) 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 -- 172.21.15.143:0/321991807 learned_addr learned my addr 172.21.15.143:0/321991807 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316529640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7315d28640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d3dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 -- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 msgr2=0x55b773d3dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d3dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 -- 172.21.15.143:0/321991807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 msgr2=0x55b773d428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316d2a640 1 -- 172.21.15.143:0/321991807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b773d3e970 con 0x55b773d3a150 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.009+0000 7f7316529640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f7316d2a640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3e140 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7308009600 tx=0x7f730804cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86eb4bf267e36189 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f72f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f72f9ffb640 1 -- 172.21.15.143:0/321991807 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308003ff0 con 0x55b773d3a150 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f72f9ffb640 1 -- 172.21.15.143:0/321991807 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7308004190 con 0x55b773d3a150 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308003c30 con 0x55b773d3a150 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f72f9ffb640 1 -- 172.21.15.143:0/321991807 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7308004390 con 0x55b773d3a150 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 10 client.17217 fetch_fsmap learned FSMap version 37 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b773db4bf0 con 0x55b773d3a150 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f72f9ffb640 1 -- 172.21.15.143:0/321991807 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7308003c30 con 0x55b773d3a150 2024-09-10T10:52:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f72f9ffb640 1 -- 172.21.15.143:0/321991807 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f7308049070 con 0x55b773d3a150 2024-09-10T10:52:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:52:57.031 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:52:57.031 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "1f2ab400-9124-4c9e-aea5-1fb2332f7bed", 2024-09-10T10:52:57.031 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:52:57.031 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 10 client.17217 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 20 client.17217 populate_metadata read hostname 'smithi143' 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773db5250 0x55b773db7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 4 client.17217 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b773c97d00 con 0x55b773db5250 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.010+0000 7f7316529640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773db5250 0x55b773db7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.011+0000 7f7316529640 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773db5250 0x55b773db7640 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6848a50ad175c7bf server_cookie=a69afb06f743dd1c in_seq=0 out_seq=0 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.011+0000 7f72f9ffb640 10 client.17217 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.011+0000 7f72f9ffb640 1 -- 172.21.15.143:0/321991807 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b773c97d00 con 0x55b773db5250 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.011+0000 7f72f9ffb640 10 client.17217 handle_command_reply: tid=0 2024-09-10T10:52:57.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.011+0000 7f731ab66740 1 client.17217 shutdown 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 msgr2=0x55b773d3e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3e140 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7308009600 tx=0x7f730804cd40 comp rx=0 tx=0).stop 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 shutdown_connections 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b773d3a150 0x55b773d3e140 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b773db5250 0x55b773db7640 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b773d2f890 0x55b773d3dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 --2- 172.21.15.143:0/321991807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b773d2ef70 0x55b773d428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f72f9ffb640 0 client.17217 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 >> 172.21.15.143:0/321991807 conn(0x55b773d2b500 msgr2=0x55b773d309c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 shutdown_connections 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 1 -- 172.21.15.143:0/321991807 wait complete. 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731ab66740 20 client.17217 trim_cache size 0 max 16384 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72fc0a2b80 msgr2=0x7f72fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731878d640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72fc0a2b80 0x7f72fc0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7308009a60 tx=0x7f730802a950 comp rx=0 tx=0).stop 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 msgr2=0x7f7310198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.012+0000 7f731878d640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 0x7f7310198840 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f730000e9f0 tx=0x7f730000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 shutdown_connections 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f72fc0a2b80 0x7f72fc0a5040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f731010c170 0x7f731019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7310103660 0x7f7310198840 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 --2- 172.21.15.143:0/943147854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7310102d40 0x7f7310198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 >> 172.21.15.143:0/943147854 conn(0x7f73100fc7a0 msgr2=0x7f73100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 shutdown_connections 2024-09-10T10:52:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:52:57.013+0000 7f731878d640 1 -- 172.21.15.143:0/943147854 wait complete. 2024-09-10T10:52:57.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:57 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:52:57.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:57 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:57.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:57 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:57.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:57 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:52:57.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:57 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:57.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:57 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:57.675 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:52:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: 2024-09-10T10:52:57.350+0000 7fa16b22b640 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x400000015e1 [...2,head] /volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31/ auth{0=1,1=1,2=1,4=1} fragtree_t(*^3) v164981 ap=2 RANDEPHEMERALPIN f(v121 m2024-09-10T10:50:54.396653+0000 177=177+0) n(v9 rc2024-09-10T10:50:54.396653+0000 b3441040 178=177+1) (inest sync->mix g=0,1,2,4) caps={15291=pAsLsXsFs/-@333} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55db23f67180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x400000015e1:[<0x10000000399/blog-31 v133422>,<0x10000000208/blogtest_in v383>,<0x10000000201/src v2970>,<0x10000000006/blogbench-1.0 v2683>,<0x10000000005/tmp v88>,<0x10000000003/client.0 v75>,<0x10000000002/e30d7478-7fa9-47ad-a2b8-7faadc44ff73 v90>,<0x10000000001/sv_0 v107>,<0x10000000000/qa v99>,<0x1/volumes v94>]//[]","memoryvalue":"(2)0x400000015e1:[<0x10000000399/blog-31 v164981>,<0x10000000208/blogtest_in v510>,<0x10000000201/src v3090>,<0x10000000006/blogbench-1.0 v2744>,<0x10000000005/tmp v144>,<0x10000000003/client.0 v133>,<0x10000000002/e30d7478-7fa9-47ad-a2b8-7faadc44ff73 v148>,<0x10000000001/sv_0 v165>,<0x10000000000/qa v153>,<0x1/volumes v148>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T10:50:54.336160+0000 178=178+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T10:50:54.396653+0000 b3441040 178=177+1)","memory_value.dirstat":"f(v121 m2024-09-10T10:50:54.396653+0000 177=177+0)","memory_value.rstat":"n(v9 rc2024-09-10T10:50:54.396653+0000 b3441040 178=177+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T10:52:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:57 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:52:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:57 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:57.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:57 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:58.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:58 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:58.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:58 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:52:58.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:58 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:58.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:58 smithi143 ceph-mon[31057]: Scrub error on inode 0x400000015e1 (/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.f log and `damage ls` output for details 2024-09-10T10:52:58.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:52:58 smithi143 ceph-mon[31057]: pgmap v588: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 500 op/s 2024-09-10T10:52:58.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:58 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:58.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:58 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:52:58.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:58 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:58.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:58 smithi178 ceph-mon[34813]: Scrub error on inode 0x400000015e1 (/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.f log and `damage ls` output for details 2024-09-10T10:52:58.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:52:58 smithi178 ceph-mon[34813]: pgmap v588: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 500 op/s 2024-09-10T10:52:58.685 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:58 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:58 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:52:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:58 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:52:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:58 smithi144 ceph-mon[34230]: Scrub error on inode 0x400000015e1 (/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.f log and `damage ls` output for details 2024-09-10T10:52:58.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:52:58 smithi144 ceph-mon[34230]: pgmap v588: 129 pgs: 129 active+clean; 380 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 500 op/s 2024-09-10T10:52:59.203 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:52:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:00.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 -- 172.21.15.143:0/777048122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 msgr2=0x7feba0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 --2- 172.21.15.143:0/777048122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 0x7feba0108a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feb9001a910 tx=0x7feb9003b430 comp rx=0 tx=0).stop 2024-09-10T10:53:00.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 -- 172.21.15.143:0/777048122 shutdown_connections 2024-09-10T10:53:00.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 --2- 172.21.15.143:0/777048122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0109130 0x7feba0110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 --2- 172.21.15.143:0/777048122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 0x7feba0108a00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 --2- 172.21.15.143:0/777048122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0104000 0x7feba01043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.421+0000 7feba7f88640 1 -- 172.21.15.143:0/777048122 >> 172.21.15.143:0/777048122 conn(0x7feba00ffa20 msgr2=0x7feba0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.422+0000 7feba7f88640 1 -- 172.21.15.143:0/777048122 shutdown_connections 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.422+0000 7feba7f88640 1 -- 172.21.15.143:0/777048122 wait complete. 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.422+0000 7feba7f88640 1 Processor -- start 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.422+0000 7feba7f88640 1 -- start start 2024-09-10T10:53:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba7f88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0104000 0x7feba0110870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba7f88640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 0x7feba010c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba7f88640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 0x7feba010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba7f88640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feba006e4c0 con 0x7feba0109130 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba7f88640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feba006e340 con 0x7feba01049b0 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba7f88640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba006e640 con 0x7feba0104000 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba6525640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 0x7feba010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba6525640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 0x7feba010ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52240/0 (socket says 172.21.15.143:52240) 2024-09-10T10:53:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba6525640 1 -- 172.21.15.143:0/634835332 learned_addr learned my addr 172.21.15.143:0/634835332 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:00.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.423+0000 7feba5d24640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0104000 0x7feba0110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feba6525640 1 -- 172.21.15.143:0/634835332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0104000 msgr2=0x7feba0110870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feba6525640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0104000 0x7feba0110870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feba6525640 1 -- 172.21.15.143:0/634835332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 msgr2=0x7feba010c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feba6525640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 0x7feba010c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feba6525640 1 -- 172.21.15.143:0/634835332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba010d580 con 0x7feba0109130 2024-09-10T10:53:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feba6525640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 0x7feba010ce50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7feb98009f60 tx=0x7feb98009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feb96ffd640 1 -- 172.21.15.143:0/634835332 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98004d20 con 0x7feba0109130 2024-09-10T10:53:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feb96ffd640 1 -- 172.21.15.143:0/634835332 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9800cd00 con 0x7feba0109130 2024-09-10T10:53:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.424+0000 7feb96ffd640 1 -- 172.21.15.143:0/634835332 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98005770 con 0x7feba0109130 2024-09-10T10:53:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.425+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba0076630 con 0x7feba0109130 2024-09-10T10:53:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.425+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba0076b20 con 0x7feba0109130 2024-09-10T10:53:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.429+0000 7febaa361740 1 Processor -- start 2024-09-10T10:53:00.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.430+0000 7feb96ffd640 1 -- 172.21.15.143:0/634835332 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7feb9800c850 con 0x7feba0109130 2024-09-10T10:53:00.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 -- start start 2024-09-10T10:53:00.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb9284a0 0x55eefb928880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb933140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb935a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eefb88e2e0 con 0x55eefb928dc0 2024-09-10T10:53:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eefb835ce0 con 0x55eefb9284a0 2024-09-10T10:53:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.431+0000 7febaa361740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eefb88a2a0 con 0x55eefb933680 2024-09-10T10:53:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.432+0000 7feb96ffd640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb8c0a2d50 0x7feb8c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.432+0000 7feba6525640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb8c0a2d50 0x7feb8c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.432+0000 7feba5d24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb933140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.432+0000 7feba5d24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb933140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52254/0 (socket says 172.21.15.143:52254) 2024-09-10T10:53:00.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.432+0000 7feba5d24640 1 -- 172.21.15.143:0/3327822765 learned_addr learned my addr 172.21.15.143:0/3327822765 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:00.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.432+0000 7feb96ffd640 1 -- 172.21.15.143:0/634835332 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feb980d7b80 con 0x7feba0109130 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5523640 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb9284a0 0x55eefb928880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5523640 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb935a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5d24640 1 -- 172.21.15.143:0/3327822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 msgr2=0x55eefb935a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5d24640 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb935a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5d24640 1 -- 172.21.15.143:0/3327822765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb9284a0 msgr2=0x55eefb928880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5d24640 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb9284a0 0x55eefb928880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba6525640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb8c0a2d50 0x7feb8c0a5210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feb980a09f0 tx=0x7feb9806ec70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5d24640 1 -- 172.21.15.143:0/3327822765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefb81a440 con 0x55eefb928dc0 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.433+0000 7feba5d24640 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb933140 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b820 tx=0x7feb9c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e346440aa9927 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.434+0000 7feb8bfff640 1 -- 172.21.15.143:0/3327822765 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00d950 con 0x55eefb928dc0 2024-09-10T10:53:00.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.434+0000 7feb8bfff640 1 -- 172.21.15.143:0/3327822765 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c019070 con 0x55eefb928dc0 2024-09-10T10:53:00.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.434+0000 7feb8bfff640 1 -- 172.21.15.143:0/3327822765 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0123e0 con 0x55eefb928dc0 2024-09-10T10:53:00.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.435+0000 7febaa361740 1 -- 172.21.15.143:0/3327822765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 msgr2=0x55eefb933140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.435+0000 7febaa361740 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb933140 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b820 tx=0x7feb9c00bce0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 -- 172.21.15.143:0/3327822765 shutdown_connections 2024-09-10T10:53:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb935a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb933140 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 --2- 172.21.15.143:0/3327822765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb9284a0 0x55eefb928880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 -- 172.21.15.143:0/3327822765 >> 172.21.15.143:0/3327822765 conn(0x55eefb924a30 msgr2=0x55eefb924e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 -- 172.21.15.143:0/3327822765 shutdown_connections 2024-09-10T10:53:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.437+0000 7febaa361740 1 -- 172.21.15.143:0/3327822765 wait complete. 2024-09-10T10:53:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.438+0000 7febaa361740 1 -- start start 2024-09-10T10:53:00.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.438+0000 7febaa361740 1 Processor -- start 2024-09-10T10:53:00.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.438+0000 7febaa361740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb9284a0 0x55eefb933000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7febaa361740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 0x55eefb9302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7febaa361740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb930830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7febaa361740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55eefb936250 con 0x55eefb9284a0 2024-09-10T10:53:00.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7febaa361740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eefb849050 con 0x55eefb928dc0 2024-09-10T10:53:00.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7febaa361740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eefb9363d0 con 0x55eefb933680 2024-09-10T10:53:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7febaa361740 10 client.0 _ll_register_callbacks cb 0x55eefb889160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7feba5d24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 0x55eefb9302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7feba5d24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 0x55eefb9302b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:49520/0 (socket says 172.21.15.143:49520) 2024-09-10T10:53:00.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.439+0000 7feba5d24640 1 -- 172.21.15.143:0/2195070633 learned_addr learned my addr 172.21.15.143:0/2195070633 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5523640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb930830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5523640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb9284a0 0x55eefb933000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5d24640 1 -- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 msgr2=0x55eefb930830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5d24640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb930830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5d24640 1 -- 172.21.15.143:0/2195070633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb9284a0 msgr2=0x55eefb933000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5d24640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb9284a0 0x55eefb933000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5d24640 1 -- 172.21.15.143:0/2195070633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eefb5ff3c0 con 0x55eefb928dc0 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feba5d24640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 0x55eefb9302b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb9c006aa0 tx=0x7feb9c014660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1879f7217cb7eef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.440+0000 7feb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.441+0000 7feb897fa640 1 -- 172.21.15.143:0/2195070633 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00dd60 con 0x55eefb928dc0 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.441+0000 7feb897fa640 1 -- 172.21.15.143:0/2195070633 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feb9c01f440 con 0x55eefb928dc0 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.441+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9c019070 con 0x55eefb928dc0 2024-09-10T10:53:00.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.441+0000 7feb897fa640 1 -- 172.21.15.143:0/2195070633 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feb9c01f660 con 0x55eefb928dc0 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.441+0000 7febaa361740 10 client.26516 fetch_fsmap learned FSMap version 38 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.441+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eefb941c40 con 0x55eefb928dc0 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7feb897fa640 1 -- 172.21.15.143:0/2195070633 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c019070 con 0x55eefb928dc0 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7feb897fa640 1 -- 172.21.15.143:0/2195070633 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7feb9c013070 con 0x55eefb928dc0 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7febaa361740 10 client.26516 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7febaa361740 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7febaa361740 20 client.26516 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:00.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7febaa361740 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb942f90 0x55eefb945380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7febaa361740 4 client.26516 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:00.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eefb89b850 con 0x55eefb942f90 2024-09-10T10:53:00.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.442+0000 7feba5523640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb942f90 0x55eefb945380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.443+0000 7feb897fa640 10 client.26516 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:00.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.443+0000 7feba5523640 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb942f90 0x55eefb945380 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3d2d3081d2b76a server_cookie=6cca7f26919b2c77 in_seq=0 out_seq=0 2024-09-10T10:53:00.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.444+0000 7feb897fa640 1 -- 172.21.15.143:0/2195070633 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eefb89b850 con 0x55eefb942f90 2024-09-10T10:53:00.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.444+0000 7feb897fa640 10 client.26516 handle_command_reply: tid=0 2024-09-10T10:53:00.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.444+0000 7febaa361740 1 client.26516 shutdown 2024-09-10T10:53:00.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 msgr2=0x55eefb9302b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 0x55eefb9302b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb9c006aa0 tx=0x7feb9c014660 comp rx=0 tx=0).stop 2024-09-10T10:53:00.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 shutdown_connections 2024-09-10T10:53:00.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb942f90 0x55eefb945380 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb933680 0x55eefb930830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb928dc0 0x55eefb9302b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 --2- 172.21.15.143:0/2195070633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb9284a0 0x55eefb933000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 >> 172.21.15.143:0/2195070633 conn(0x55eefb924a30 msgr2=0x55eefb923b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.445+0000 7feb897fa640 0 client.26516 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:00.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.446+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 shutdown_connections 2024-09-10T10:53:00.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.446+0000 7febaa361740 1 -- 172.21.15.143:0/2195070633 wait complete. 2024-09-10T10:53:00.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.446+0000 7febaa361740 20 client.26516 trim_cache size 0 max 16384 2024-09-10T10:53:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.479+0000 7febaa361740 1 Processor -- start 2024-09-10T10:53:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.479+0000 7febaa361740 1 -- start start 2024-09-10T10:53:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7febaa361740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7febaa361740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb94a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7febaa361740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb94af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7febaa361740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feba01a4270 con 0x55eefb928dc0 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7febaa361740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feba01a5e10 con 0x55eefb933680 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7febaa361740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba01a4940 con 0x55eefb9284a0 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb94a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb94a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52270/0 (socket says 172.21.15.143:52270) 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 -- 172.21.15.143:0/1191296600 learned_addr learned my addr 172.21.15.143:0/1191296600 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5523640 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb94af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5523640 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 -- 172.21.15.143:0/1191296600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 msgr2=0x55eefb93f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 -- 172.21.15.143:0/1191296600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 msgr2=0x55eefb94af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb94af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5d24640 1 -- 172.21.15.143:0/1191296600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefb936380 con 0x55eefb928dc0 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.480+0000 7feba5523640 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb94af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.481+0000 7feba5d24640 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb94a9e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7feb9c02a040 tx=0x7feb9c01dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4042cce0c4001 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.481+0000 7feb8b7fe640 1 -- 172.21.15.143:0/1191296600 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00a070 con 0x55eefb928dc0 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.481+0000 7feb8b7fe640 1 -- 172.21.15.143:0/1191296600 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c019070 con 0x55eefb928dc0 2024-09-10T10:53:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.481+0000 7feb8b7fe640 1 -- 172.21.15.143:0/1191296600 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c01fa60 con 0x55eefb928dc0 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.482+0000 7febaa361740 1 -- 172.21.15.143:0/1191296600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 msgr2=0x55eefb94a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.482+0000 7febaa361740 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb94a9e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7feb9c02a040 tx=0x7feb9c01dc00 comp rx=0 tx=0).stop 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.482+0000 7febaa361740 1 -- 172.21.15.143:0/1191296600 shutdown_connections 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.482+0000 7febaa361740 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb94af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.482+0000 7febaa361740 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb94a9e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.483+0000 7febaa361740 1 --2- 172.21.15.143:0/1191296600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.483+0000 7febaa361740 1 -- 172.21.15.143:0/1191296600 >> 172.21.15.143:0/1191296600 conn(0x55eefb924a30 msgr2=0x55eefb932330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.483+0000 7febaa361740 1 -- 172.21.15.143:0/1191296600 shutdown_connections 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.483+0000 7febaa361740 1 -- 172.21.15.143:0/1191296600 wait complete. 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.484+0000 7febaa361740 1 -- start start 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.484+0000 7febaa361740 1 Processor -- start 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.484+0000 7febaa361740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.484+0000 7febaa361740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb93bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.484+0000 7febaa361740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb936f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7febaa361740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feba01a5710 con 0x55eefb928dc0 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7febaa361740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feba01a5040 con 0x55eefb933680 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7febaa361740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feba01a2270 con 0x55eefb9284a0 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7febaa361740 10 client.0 _ll_register_callbacks cb 0x7feb9c00b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:00.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7feba5523640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7feba5523640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:53532/0 (socket says 172.21.15.143:53532) 2024-09-10T10:53:00.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7feba5523640 1 -- 172.21.15.143:0/3113804981 learned_addr learned my addr 172.21.15.143:0/3113804981 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:00.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7feba5d24640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb93bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.485+0000 7feba5523640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb936f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:00 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:00.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:00 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:00.490 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:00 smithi143 ceph-mon[31057]: pgmap v589: 129 pgs: 129 active+clean; 373 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 824 KiB/s wr, 605 op/s 2024-09-10T10:53:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feba5d24640 1 -- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 msgr2=0x55eefb93b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feba5d24640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feba5d24640 1 -- 172.21.15.143:0/3113804981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 msgr2=0x55eefb936f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feba5d24640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb936f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feba5d24640 1 -- 172.21.15.143:0/3113804981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eefb929a20 con 0x55eefb928dc0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feba5d24640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb93bef0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x55eefb9319d0 tx=0x7feb9c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd46e2abfc7e590f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.486+0000 7feb897fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.487+0000 7feb897fa640 1 -- 172.21.15.143:0/3113804981 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00a070 con 0x55eefb928dc0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.487+0000 7feb897fa640 1 -- 172.21.15.143:0/3113804981 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7feb9c01fc10 con 0x55eefb928dc0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.487+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9c019070 con 0x55eefb928dc0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.488+0000 7feb897fa640 1 -- 172.21.15.143:0/3113804981 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feb9c00d530 con 0x55eefb928dc0 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.488+0000 7febaa361740 10 client.17232 fetch_fsmap learned FSMap version 38 2024-09-10T10:53:00.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.488+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eefb9ad440 con 0x55eefb928dc0 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.488+0000 7feb897fa640 1 -- 172.21.15.143:0/3113804981 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c019070 con 0x55eefb928dc0 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7feb897fa640 1 -- 172.21.15.143:0/3113804981 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7feb9c028430 con 0x55eefb928dc0 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7febaa361740 10 client.17232 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7febaa361740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7febaa361740 20 client.17232 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7febaa361740 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb9ada80 0x55eefb9afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7febaa361740 4 client.17232 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.489+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eefb895d60 con 0x55eefb9ada80 2024-09-10T10:53:00.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.490+0000 7feba5523640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb9ada80 0x55eefb9afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.491+0000 7feba5523640 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb9ada80 0x55eefb9afea0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451b346fe5b38436 server_cookie=aa298cbe730ae64d in_seq=0 out_seq=0 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.491+0000 7feb897fa640 10 client.17232 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7feb897fa640 1 -- 172.21.15.143:0/3113804981 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eefb895d60 con 0x55eefb9ada80 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7feb897fa640 10 client.17232 handle_command_reply: tid=0 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 client.17232 shutdown 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 msgr2=0x55eefb93bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb93bef0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x55eefb9319d0 tx=0x7feb9c00dd90 comp rx=0 tx=0).stop 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 shutdown_connections 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eefb933680 0x55eefb936f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55eefb928dc0 0x55eefb93bef0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55eefb9ada80 0x55eefb9afea0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 --2- 172.21.15.143:0/3113804981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eefb9284a0 0x55eefb93b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 >> 172.21.15.143:0/3113804981 conn(0x55eefb924a30 msgr2=0x55eefb923370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7feb897fa640 0 client.17232 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 shutdown_connections 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 1 -- 172.21.15.143:0/3113804981 wait complete. 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.492+0000 7febaa361740 20 client.17232 trim_cache size 0 max 16384 2024-09-10T10:53:00.497 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": { 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: "1f2ab400-9124-4c9e-aea5-1fb2332f7bed": { 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: "path": "/", 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: "tag": "1f2ab400-9124-4c9e-aea5-1fb2332f7bed", 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: "options": "recursive,force" 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:53:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb8c0a2d50 msgr2=0x7feb8c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb8c0a2d50 0x7feb8c0a5210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feb980a09f0 tx=0x7feb9806ec70 comp rx=0 tx=0).stop 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 msgr2=0x7feba010ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 0x7feba010ce50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7feb98009f60 tx=0x7feb98009f90 comp rx=0 tx=0).stop 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 shutdown_connections 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7feb8c0a2d50 0x7feb8c0a5210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feba0109130 0x7feba010ce50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba01049b0 0x7feba010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 --2- 172.21.15.143:0/634835332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feba0104000 0x7feba0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 >> 172.21.15.143:0/634835332 conn(0x7feba00ffa20 msgr2=0x7feba0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 shutdown_connections 2024-09-10T10:53:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:00.493+0000 7feba7f88640 1 -- 172.21.15.143:0/634835332 wait complete. 2024-09-10T10:53:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:00 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:00 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:00.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:00 smithi178 ceph-mon[34813]: pgmap v589: 129 pgs: 129 active+clean; 373 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 824 KiB/s wr, 605 op/s 2024-09-10T10:53:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:00 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:00 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:00.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:00 smithi144 ceph-mon[34230]: pgmap v589: 129 pgs: 129 active+clean; 373 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 824 KiB/s wr, 605 op/s 2024-09-10T10:53:01.197 INFO:tasks.cephfs.filesystem:scrub status for tag:1f2ab400-9124-4c9e-aea5-1fb2332f7bed - {'path': '/', 'tag': '1f2ab400-9124-4c9e-aea5-1fb2332f7bed', 'options': 'recursive,force'} 2024-09-10T10:53:01.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:01 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:01.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:01 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:02.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:01 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:02.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:02 smithi178 ceph-mon[34813]: pgmap v590: 129 pgs: 129 active+clean; 369 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 KiB/s wr, 789 op/s 2024-09-10T10:53:02.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:02 smithi144 ceph-mon[34230]: pgmap v590: 129 pgs: 129 active+clean; 369 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 KiB/s wr, 789 op/s 2024-09-10T10:53:03.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:02 smithi143 ceph-mon[31057]: pgmap v590: 129 pgs: 129 active+clean; 369 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 KiB/s wr, 789 op/s 2024-09-10T10:53:04.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:03 smithi143 ceph-mon[31057]: pgmap v591: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 1.32k op/s 2024-09-10T10:53:04.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:03 smithi178 ceph-mon[34813]: pgmap v591: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 1.32k op/s 2024-09-10T10:53:04.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:03 smithi144 ceph-mon[34230]: pgmap v591: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 1.32k op/s 2024-09-10T10:53:06.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:05 smithi143 ceph-mon[31057]: pgmap v592: 129 pgs: 129 active+clean; 343 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 793 KiB/s wr, 1.25k op/s 2024-09-10T10:53:06.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:05 smithi178 ceph-mon[34813]: pgmap v592: 129 pgs: 129 active+clean; 343 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 793 KiB/s wr, 1.25k op/s 2024-09-10T10:53:06.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:05 smithi144 ceph-mon[34230]: pgmap v592: 129 pgs: 129 active+clean; 343 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 793 KiB/s wr, 1.25k op/s 2024-09-10T10:53:07.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:06 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:53:07.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:06 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:53:07.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:06 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:53:08.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:53:08.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:53:08.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:53:08.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:07 smithi143 ceph-mon[31057]: pgmap v593: 129 pgs: 129 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1009 KiB/s wr, 1.38k op/s 2024-09-10T10:53:08.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:53:08.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:53:08.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:53:08.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:07 smithi178 ceph-mon[34813]: pgmap v593: 129 pgs: 129 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1009 KiB/s wr, 1.38k op/s 2024-09-10T10:53:08.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:53:08.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:53:08.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:53:08.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:07 smithi144 ceph-mon[34230]: pgmap v593: 129 pgs: 129 active+clean; 328 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1009 KiB/s wr, 1.38k op/s 2024-09-10T10:53:09.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:09 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:53:09.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:09 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:53:09.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:09 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:53:09.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:09 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:53:09.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:09 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:53:09.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:09 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:53:10.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:10 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:53:10.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:10 smithi143 ceph-mon[31057]: pgmap v594: 129 pgs: 129 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 941 KiB/s wr, 1.32k op/s 2024-09-10T10:53:10.674 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:10 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:53:10.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:10 smithi178 ceph-mon[34813]: pgmap v594: 129 pgs: 129 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 941 KiB/s wr, 1.32k op/s 2024-09-10T10:53:10.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:10 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:53:10.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:10 smithi144 ceph-mon[34230]: pgmap v594: 129 pgs: 129 active+clean; 319 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 941 KiB/s wr, 1.32k op/s 2024-09-10T10:53:11.737 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:11 smithi143 ceph-mon[31057]: pgmap v595: 129 pgs: 129 active+clean; 316 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 KiB/s wr, 1.22k op/s 2024-09-10T10:53:12.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:11 smithi178 ceph-mon[34813]: pgmap v595: 129 pgs: 129 active+clean; 316 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 KiB/s wr, 1.22k op/s 2024-09-10T10:53:12.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:11 smithi144 ceph-mon[34230]: pgmap v595: 129 pgs: 129 active+clean; 316 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 KiB/s wr, 1.22k op/s 2024-09-10T10:53:14.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:14 smithi143 ceph-mon[31057]: pgmap v596: 129 pgs: 129 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 1.11k op/s 2024-09-10T10:53:14.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:14 smithi178 ceph-mon[34813]: pgmap v596: 129 pgs: 129 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 1.11k op/s 2024-09-10T10:53:14.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:14 smithi144 ceph-mon[34230]: pgmap v596: 129 pgs: 129 active+clean; 301 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 1.11k op/s 2024-09-10T10:53:15.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:15 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:15.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:15 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:15.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:15 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:16.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:16 smithi143 ceph-mon[31057]: pgmap v597: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 906 KiB/s wr, 535 op/s 2024-09-10T10:53:16.424 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:16 smithi178 ceph-mon[34813]: pgmap v597: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 906 KiB/s wr, 535 op/s 2024-09-10T10:53:16.435 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:16 smithi144 ceph-mon[34230]: pgmap v597: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 906 KiB/s wr, 535 op/s 2024-09-10T10:53:18.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:17 smithi143 ceph-mon[31057]: pgmap v598: 129 pgs: 129 active+clean; 286 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 617 op/s 2024-09-10T10:53:18.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:17 smithi178 ceph-mon[34813]: pgmap v598: 129 pgs: 129 active+clean; 286 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 617 op/s 2024-09-10T10:53:18.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:17 smithi144 ceph-mon[34230]: pgmap v598: 129 pgs: 129 active+clean; 286 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 617 op/s 2024-09-10T10:53:19.976 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:19 smithi178 ceph-mon[34813]: pgmap v599: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 969 KiB/s wr, 520 op/s 2024-09-10T10:53:19.988 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:19 smithi144 ceph-mon[34230]: pgmap v599: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 969 KiB/s wr, 520 op/s 2024-09-10T10:53:20.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:19 smithi143 ceph-mon[31057]: pgmap v599: 129 pgs: 129 active+clean; 280 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 969 KiB/s wr, 520 op/s 2024-09-10T10:53:22.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:21 smithi143 ceph-mon[31057]: pgmap v600: 129 pgs: 129 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 529 op/s 2024-09-10T10:53:22.177 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:21 smithi178 ceph-mon[34813]: pgmap v600: 129 pgs: 129 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 529 op/s 2024-09-10T10:53:22.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:21 smithi144 ceph-mon[34230]: pgmap v600: 129 pgs: 129 active+clean; 275 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 971 KiB/s wr, 529 op/s 2024-09-10T10:53:24.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:23 smithi143 ceph-mon[31057]: pgmap v601: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 603 op/s 2024-09-10T10:53:24.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:23 smithi178 ceph-mon[34813]: pgmap v601: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 603 op/s 2024-09-10T10:53:24.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:23 smithi144 ceph-mon[34230]: pgmap v601: 129 pgs: 129 active+clean; 260 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 603 op/s 2024-09-10T10:53:26.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:25 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:26.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:25 smithi143 ceph-mon[31057]: pgmap v602: 129 pgs: 129 active+clean; 252 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 931 KiB/s wr, 510 op/s 2024-09-10T10:53:26.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:25 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:26.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:25 smithi178 ceph-mon[34813]: pgmap v602: 129 pgs: 129 active+clean; 252 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 931 KiB/s wr, 510 op/s 2024-09-10T10:53:26.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:25 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:26.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:25 smithi144 ceph-mon[34230]: pgmap v602: 129 pgs: 129 active+clean; 252 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 931 KiB/s wr, 510 op/s 2024-09-10T10:53:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:27 smithi143 ceph-mon[31057]: pgmap v603: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 618 op/s 2024-09-10T10:53:28.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:27 smithi178 ceph-mon[34813]: pgmap v603: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 618 op/s 2024-09-10T10:53:28.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:27 smithi144 ceph-mon[34230]: pgmap v603: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 618 op/s 2024-09-10T10:53:30.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:29 smithi143 ceph-mon[31057]: pgmap v604: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.0 MiB/s wr, 539 op/s 2024-09-10T10:53:30.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:29 smithi178 ceph-mon[34813]: pgmap v604: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.0 MiB/s wr, 539 op/s 2024-09-10T10:53:30.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:29 smithi144 ceph-mon[34230]: pgmap v604: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.0 MiB/s wr, 539 op/s 2024-09-10T10:53:31.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:53:31.536 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:32.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:31 smithi143 ceph-mon[31057]: pgmap v605: 129 pgs: 129 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 749 KiB/s wr, 537 op/s 2024-09-10T10:53:32.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:31 smithi178 ceph-mon[34813]: pgmap v605: 129 pgs: 129 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 749 KiB/s wr, 537 op/s 2024-09-10T10:53:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:31 smithi144 ceph-mon[34230]: pgmap v605: 129 pgs: 129 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 749 KiB/s wr, 537 op/s 2024-09-10T10:53:32.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 -- 172.21.15.143:0/2649971879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540102dd0 msgr2=0x7f35401031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 --2- 172.21.15.143:0/2649971879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540102dd0 0x7f35401031b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f353c009930 tx=0x7f353c02a440 comp rx=0 tx=0).stop 2024-09-10T10:53:32.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 -- 172.21.15.143:0/2649971879 shutdown_connections 2024-09-10T10:53:32.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 --2- 172.21.15.143:0/2649971879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540108300 0x7f354010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 --2- 172.21.15.143:0/2649971879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540103780 0x7f3540107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 --2- 172.21.15.143:0/2649971879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540102dd0 0x7f35401031b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 -- 172.21.15.143:0/2649971879 >> 172.21.15.143:0/2649971879 conn(0x7f35400fc7d0 msgr2=0x7f35400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 -- 172.21.15.143:0/2649971879 shutdown_connections 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.575+0000 7f35477f6640 1 -- 172.21.15.143:0/2649971879 wait complete. 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 Processor -- start 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 -- start start 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540102dd0 0x7f3540198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 0x7f3540198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540108300 0x7f354019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3540111ff0 con 0x7f3540103780 2024-09-10T10:53:32.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3540111e70 con 0x7f3540102dd0 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f35477f6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3540112170 con 0x7f3540108300 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f3544d91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 0x7f3540198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f3544d91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 0x7f3540198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55210/0 (socket says 172.21.15.143:55210) 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f3544d91640 1 -- 172.21.15.143:0/1897343438 learned_addr learned my addr 172.21.15.143:0/1897343438 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.576+0000 7f3545d93640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540108300 0x7f354019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3545592640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540102dd0 0x7f3540198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3544d91640 1 -- 172.21.15.143:0/1897343438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540108300 msgr2=0x7f354019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3544d91640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540108300 0x7f354019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3544d91640 1 -- 172.21.15.143:0/1897343438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540102dd0 msgr2=0x7f3540198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3544d91640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540102dd0 0x7f3540198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3544d91640 1 -- 172.21.15.143:0/1897343438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354019d520 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f3544d91640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 0x7f3540198ab0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3530006fd0 tx=0x7f353000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f35367fc640 1 -- 172.21.15.143:0/1897343438 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530011830 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354019d810 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f35367fc640 1 -- 172.21.15.143:0/1897343438 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3530016070 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f35367fc640 1 -- 172.21.15.143:0/1897343438 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530010df0 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.577+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401a50f0 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.579+0000 7f35367fc640 1 -- 172.21.15.143:0/1897343438 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f35300119d0 con 0x7f3540103780 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 Processor -- start 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 -- start start 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd75880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd80140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd82a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56238fcdb2e0 con 0x56238fd75dc0 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56238fc82ce0 con 0x56238fd80680 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3549bcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56238fcd72a0 con 0x56238fd754a0 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f35367fc640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f352c0a2b80 0x7f352c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3545592640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd80140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.580+0000 7f3545592640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd80140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55214/0 (socket says 172.21.15.143:55214) 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545592640 1 -- 172.21.15.143:0/4217389836 learned_addr learned my addr 172.21.15.143:0/4217389836 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd75880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3544d91640 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd82a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f352c0a2b80 0x7f352c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 -- 172.21.15.143:0/4217389836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 msgr2=0x56238fd82a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd82a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f35367fc640 1 -- 172.21.15.143:0/1897343438 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f35300ce9f0 con 0x7f3540103780 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 -- 172.21.15.143:0/4217389836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 msgr2=0x56238fd80140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd80140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 -- 172.21.15.143:0/4217389836 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238fc67440 con 0x56238fd754a0 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545592640 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd80140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd75880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3538009370 tx=0x7f3538003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8e139db51f5ade server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f3545d93640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f352c0a2b80 0x7f352c0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3538009990 tx=0x7f3538020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f352b7fe640 1 -- 172.21.15.143:0/4217389836 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538026070 con 0x56238fd754a0 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f352b7fe640 1 -- 172.21.15.143:0/4217389836 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f353801c040 con 0x56238fd754a0 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.581+0000 7f352b7fe640 1 -- 172.21.15.143:0/4217389836 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353801b050 con 0x56238fd754a0 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 -- 172.21.15.143:0/4217389836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 msgr2=0x56238fd75880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd75880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3538009370 tx=0x7f3538003f90 comp rx=0 tx=0).stop 2024-09-10T10:53:32.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 -- 172.21.15.143:0/4217389836 shutdown_connections 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd82a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd80140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 --2- 172.21.15.143:0/4217389836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd75880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 -- 172.21.15.143:0/4217389836 >> 172.21.15.143:0/4217389836 conn(0x56238fd71a30 msgr2=0x56238fd71e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 -- 172.21.15.143:0/4217389836 shutdown_connections 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.582+0000 7f3549bcf740 1 -- 172.21.15.143:0/4217389836 wait complete. 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 -- start start 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 Processor -- start 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd754a0 0x56238fd80030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd7d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56238fd83250 con 0x56238fd75dc0 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56238fc96050 con 0x56238fd754a0 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56238fd833d0 con 0x56238fd80680 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.583+0000 7f3549bcf740 10 client.0 _ll_register_callbacks cb 0x56238fcd6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3545d93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd7d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3545d93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd7d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45282/0 (socket says 172.21.15.143:45282) 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3545d93640 1 -- 172.21.15.143:0/896745590 learned_addr learned my addr 172.21.15.143:0/896745590 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3545592640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd754a0 0x56238fd80030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 -- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 msgr2=0x56238fd7d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd7d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 -- 172.21.15.143:0/896745590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd754a0 msgr2=0x56238fd80030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd754a0 0x56238fd80030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 -- 172.21.15.143:0/896745590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238fa4c3c0 con 0x56238fd75dc0 2024-09-10T10:53:32.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3545d93640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd7d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3544d91640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd7d2e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f353009abf0 tx=0x7f353000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0371f71c546caf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3528ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3528ff9640 1 -- 172.21.15.143:0/896745590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353009b5f0 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.584+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353801c040 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3528ff9640 1 -- 172.21.15.143:0/896745590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f35300991f0 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3528ff9640 1 -- 172.21.15.143:0/896745590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f353009aef0 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 10 client.17244 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56238fd8ed10 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3528ff9640 1 -- 172.21.15.143:0/896745590 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f353801c040 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3528ff9640 1 -- 172.21.15.143:0/896745590 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7f353009a340 con 0x56238fd75dc0 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 10 client.17244 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 20 client.17244 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fd90120 0x56238fd92510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 4 client.17244 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:32.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56238fce8850 con 0x56238fd90120 2024-09-10T10:53:32.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.585+0000 7f3545592640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fd90120 0x56238fd92510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.586+0000 7f3528ff9640 10 client.17244 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:32.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.586+0000 7f3545592640 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fd90120 0x56238fd92510 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a358e092b29e9697 server_cookie=3767d59f26551abd in_seq=0 out_seq=0 2024-09-10T10:53:32.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.586+0000 7f3528ff9640 1 -- 172.21.15.143:0/896745590 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56238fce8850 con 0x56238fd90120 2024-09-10T10:53:32.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.586+0000 7f3528ff9640 10 client.17244 handle_command_reply: tid=0 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.586+0000 7f3549bcf740 1 client.17244 shutdown 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 msgr2=0x56238fd7d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd7d2e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f353009abf0 tx=0x7f353000c3a0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 shutdown_connections 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fd90120 0x56238fd92510 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd7d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd75dc0 0x56238fd7d2e0 secure :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f353009abf0 tx=0x7f353000c3a0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3528ff9640 0 client.17244 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 --2- 172.21.15.143:0/896745590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd754a0 0x56238fd80030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 >> 172.21.15.143:0/896745590 conn(0x56238fd71a30 msgr2=0x56238fd77760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 shutdown_connections 2024-09-10T10:53:32.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.587+0000 7f3549bcf740 1 -- 172.21.15.143:0/896745590 wait complete. 2024-09-10T10:53:32.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.588+0000 7f3549bcf740 20 client.17244 trim_cache size 0 max 16384 2024-09-10T10:53:32.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 Processor -- start 2024-09-10T10:53:32.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 -- start start 2024-09-10T10:53:32.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 0x56238fd7bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd97dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd98300 0x56238fd7a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f35401aca40 con 0x56238fd754a0 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35401ae6a0 con 0x56238fd98300 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.606+0000 7f3549bcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35401ad110 con 0x56238fd80680 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 0x56238fd7bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 0x56238fd7bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55224/0 (socket says 172.21.15.143:55224) 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3544d91640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd97dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 -- 172.21.15.143:0/683272147 learned_addr learned my addr 172.21.15.143:0/683272147 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545d93640 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd98300 0x56238fd7a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 -- 172.21.15.143:0/683272147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 msgr2=0x56238fd97dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd97dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 -- 172.21.15.143:0/683272147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd98300 msgr2=0x56238fd7a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd98300 0x56238fd7a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 -- 172.21.15.143:0/683272147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238fd7c290 con 0x56238fd754a0 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f3545592640 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 0x56238fd7bbb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f353c02aa00 tx=0x7f353c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29facf10fa763c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f352affd640 1 -- 172.21.15.143:0/683272147 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c035040 con 0x56238fd754a0 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f352affd640 1 -- 172.21.15.143:0/683272147 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f353c04a070 con 0x56238fd754a0 2024-09-10T10:53:32.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.607+0000 7f352affd640 1 -- 172.21.15.143:0/683272147 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c003850 con 0x56238fd754a0 2024-09-10T10:53:32.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 -- 172.21.15.143:0/683272147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 msgr2=0x56238fd7bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 0x56238fd7bbb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f353c02aa00 tx=0x7f353c005a40 comp rx=0 tx=0).stop 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 -- 172.21.15.143:0/683272147 shutdown_connections 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd98300 0x56238fd7a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd80680 0x56238fd97dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 --2- 172.21.15.143:0/683272147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd754a0 0x56238fd7bbb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 -- 172.21.15.143:0/683272147 >> 172.21.15.143:0/683272147 conn(0x56238fd71a30 msgr2=0x56238fd7f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 -- 172.21.15.143:0/683272147 shutdown_connections 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 -- 172.21.15.143:0/683272147 wait complete. 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.608+0000 7f3549bcf740 1 -- start start 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 Processor -- start 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd86fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd87510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd98300 0x56238fd88450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f35401adf10 con 0x56238fd98300 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35401ad810 con 0x56238fd80680 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35401aaa20 con 0x56238fd754a0 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3549bcf740 10 client.0 _ll_register_callbacks cb 0x7f35300107b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3545d93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd98300 0x56238fd88450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3545592640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd86fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3544d91640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd87510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3545d93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd98300 0x56238fd88450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55240/0 (socket says 172.21.15.143:55240) 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3545592640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd86fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45302/0 (socket says 172.21.15.143:45302) 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.609+0000 7f3545d93640 1 -- 172.21.15.143:0/578996945 learned_addr learned my addr 172.21.15.143:0/578996945 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3544d91640 1 -- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 msgr2=0x56238fd86fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3544d91640 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd86fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3544d91640 1 -- 172.21.15.143:0/578996945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd98300 msgr2=0x56238fd88450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3544d91640 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd98300 0x56238fd88450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3544d91640 1 -- 172.21.15.143:0/578996945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238fd7dbf0 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3544d91640 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd87510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x56238fd7e3d0 tx=0x7f35300047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3939c1511c1868 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3528ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3528ff9640 1 -- 172.21.15.143:0/578996945 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530097190 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3528ff9640 1 -- 172.21.15.143:0/578996945 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f353009a340 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353c04a070 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3528ff9640 1 -- 172.21.15.143:0/578996945 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f353009b5f0 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3549bcf740 10 client.26537 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56238fdfb7b0 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3528ff9640 1 -- 172.21.15.143:0/578996945 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35300b6020 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3528ff9640 1 -- 172.21.15.143:0/578996945 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7f3530096c90 con 0x56238fd80680 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.610+0000 7f3549bcf740 10 client.26537 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:32.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 10 client.26537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 20 client.26537 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fdfbf00 0x56238fdfe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 4 client.26537 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56238fce2d60 con 0x56238fdfbf00 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3545592640 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fdfbf00 0x56238fdfe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3545592640 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fdfbf00 0x56238fdfe320 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe436d8980bb3a6 server_cookie=4a24cc87c34dc851 in_seq=0 out_seq=0 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3528ff9640 10 client.26537 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3528ff9640 1 -- 172.21.15.143:0/578996945 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56238fce2d60 con 0x56238fdfbf00 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3528ff9640 10 client.26537 handle_command_reply: tid=0 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 client.26537 shutdown 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 msgr2=0x56238fd87510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd87510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x56238fd7e3d0 tx=0x7f35300047d0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 shutdown_connections 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56238fd98300 0x56238fd88450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56238fd80680 0x56238fd87510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x56238fdfbf00 0x56238fdfe320 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 --2- 172.21.15.143:0/578996945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56238fd754a0 0x56238fd86fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3528ff9640 0 client.26537 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.611+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 >> 172.21.15.143:0/578996945 conn(0x56238fd71a30 msgr2=0x56238fd7f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 shutdown_connections 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f3549bcf740 1 -- 172.21.15.143:0/578996945 wait complete. 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f3549bcf740 20 client.26537 trim_cache size 0 max 16384 2024-09-10T10:53:32.618 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:53:32.619 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:53:32.619 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:53:32.619 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:53:32.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f352c0a2b80 msgr2=0x7f352c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f352c0a2b80 0x7f352c0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3538009990 tx=0x7f3538020000 comp rx=0 tx=0).stop 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 msgr2=0x7f3540198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 0x7f3540198ab0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3530006fd0 tx=0x7f353000b8e0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 shutdown_connections 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f352c0a2b80 0x7f352c0a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540108300 0x7f354019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103780 0x7f3540198ab0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 --2- 172.21.15.143:0/1897343438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540102dd0 0x7f3540198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 >> 172.21.15.143:0/1897343438 conn(0x7f35400fc7d0 msgr2=0x7f3540105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.612+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 shutdown_connections 2024-09-10T10:53:32.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:32.613+0000 7f35477f6640 1 -- 172.21.15.143:0/1897343438 wait complete. 2024-09-10T10:53:33.445 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:53:33.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:53:33.877 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:33.999 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:33 smithi143 ceph-mon[31057]: pgmap v606: 129 pgs: 129 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 591 op/s 2024-09-10T10:53:34.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:33 smithi178 ceph-mon[34813]: pgmap v606: 129 pgs: 129 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 591 op/s 2024-09-10T10:53:34.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:33 smithi144 ceph-mon[34230]: pgmap v606: 129 pgs: 129 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 591 op/s 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1449803845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0102790 msgr2=0x7f6aa01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1449803845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0102790 0x7f6aa01067e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009900 tx=0x7f6a9402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1449803845 shutdown_connections 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1449803845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6aa0106f10 0x7f6aa010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1449803845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0102790 0x7f6aa01067e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1449803845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0101de0 0x7f6aa01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.428+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1449803845 >> 172.21.15.143:0/1449803845 conn(0x7f6aa00fd880 msgr2=0x7f6aa00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.429+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1449803845 shutdown_connections 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.429+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1449803845 wait complete. 2024-09-10T10:53:34.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.429+0000 7f6aa5dfe640 1 Processor -- start 2024-09-10T10:53:34.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 -- start start 2024-09-10T10:53:34.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0101de0 0x7f6aa010e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6aa0102790 0x7f6aa010ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0106f10 0x7f6aa010bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0075e30 con 0x7f6aa0102790 2024-09-10T10:53:34.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0075cb0 con 0x7f6aa0101de0 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6aa5dfe640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0075fb0 con 0x7f6aa0106f10 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0106f10 0x7f6aa010bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0106f10 0x7f6aa010bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:45312/0 (socket says 172.21.15.143:45312) 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6a9ffff640 1 -- 172.21.15.143:0/1650646323 learned_addr learned my addr 172.21.15.143:0/1650646323 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6a9effd640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6aa0102790 0x7f6aa010ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.430+0000 7f6a9f7fe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0101de0 0x7f6aa010e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.431+0000 7f6a9f7fe640 1 -- 172.21.15.143:0/1650646323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0106f10 msgr2=0x7f6aa010bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.431+0000 7f6a9f7fe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0106f10 0x7f6aa010bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.431+0000 7f6a9f7fe640 1 -- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6aa0102790 msgr2=0x7f6aa010ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.431+0000 7f6a9f7fe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6aa0102790 0x7f6aa010ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.431+0000 7f6a9f7fe640 1 -- 172.21.15.143:0/1650646323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa010c860 con 0x7f6aa0101de0 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.431+0000 7f6a9f7fe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0101de0 0x7f6aa010e970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800eb00 tx=0x7f6a9800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.432+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98010430 con 0x7f6aa0101de0 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.432+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a98010a80 con 0x7f6aa0101de0 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.432+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98017b10 con 0x7f6aa0101de0 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.432+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa01a06b0 con 0x7f6aa0101de0 2024-09-10T10:53:34.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.432+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa01a0ba0 con 0x7f6aa0101de0 2024-09-10T10:53:34.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.435+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa01032c0 con 0x7f6aa0101de0 2024-09-10T10:53:34.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.435+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6a98010c30 con 0x7f6aa0101de0 2024-09-10T10:53:34.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.436+0000 7f6a9cff9640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a880a2d10 0x7f6a880a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.436+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6a980cfc90 con 0x7f6aa0101de0 2024-09-10T10:53:34.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.437+0000 7f6a9effd640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a880a2d10 0x7f6a880a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.440+0000 7f6a9effd640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a880a2d10 0x7f6a880a51d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6a94005ab0 tx=0x7f6a94000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.441+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a9809c890 con 0x7f6aa0101de0 2024-09-10T10:53:34.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.795+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa01a1680 con 0x7f6aa0101de0 2024-09-10T10:53:34.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.796+0000 7f6a9cff9640 1 -- 172.21.15.143:0/1650646323 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12010 (secure 0 0 0) 0x7f6a9809c190 con 0x7f6aa0101de0 2024-09-10T10:53:34.798 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:53:34.798 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":40,"btime":"2024-09-10T10:53:24:699958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:53:24.112818+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:53:34.798 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 40 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.798+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a880a2d10 msgr2=0x7f6a880a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.798+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a880a2d10 0x7f6a880a51d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6a94005ab0 tx=0x7f6a94000bc0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0101de0 msgr2=0x7f6aa010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0101de0 0x7f6aa010e970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800eb00 tx=0x7f6a9800c2d0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 shutdown_connections 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6a880a2d10 0x7f6a880a51d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa0106f10 0x7f6aa010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6aa0102790 0x7f6aa010ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 --2- 172.21.15.143:0/1650646323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa0101de0 0x7f6aa010e970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 >> 172.21.15.143:0/1650646323 conn(0x7f6aa00fd880 msgr2=0x7f6aa00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:34.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 shutdown_connections 2024-09-10T10:53:34.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:34.799+0000 7f6aa5dfe640 1 -- 172.21.15.143:0/1650646323 wait complete. 2024-09-10T10:53:35.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:34 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1650646323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:35.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:34 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1650646323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:35.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:34 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1650646323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:35.632 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:53:35.905 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:36.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:35 smithi178 ceph-mon[34813]: pgmap v607: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 997 KiB/s wr, 479 op/s 2024-09-10T10:53:36.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:35 smithi144 ceph-mon[34230]: pgmap v607: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 997 KiB/s wr, 479 op/s 2024-09-10T10:53:36.192 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:35 smithi143 ceph-mon[31057]: pgmap v607: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 997 KiB/s wr, 479 op/s 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 -- 172.21.15.143:0/587294505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38074590 msgr2=0x7f8a38074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 --2- 172.21.15.143:0/587294505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38074590 0x7f8a38074970 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009900 tx=0x7f8a2802a480 comp rx=0 tx=0).stop 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 -- 172.21.15.143:0/587294505 shutdown_connections 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 --2- 172.21.15.143:0/587294505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38112ba0 0x7f8a38114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 --2- 172.21.15.143:0/587294505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a38074eb0 0x7f8a38112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 --2- 172.21.15.143:0/587294505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38074590 0x7f8a38074970 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 -- 172.21.15.143:0/587294505 >> 172.21.15.143:0/587294505 conn(0x7f8a380fd8b0 msgr2=0x7f8a380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.660+0000 7f8a407f2640 1 -- 172.21.15.143:0/587294505 shutdown_connections 2024-09-10T10:53:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.661+0000 7f8a407f2640 1 -- 172.21.15.143:0/587294505 wait complete. 2024-09-10T10:53:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 Processor -- start 2024-09-10T10:53:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 -- start start 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a38074590 0x7f8a381a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 0x7f8a381a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38112ba0 0x7f8a3819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38118430 con 0x7f8a38112ba0 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381182b0 con 0x7f8a38074eb0 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a407f2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381185b0 con 0x7f8a38074590 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.662+0000 7f8a3dd8d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 0x7f8a381a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 0x7f8a381a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:38876/0 (socket says 172.21.15.143:38876) 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/301218133 learned_addr learned my addr 172.21.15.143:0/301218133 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38112ba0 0x7f8a3819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a38074590 0x7f8a381a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/301218133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a38074590 msgr2=0x7f8a381a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a38074590 0x7f8a381a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38112ba0 msgr2=0x7f8a3819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38112ba0 0x7f8a3819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/301218133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3819efc0 con 0x7f8a38074eb0 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38112ba0 0x7f8a3819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.663+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 0x7f8a381a4be0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9f0 tx=0x7f8a3400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.664+0000 7f8a2f7fe640 1 -- 172.21.15.143:0/301218133 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400cd80 con 0x7f8a38074eb0 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.664+0000 7f8a2f7fe640 1 -- 172.21.15.143:0/301218133 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a340054d0 con 0x7f8a38074eb0 2024-09-10T10:53:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.664+0000 7f8a2f7fe640 1 -- 172.21.15.143:0/301218133 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400cd80 con 0x7f8a38074eb0 2024-09-10T10:53:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.664+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3819f2b0 con 0x7f8a38074eb0 2024-09-10T10:53:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.664+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381ab500 con 0x7f8a38074eb0 2024-09-10T10:53:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 Processor -- start 2024-09-10T10:53:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 -- start start 2024-09-10T10:53:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 0x55b9a7b99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7b99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b9a7aff2f0 con 0x55b9a7ba4370 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b9a7aa6cf0 con 0x55b9a7b9ab40 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.669+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b9a7afb2b0 con 0x55b9a7b995d0 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58982/0 (socket says 172.21.15.143:58982) 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/334726183 learned_addr learned my addr 172.21.15.143:0/334726183 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a2f7fe640 1 -- 172.21.15.143:0/301218133 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8a34004e40 con 0x7f8a38074eb0 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/334726183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 msgr2=0x55b9a7b99a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:53:36.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7b99a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/334726183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 msgr2=0x55b9a7b99090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 0x55b9a7b99090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.670+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/334726183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a7a8b3a0 con 0x55b9a7ba4370 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.671+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba6760 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8a340d2f90 tx=0x7f8a340d71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de4b3fb6c378294 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.671+0000 7f8a2f7fe640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a240a3010 0x7f8a240a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.671+0000 7f8a2cff9640 1 -- 172.21.15.143:0/334726183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a340e2070 con 0x55b9a7ba4370 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.671+0000 7f8a2cff9640 1 -- 172.21.15.143:0/334726183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a340bd0c0 con 0x55b9a7ba4370 2024-09-10T10:53:36.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.671+0000 7f8a2f7fe640 1 -- 172.21.15.143:0/301218133 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8a3400cb80 con 0x7f8a38074eb0 2024-09-10T10:53:36.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a240a3010 0x7f8a240a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a2cff9640 1 -- 172.21.15.143:0/334726183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3409b100 con 0x55b9a7ba4370 2024-09-10T10:53:36.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 -- 172.21.15.143:0/334726183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 msgr2=0x55b9a7ba6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba6760 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8a340d2f90 tx=0x7f8a340d71a0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a240a3010 0x7f8a240a54d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8a300092f0 tx=0x7f8a30009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 -- 172.21.15.143:0/334726183 shutdown_connections 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba6760 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7b99a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/334726183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 0x55b9a7b99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.672+0000 7f8a42bcb740 1 -- 172.21.15.143:0/334726183 >> 172.21.15.143:0/334726183 conn(0x55b9a7b62310 msgr2=0x55b9a7b95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.673+0000 7f8a42bcb740 1 -- 172.21.15.143:0/334726183 shutdown_connections 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.673+0000 7f8a42bcb740 1 -- 172.21.15.143:0/334726183 wait complete. 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.673+0000 7f8a42bcb740 1 -- start start 2024-09-10T10:53:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.673+0000 7f8a42bcb740 1 Processor -- start 2024-09-10T10:53:36.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7ba3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 0x55b9a7ba0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b9a7bb8120 con 0x55b9a7b9ab40 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b9a7aba060 con 0x55b9a7ba4370 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b9a7bb82a0 con 0x55b9a7b995d0 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a42bcb740 10 client.0 _ll_register_callbacks cb 0x55b9a780d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a3dd8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 0x55b9a7ba0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a3dd8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 0x55b9a7ba0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58990/0 (socket says 172.21.15.143:58990) 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/3906695210 learned_addr learned my addr 172.21.15.143:0/3906695210 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7ba3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.674+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/3906695210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 msgr2=0x55b9a7ba3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7ba3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 msgr2=0x55b9a7ba15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a3dd8d640 1 -- 172.21.15.143:0/3906695210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a78703c0 con 0x55b9a7b9ab40 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7ba3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 0x55b9a7ba0fe0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9c0 tx=0x7f8a3409b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeb71ee6b5d6267 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a21ffb640 1 -- 172.21.15.143:0/3906695210 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34058900 con 0x55b9a7b9ab40 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a21ffb640 1 -- 172.21.15.143:0/3906695210 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8a3400cd80 con 0x55b9a7b9ab40 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a340bd0c0 con 0x55b9a7b9ab40 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a21ffb640 1 -- 172.21.15.143:0/3906695210 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8a34058ad0 con 0x55b9a7b9ab40 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a42bcb740 10 client.17268 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:36.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.675+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b9a7bb8120 con 0x55b9a7b9ab40 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a21ffb640 1 -- 172.21.15.143:0/3906695210 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a34053020 con 0x55b9a7b9ab40 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a21ffb640 1 -- 172.21.15.143:0/3906695210 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7f8a34086070 con 0x55b9a7b9ab40 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a42bcb740 10 client.17268 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a42bcb740 10 client.17268 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a42bcb740 20 client.17268 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7bc4d00 0x55b9a7bc70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a42bcb740 4 client.17268 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.676+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a7b0c860 con 0x55b9a7bc4d00 2024-09-10T10:53:36.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.677+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7bc4d00 0x55b9a7bc70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.677+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7bc4d00 0x55b9a7bc70f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7cfda528ecd7814e server_cookie=638597d1214f83be in_seq=0 out_seq=0 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.677+0000 7f8a21ffb640 10 client.17268 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.678+0000 7f8a21ffb640 1 -- 172.21.15.143:0/3906695210 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9a7b0c860 con 0x55b9a7bc4d00 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.678+0000 7f8a21ffb640 10 client.17268 handle_command_reply: tid=0 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 client.17268 shutdown 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 msgr2=0x55b9a7ba0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 0x55b9a7ba0fe0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9c0 tx=0x7f8a3409b350 comp rx=0 tx=0).stop 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 shutdown_connections 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7bc4d00 0x55b9a7bc70f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7ba15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b9ab40 0x55b9a7ba0fe0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/3906695210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b995d0 0x55b9a7ba3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 >> 172.21.15.143:0/3906695210 conn(0x55b9a7b62310 msgr2=0x55b9a7b9b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a21ffb640 0 client.17268 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 shutdown_connections 2024-09-10T10:53:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 1 -- 172.21.15.143:0/3906695210 wait complete. 2024-09-10T10:53:36.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.679+0000 7f8a42bcb740 20 client.17268 trim_cache size 0 max 16384 2024-09-10T10:53:36.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.704+0000 7f8a42bcb740 1 Processor -- start 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.704+0000 7f8a42bcb740 1 -- start start 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 0x55b9a7bc13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 0x55b9a7b9bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7ba4370 0x55b9a7b9c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381ac9c0 con 0x55b9a7b995d0 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381ae620 con 0x55b9a7b9ab40 2024-09-10T10:53:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381ad090 con 0x55b9a7ba4370 2024-09-10T10:53:36.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a3e58e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 0x55b9a7bc13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a3e58e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 0x55b9a7bc13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59006/0 (socket says 172.21.15.143:59006) 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a3e58e640 1 -- 172.21.15.143:0/810717634 learned_addr learned my addr 172.21.15.143:0/810717634 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.705+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7ba4370 0x55b9a7b9c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.706+0000 7f8a3e58e640 1 -- 172.21.15.143:0/810717634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7ba4370 msgr2=0x55b9a7b9c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.706+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7ba4370 0x55b9a7b9c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.706+0000 7f8a3e58e640 1 -- 172.21.15.143:0/810717634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 msgr2=0x55b9a7b9bfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.706+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 0x55b9a7b9bfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.706+0000 7f8a3e58e640 1 -- 172.21.15.143:0/810717634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a7b96960 con 0x55b9a7b995d0 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.706+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 0x55b9a7bc13c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005ab0 tx=0x7f8a28032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03bfb2702df466b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.707+0000 7f8a23fff640 1 -- 172.21.15.143:0/810717634 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28049070 con 0x55b9a7b995d0 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.707+0000 7f8a23fff640 1 -- 172.21.15.143:0/810717634 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a28005040 con 0x55b9a7b995d0 2024-09-10T10:53:36.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.707+0000 7f8a23fff640 1 -- 172.21.15.143:0/810717634 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28035040 con 0x55b9a7b995d0 2024-09-10T10:53:36.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 -- 172.21.15.143:0/810717634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 msgr2=0x55b9a7bc13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 0x55b9a7bc13c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005ab0 tx=0x7f8a28032c20 comp rx=0 tx=0).stop 2024-09-10T10:53:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 -- 172.21.15.143:0/810717634 shutdown_connections 2024-09-10T10:53:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7ba4370 0x55b9a7b9c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7b9ab40 0x55b9a7b9bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/810717634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7b995d0 0x55b9a7bc13c0 secure :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005ab0 tx=0x7f8a28032c20 comp rx=0 tx=0).stop 2024-09-10T10:53:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.708+0000 7f8a42bcb740 1 -- 172.21.15.143:0/810717634 >> 172.21.15.143:0/810717634 conn(0x55b9a7b62310 msgr2=0x55b9a7ba00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.711+0000 7f8a42bcb740 1 -- 172.21.15.143:0/810717634 shutdown_connections 2024-09-10T10:53:36.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.711+0000 7f8a42bcb740 1 -- 172.21.15.143:0/810717634 wait complete. 2024-09-10T10:53:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 -- start start 2024-09-10T10:53:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 Processor -- start 2024-09-10T10:53:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 0x55b9a7bbda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7bb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7bb9290 0x55b9a7bb9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381ade90 con 0x55b9a7bb9290 2024-09-10T10:53:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381ad790 con 0x55b9a7ba4370 2024-09-10T10:53:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38073620 con 0x55b9a7b9ab40 2024-09-10T10:53:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.712+0000 7f8a42bcb740 10 client.0 _ll_register_callbacks cb 0x7f8a340e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.713+0000 7f8a3e58e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 0x55b9a7bbda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.713+0000 7f8a3e58e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 0x55b9a7bbda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:49548/0 (socket says 172.21.15.143:49548) 2024-09-10T10:53:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.713+0000 7f8a3e58e640 1 -- 172.21.15.143:0/579258834 learned_addr learned my addr 172.21.15.143:0/579258834 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3ed8f640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7bb9290 0x55b9a7bb9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7bb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3e58e640 1 -- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 msgr2=0x55b9a7bb8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7bb8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3e58e640 1 -- 172.21.15.143:0/579258834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7bb9290 msgr2=0x55b9a7bb9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7bb9290 0x55b9a7bb9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3e58e640 1 -- 172.21.15.143:0/579258834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a7c2e7b0 con 0x55b9a7b9ab40 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a3e58e640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 0x55b9a7bbda70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005940 tx=0x7f8a28032c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e440c517224eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.714+0000 7f8a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:53:36.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.715+0000 7f8a21ffb640 1 -- 172.21.15.143:0/579258834 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28052060 con 0x55b9a7b9ab40 2024-09-10T10:53:36.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.715+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a28005040 con 0x55b9a7b9ab40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.715+0000 7f8a21ffb640 1 -- 172.21.15.143:0/579258834 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8a280052f0 con 0x55b9a7b9ab40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.715+0000 7f8a21ffb640 1 -- 172.21.15.143:0/579258834 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8a28059050 con 0x55b9a7b9ab40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.715+0000 7f8a42bcb740 10 client.26662 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.715+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b9a7c2f980 con 0x55b9a7b9ab40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a21ffb640 1 -- 172.21.15.143:0/579258834 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a28005040 con 0x55b9a7b9ab40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a21ffb640 1 -- 172.21.15.143:0/579258834 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7f8a28032300 con 0x55b9a7b9ab40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a42bcb740 10 client.26662 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a42bcb740 10 client.26662 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a42bcb740 20 client.26662 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7c2fe50 0x55b9a7c32270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a42bcb740 4 client.26662 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9a7b06d70 con 0x55b9a7c2fe50 2024-09-10T10:53:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.716+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7c2fe50 0x55b9a7c32270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.718+0000 7f8a3dd8d640 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7c2fe50 0x55b9a7c32270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10c5751ae400fda9 server_cookie=81a34d8589e2099c in_seq=0 out_seq=0 2024-09-10T10:53:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.718+0000 7f8a21ffb640 10 client.26662 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.718+0000 7f8a21ffb640 1 -- 172.21.15.143:0/579258834 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9a7b06d70 con 0x55b9a7c2fe50 2024-09-10T10:53:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.718+0000 7f8a21ffb640 10 client.26662 handle_command_reply: tid=0 2024-09-10T10:53:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.718+0000 7f8a42bcb740 1 client.26662 shutdown 2024-09-10T10:53:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 msgr2=0x55b9a7bbda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 0x55b9a7bbda70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a28005940 tx=0x7f8a28032c90 comp rx=0 tx=0).stop 2024-09-10T10:53:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 shutdown_connections 2024-09-10T10:53:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x55b9a7c2fe50 0x55b9a7c32270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b9a7bb9290 0x55b9a7bb9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9a7ba4370 0x55b9a7bb8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 --2- 172.21.15.143:0/579258834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b9a7b9ab40 0x55b9a7bbda70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 >> 172.21.15.143:0/579258834 conn(0x55b9a7b62310 msgr2=0x55b9a7ba2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a21ffb640 0 client.26662 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.720+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 shutdown_connections 2024-09-10T10:53:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.721+0000 7f8a42bcb740 1 -- 172.21.15.143:0/579258834 wait complete. 2024-09-10T10:53:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.721+0000 7f8a42bcb740 20 client.26662 trim_cache size 0 max 16384 2024-09-10T10:53:36.723 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a240a3010 msgr2=0x7f8a240a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a240a3010 0x7f8a240a54d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8a300092f0 tx=0x7f8a30009280 comp rx=0 tx=0).stop 2024-09-10T10:53:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 msgr2=0x7f8a381a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 0x7f8a381a4be0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9f0 tx=0x7f8a3400edf0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 shutdown_connections 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a240a3010 0x7f8a240a54d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a38112ba0 0x7f8a3819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a38074eb0 0x7f8a381a4be0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 --2- 172.21.15.143:0/301218133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a38074590 0x7f8a381a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 >> 172.21.15.143:0/301218133 conn(0x7f8a380fd8b0 msgr2=0x7f8a380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 shutdown_connections 2024-09-10T10:53:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:36.722+0000 7f8a407f2640 1 -- 172.21.15.143:0/301218133 wait complete. 2024-09-10T10:53:38.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:37 smithi143 ceph-mon[31057]: pgmap v608: 129 pgs: 129 active+clean; 195 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.0 MiB/s wr, 573 op/s 2024-09-10T10:53:38.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:37 smithi178 ceph-mon[34813]: pgmap v608: 129 pgs: 129 active+clean; 195 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.0 MiB/s wr, 573 op/s 2024-09-10T10:53:38.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:37 smithi144 ceph-mon[34230]: pgmap v608: 129 pgs: 129 active+clean; 195 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.0 MiB/s wr, 573 op/s 2024-09-10T10:53:38.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:53:38.683 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 -- 172.21.15.143:0/1200546661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102ef0 msgr2=0x7fc1b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/1200546661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102ef0 0x7fc1b010da40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009ca0 tx=0x7fc1a802a950 comp rx=0 tx=0).stop 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 -- 172.21.15.143:0/1200546661 shutdown_connections 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/1200546661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102ef0 0x7fc1b010da40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/1200546661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0102570 0x7fc1b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/1200546661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b0100e20 0x7fc1b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 -- 172.21.15.143:0/1200546661 >> 172.21.15.143:0/1200546661 conn(0x7fc1b00fc7d0 msgr2=0x7fc1b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 -- 172.21.15.143:0/1200546661 shutdown_connections 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 -- 172.21.15.143:0/1200546661 wait complete. 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.243+0000 7fc1b5f55640 1 Processor -- start 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 -- start start 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 0x7fc1b01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102570 0x7fc1b0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b0102ef0 0x7fc1b01a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0110e30 con 0x7fc1b0102570 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0110cb0 con 0x7fc1b0102ef0 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1b5f55640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0110fb0 con 0x7fc1b0100e20 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 0x7fc1b01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 0x7fc1b01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:49572/0 (socket says 172.21.15.143:49572) 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 -- 172.21.15.143:0/131950850 learned_addr learned my addr 172.21.15.143:0/131950850 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102570 0x7fc1b0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1affff640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b0102ef0 0x7fc1b01a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 -- 172.21.15.143:0/131950850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b0102ef0 msgr2=0x7fc1b01a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b0102ef0 0x7fc1b01a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 -- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102570 msgr2=0x7fc1b0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102570 0x7fc1b0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 -- 172.21.15.143:0/131950850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b01970e0 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.244+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 0x7fc1b01992f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9f0 tx=0x7fc1a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.245+0000 7fc1acff9640 1 -- 172.21.15.143:0/131950850 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000cd80 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.245+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b01973d0 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.245+0000 7fc1acff9640 1 -- 172.21.15.143:0/131950850 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a00108e0 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.245+0000 7fc1acff9640 1 -- 172.21.15.143:0/131950850 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0018920 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.245+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b0197960 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.247+0000 7fc1acff9640 1 -- 172.21.15.143:0/131950850 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc1a0010a90 con 0x7fc1b0100e20 2024-09-10T10:53:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.247+0000 7fc1b832e740 1 Processor -- start 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.247+0000 7fc1b832e740 1 -- start start 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b54c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b54ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1acff9640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1980a2b80 0x7fc1980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b559760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1b832e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56058b4b22f0 con 0x56058b557370 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1b832e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56058b459cf0 con 0x56058b54db40 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1b832e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56058b4ae2b0 con 0x56058b54c5d0 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b54ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b559760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b54ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:49588/0 (socket says 172.21.15.143:49588) 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1af7fe640 1 -- 172.21.15.143:0/4140618390 learned_addr learned my addr 172.21.15.143:0/4140618390 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:39.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b54c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1980a2b80 0x7fc1980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1acff9640 1 -- 172.21.15.143:0/131950850 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1a00cf670 con 0x7fc1b0100e20 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 -- 172.21.15.143:0/4140618390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 msgr2=0x56058b54ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b54ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 -- 172.21.15.143:0/4140618390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 msgr2=0x56058b54c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b54c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 -- 172.21.15.143:0/4140618390 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058b43e3a0 con 0x56058b557370 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1affff640 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b559760 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8005cc0 tx=0x7fc1a8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceee579747bb3d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.248+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1980a2b80 0x7fc1980a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc1b006aac0 tx=0x7fc1a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc195ffb640 1 -- 172.21.15.143:0/4140618390 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8034040 con 0x56058b557370 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc195ffb640 1 -- 172.21.15.143:0/4140618390 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a802fa60 con 0x56058b557370 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc195ffb640 1 -- 172.21.15.143:0/4140618390 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004230 con 0x56058b557370 2024-09-10T10:53:39.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 -- 172.21.15.143:0/4140618390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 msgr2=0x56058b559760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b559760 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8005cc0 tx=0x7fc1a8009360 comp rx=0 tx=0).stop 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 -- 172.21.15.143:0/4140618390 shutdown_connections 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b559760 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b54ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 --2- 172.21.15.143:0/4140618390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b54c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 -- 172.21.15.143:0/4140618390 >> 172.21.15.143:0/4140618390 conn(0x56058b515310 msgr2=0x56058b548c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.282 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 -- 172.21.15.143:0/4140618390 shutdown_connections 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.249+0000 7fc1b832e740 1 -- 172.21.15.143:0/4140618390 wait complete. 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 -- start start 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 Processor -- start 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b556dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b554080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b554810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56058b55a010 con 0x56058b557370 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56058b46d060 con 0x56058b54db40 2024-09-10T10:53:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56058b55a190 con 0x56058b54c5d0 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1b832e740 10 client.0 _ll_register_callbacks cb 0x56058b1c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b554810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b556dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b554810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59052/0 (socket says 172.21.15.143:59052) 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1affff640 1 -- 172.21.15.143:0/1230389147 learned_addr learned my addr 172.21.15.143:0/1230389147 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.250+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b554080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1aeffd640 1 -- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 msgr2=0x56058b556dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b556dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1aeffd640 1 -- 172.21.15.143:0/1230389147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 msgr2=0x56058b554810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b554810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1aeffd640 1 -- 172.21.15.143:0/1230389147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56058b2233c0 con 0x56058b54db40 2024-09-10T10:53:39.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b556dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1affff640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b554810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b554080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40044f0 tx=0x7fc1a4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c855aa37bfc15b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc18effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc18effd640 1 -- 172.21.15.143:0/1230389147 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a401e830 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a802fa60 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc18effd640 1 -- 172.21.15.143:0/1230389147 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1a401e9d0 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc18effd640 1 -- 172.21.15.143:0/1230389147 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc1a401ebd0 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1b832e740 10 client.26567 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56058b565be0 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.251+0000 7fc18effd640 1 -- 172.21.15.143:0/1230389147 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a802fa60 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc18effd640 1 -- 172.21.15.143:0/1230389147 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7fc1a401d3b0 con 0x56058b54db40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1b832e740 10 client.26567 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1b832e740 10 client.26567 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:53:39.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1b832e740 20 client.26567 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1b832e740 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b567170 0x56058b569590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1b832e740 4 client.26567 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56058b4bf860 con 0x56058b567170 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b567170 0x56058b569590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc18effd640 10 client.26567 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.252+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b567170 0x56058b569590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=790352a2ae41107e server_cookie=ee42b0987fa8dc1d in_seq=0 out_seq=0 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc18effd640 1 -- 172.21.15.143:0/1230389147 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56058b4bf860 con 0x56058b567170 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc18effd640 10 client.26567 handle_command_reply: tid=0 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 client.26567 shutdown 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 msgr2=0x56058b554080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b554080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40044f0 tx=0x7fc1a4004730 comp rx=0 tx=0).stop 2024-09-10T10:53:39.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 shutdown_connections 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b567170 0x56058b569590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b554810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b554080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 --2- 172.21.15.143:0/1230389147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54c5d0 0x56058b556dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 >> 172.21.15.143:0/1230389147 conn(0x56058b515310 msgr2=0x56058b54e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc18effd640 0 client.26567 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 shutdown_connections 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 1 -- 172.21.15.143:0/1230389147 wait complete. 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.253+0000 7fc1b832e740 20 client.26567 trim_cache size 0 max 16384 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 Processor -- start 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 -- start start 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54c5d0 0x56058b54d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54db40 0x56058b54f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b54f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b0d90 con 0x56058b557370 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b29f0 con 0x56058b54c5d0 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1b832e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b1460 con 0x56058b54db40 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b54f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b54f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59058/0 (socket says 172.21.15.143:59058) 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 -- 172.21.15.143:0/142121523 learned_addr learned my addr 172.21.15.143:0/142121523 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54db40 0x56058b54f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54c5d0 0x56058b54d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 -- 172.21.15.143:0/142121523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54db40 msgr2=0x56058b54f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54db40 0x56058b54f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 -- 172.21.15.143:0/142121523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54c5d0 msgr2=0x56058b54d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54c5d0 0x56058b54d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.267+0000 7fc1affff640 1 -- 172.21.15.143:0/142121523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058b5659f0 con 0x56058b557370 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1affff640 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b54f800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009ca0 tx=0x7fc1a802b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb474bb59e60ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1957fa640 1 -- 172.21.15.143:0/142121523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8034040 con 0x56058b557370 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1957fa640 1 -- 172.21.15.143:0/142121523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a8003e20 con 0x56058b557370 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1957fa640 1 -- 172.21.15.143:0/142121523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8030440 con 0x56058b557370 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 -- 172.21.15.143:0/142121523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 msgr2=0x56058b54f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b54f800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009ca0 tx=0x7fc1a802b0d0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 -- 172.21.15.143:0/142121523 shutdown_connections 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b557370 0x56058b54f800 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b54db40 0x56058b54f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 --2- 172.21.15.143:0/142121523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54c5d0 0x56058b54d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 -- 172.21.15.143:0/142121523 >> 172.21.15.143:0/142121523 conn(0x56058b515310 msgr2=0x56058b543a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.268+0000 7fc1b832e740 1 -- 172.21.15.143:0/142121523 shutdown_connections 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 -- 172.21.15.143:0/142121523 wait complete. 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 -- start start 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 Processor -- start 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 0x56058b55a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b55a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b557370 0x56058b55ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b2260 con 0x56058b54c5d0 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01b1b60 con 0x56058b54db40 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01aed70 con 0x56058b557370 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1b832e740 10 client.0 _ll_register_callbacks cb 0x7fc1a4011be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 0x56058b55a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 0x56058b55a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59062/0 (socket says 172.21.15.143:59062) 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1af7fe640 1 -- 172.21.15.143:0/3004839051 learned_addr learned my addr 172.21.15.143:0/3004839051 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.269+0000 7fc1affff640 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b557370 0x56058b55ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1af7fe640 1 -- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b557370 msgr2=0x56058b55ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b557370 0x56058b55ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1af7fe640 1 -- 172.21.15.143:0/3004839051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 msgr2=0x56058b55a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b55a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1af7fe640 1 -- 172.21.15.143:0/3004839051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56058b5cfe50 con 0x56058b54c5d0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1af7fe640 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 0x56058b55a3e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c6a0 tx=0x7fc1a00cf870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8359b55adfbe0def server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc18effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc18effd640 1 -- 172.21.15.143:0/3004839051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a009c0c0 con 0x56058b54c5d0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a8003e20 con 0x56058b54c5d0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc18effd640 1 -- 172.21.15.143:0/3004839051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1a0099f20 con 0x56058b54c5d0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc18effd640 1 -- 172.21.15.143:0/3004839051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc1a00b9050 con 0x56058b54c5d0 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1b832e740 10 client.17292 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56058b5d1190 con 0x56058b54c5d0 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.270+0000 7fc18effd640 1 -- 172.21.15.143:0/3004839051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1a8003e20 con 0x56058b54c5d0 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc18effd640 1 -- 172.21.15.143:0/3004839051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7fc1a009b070 con 0x56058b54c5d0 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1b832e740 10 client.17292 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1b832e740 10 client.17292 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1b832e740 20 client.17292 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1b832e740 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b5d1960 0x56058b5d3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1b832e740 4 client.17292 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56058b4b9d70 con 0x56058b5d1960 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b5d1960 0x56058b5d3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc1aeffd640 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b5d1960 0x56058b5d3dc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e22e3dc70b21094 server_cookie=3a2bc77a37bc17e1 in_seq=0 out_seq=0 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.271+0000 7fc18effd640 10 client.17292 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc18effd640 1 -- 172.21.15.143:0/3004839051 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56058b4b9d70 con 0x56058b5d1960 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc18effd640 10 client.17292 handle_command_reply: tid=0 2024-09-10T10:53:39.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 client.17292 shutdown 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 msgr2=0x56058b55a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 0x56058b55a3e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c6a0 tx=0x7fc1a00cf870 comp rx=0 tx=0).stop 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 shutdown_connections 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x56058b5d1960 0x56058b5d3dc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56058b557370 0x56058b55ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56058b54db40 0x56058b55a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc18effd640 0 client.17292 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 --2- 172.21.15.143:0/3004839051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56058b54c5d0 0x56058b55a3e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 >> 172.21.15.143:0/3004839051 conn(0x56058b515310 msgr2=0x56058b54f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 shutdown_connections 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 1 -- 172.21.15.143:0/3004839051 wait complete. 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b832e740 20 client.17292 trim_cache size 0 max 16384 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1980a2b80 msgr2=0x7fc1980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1980a2b80 0x7fc1980a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc1b006aac0 tx=0x7fc1a4005f90 comp rx=0 tx=0).stop 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 msgr2=0x7fc1b01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 0x7fc1b01992f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9f0 tx=0x7fc1a000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 shutdown_connections 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1980a2b80 0x7fc1980a5040 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b0102ef0 0x7fc1b01a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1b0102570 0x7fc1b0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.272+0000 7fc1b5f55640 1 --2- 172.21.15.143:0/131950850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1b0100e20 0x7fc1b01992f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.273+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 >> 172.21.15.143:0/131950850 conn(0x7fc1b00fc7d0 msgr2=0x7fc1b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.273+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 shutdown_connections 2024-09-10T10:53:39.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:39.273+0000 7fc1b5f55640 1 -- 172.21.15.143:0/131950850 wait complete. 2024-09-10T10:53:40.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:39 smithi143 ceph-mon[31057]: pgmap v609: 129 pgs: 129 active+clean; 187 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 833 KiB/s wr, 496 op/s 2024-09-10T10:53:40.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:53:40.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:39 smithi178 ceph-mon[34813]: pgmap v609: 129 pgs: 129 active+clean; 187 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 833 KiB/s wr, 496 op/s 2024-09-10T10:53:40.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:39 smithi144 ceph-mon[34230]: pgmap v609: 129 pgs: 129 active+clean; 187 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 833 KiB/s wr, 496 op/s 2024-09-10T10:53:40.360 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:40.714 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-10T10:53:40.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T10:53:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 -- 172.21.15.143:0/1071623778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 msgr2=0x7fbdc4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 --2- 172.21.15.143:0/1071623778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4101200 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402a480 comp rx=0 tx=0).stop 2024-09-10T10:53:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 -- 172.21.15.143:0/1071623778 shutdown_connections 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 --2- 172.21.15.143:0/1071623778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 0x7fbdc410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 --2- 172.21.15.143:0/1071623778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc4102570 0x7fbdc41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 --2- 172.21.15.143:0/1071623778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4101200 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 -- 172.21.15.143:0/1071623778 >> 172.21.15.143:0/1071623778 conn(0x7fbdc40fc7d0 msgr2=0x7fbdc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 -- 172.21.15.143:0/1071623778 shutdown_connections 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.836+0000 7fbdcc263640 1 -- 172.21.15.143:0/1071623778 wait complete. 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 Processor -- start 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 -- start start 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc4102570 0x7fbdc4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 0x7fbdc41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4110e30 con 0x7fbdc4100e20 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4110cb0 con 0x7fbdc4102ef0 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdcc263640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4110fb0 con 0x7fbdc4102570 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59068/0 (socket says 172.21.15.143:59068) 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2774427416 learned_addr learned my addr 172.21.15.143:0/2774427416 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc4102570 0x7fbdc4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdca800640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 0x7fbdc41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2774427416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc4102570 msgr2=0x7fbdc4103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc4102570 0x7fbdc4103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2774427416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 msgr2=0x7fbdc41041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 0x7fbdc41041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2774427416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41a6990 con 0x7fbdc4100e20 2024-09-10T10:53:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.837+0000 7fbdca800640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 0x7fbdc41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.838+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4103750 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006010 tx=0x7fbdb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.838+0000 7fbdbaffd640 1 -- 172.21.15.143:0/2774427416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb403f030 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.838+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41a6c20 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.838+0000 7fbdbaffd640 1 -- 172.21.15.143:0/2774427416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdb4003aa0 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.838+0000 7fbdbaffd640 1 -- 172.21.15.143:0/2774427416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4035070 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.838+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41a7150 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.840+0000 7fbdbaffd640 1 -- 172.21.15.143:0/2774427416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbdb4042030 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.841+0000 7fbdbaffd640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbdb00a2b80 0x7fbdb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.841+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbdb00a2b80 0x7fbdb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.841+0000 7fbdbaffd640 1 -- 172.21.15.143:0/2774427416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbdb40efa70 con 0x7fbdc4100e20 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.841+0000 7fbdce63c740 1 Processor -- start 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.841+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbdb00a2b80 0x7fbdb00a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41051e0 tx=0x7fbdc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.841+0000 7fbdce63c740 1 -- start start 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdce63c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55603a4342f0 con 0x55603a4d8350 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdce63c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55603a3dbcf0 con 0x55603a4e1b80 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdce63c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55603a4302b0 con 0x55603a4d6de0 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4d68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59070/0 (socket says 172.21.15.143:59070) 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1572401626 learned_addr learned my addr 172.21.15.143:0/1572401626 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdca800640 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1572401626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 msgr2=0x55603a4d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1572401626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 msgr2=0x55603a4e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4e3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1572401626 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603a3c03a0 con 0x55603a4d8350 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4d68a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4032a80 tx=0x7fbdb40bb930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0cb06037f0b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdaffff640 1 -- 172.21.15.143:0/1572401626 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4049040 con 0x55603a4d8350 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdaffff640 1 -- 172.21.15.143:0/1572401626 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdb40c02c0 con 0x55603a4d8350 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.842+0000 7fbdaffff640 1 -- 172.21.15.143:0/1572401626 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4003c50 con 0x55603a4d8350 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 -- 172.21.15.143:0/1572401626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 msgr2=0x55603a4d68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4d68a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4032a80 tx=0x7fbdb40bb930 comp rx=0 tx=0).stop 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 -- 172.21.15.143:0/1572401626 shutdown_connections 2024-09-10T10:53:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1572401626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4d68a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 -- 172.21.15.143:0/1572401626 >> 172.21.15.143:0/1572401626 conn(0x55603a497310 msgr2=0x55603a4d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 -- 172.21.15.143:0/1572401626 shutdown_connections 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.843+0000 7fbdce63c740 1 -- 172.21.15.143:0/1572401626 wait complete. 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 -- start start 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 Processor -- start 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a4ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55603a4e4640 con 0x55603a4e1b80 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55603a3ef060 con 0x55603a4d8350 2024-09-10T10:53:40.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdce63c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55603a4e47c0 con 0x55603a4d6de0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a4ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.844+0000 7fbdca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a4ded40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59086/0 (socket says 172.21.15.143:59086) 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdca800640 1 -- 172.21.15.143:0/1551743192 learned_addr learned my addr 172.21.15.143:0/1551743192 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdce63c740 10 client.0 _ll_register_callbacks cb 0x55603a1428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1551743192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 msgr2=0x55603a4de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4de7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1551743192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 msgr2=0x55603a4ded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a4ded40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc9fff640 1 -- 172.21.15.143:0/1551743192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603a1a53c0 con 0x55603a4d6de0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e1510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbdb403d040 tx=0x7fbdb40b8580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ae63f57d8f91727 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdad7fa640 1 -- 172.21.15.143:0/1551743192 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4049030 con 0x55603a4d6de0 2024-09-10T10:53:40.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdad7fa640 1 -- 172.21.15.143:0/1551743192 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbdb4003c50 con 0x55603a4d6de0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb40c02c0 con 0x55603a4d6de0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdad7fa640 1 -- 172.21.15.143:0/1551743192 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbdb40c46f0 con 0x55603a4d6de0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdce63c740 10 client.26689 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55603a4e4640 con 0x55603a4d6de0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.845+0000 7fbdad7fa640 1 -- 172.21.15.143:0/1551743192 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdb40c02c0 con 0x55603a4d6de0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdad7fa640 1 -- 172.21.15.143:0/1551743192 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7fbdb4003710 con 0x55603a4d6de0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdce63c740 10 client.26689 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdce63c740 10 client.26689 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdce63c740 20 client.26689 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a4f1190 0x55603a4f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdce63c740 4 client.26689 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55603a441860 con 0x55603a4f1190 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a4f1190 0x55603a4f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a4f1190 0x55603a4f35d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=620bdcc66968d00c server_cookie=a263b53b76928976 in_seq=0 out_seq=0 2024-09-10T10:53:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.846+0000 7fbdad7fa640 10 client.26689 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdad7fa640 1 -- 172.21.15.143:0/1551743192 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55603a441860 con 0x55603a4f1190 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdad7fa640 10 client.26689 handle_command_reply: tid=0 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 client.26689 shutdown 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 msgr2=0x55603a4e1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e1510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbdb403d040 tx=0x7fbdb40b8580 comp rx=0 tx=0).stop 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 shutdown_connections 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a4f1190 0x55603a4f35d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a4ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdad7fa640 0 client.26689 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4de7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 --2- 172.21.15.143:0/1551743192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e1510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 >> 172.21.15.143:0/1551743192 conn(0x55603a497310 msgr2=0x55603a4d9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 shutdown_connections 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 1 -- 172.21.15.143:0/1551743192 wait complete. 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.847+0000 7fbdce63c740 20 client.26689 trim_cache size 0 max 16384 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 Processor -- start 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 -- start start 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4ed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4de080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ac960 con 0x55603a4d8350 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ae5c0 con 0x55603a4e1b80 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.860+0000 7fbdce63c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ad030 con 0x55603a4d6de0 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc97fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4de080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc97fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4de080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59088/0 (socket says 172.21.15.143:59088) 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc97fe640 1 -- 172.21.15.143:0/2670290192 learned_addr learned my addr 172.21.15.143:0/2670290192 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4ed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdca800640 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2670290192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 msgr2=0x55603a4dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2670290192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 msgr2=0x55603a4de080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4de080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 -- 172.21.15.143:0/2670290192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603a4da340 con 0x55603a4d6de0 2024-09-10T10:53:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdca800640 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4ed810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40bb450 tx=0x7fbdb402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aceccd6129b2232 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdaf7fe640 1 -- 172.21.15.143:0/2670290192 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4049070 con 0x55603a4d6de0 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdaf7fe640 1 -- 172.21.15.143:0/2670290192 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdb40b85f0 con 0x55603a4d6de0 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.861+0000 7fbdaf7fe640 1 -- 172.21.15.143:0/2670290192 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40bb660 con 0x55603a4d6de0 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 -- 172.21.15.143:0/2670290192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 msgr2=0x55603a4ed810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4ed810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40bb450 tx=0x7fbdb402a480 comp rx=0 tx=0).stop 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 -- 172.21.15.143:0/2670290192 shutdown_connections 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4e1b80 0x55603a4dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4d8350 0x55603a4de080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 --2- 172.21.15.143:0/2670290192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4ed810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 -- 172.21.15.143:0/2670290192 >> 172.21.15.143:0/2670290192 conn(0x55603a497310 msgr2=0x55603a4dcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 -- 172.21.15.143:0/2670290192 shutdown_connections 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 -- 172.21.15.143:0/2670290192 wait complete. 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.862+0000 7fbdce63c740 1 -- start start 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 Processor -- start 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a553e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ade30 con 0x55603a4e1b80 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41ad730 con 0x55603a4d8350 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41aa9a0 con 0x55603a4d6de0 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdce63c740 10 client.0 _ll_register_callbacks cb 0x7fbdbc008450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a553e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a553e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59104/0 (socket says 172.21.15.143:59104) 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 -- 172.21.15.143:0/648373907 learned_addr learned my addr 172.21.15.143:0/648373907 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdc9fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:49646/0 (socket says 172.21.15.143:49646) 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 -- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 msgr2=0x55603a4e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 -- 172.21.15.143:0/648373907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 msgr2=0x55603a4e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdca800640 1 -- 172.21.15.143:0/648373907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603a554520 con 0x55603a4e1b80 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.863+0000 7fbdc97fe640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdca800640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a553e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00fa40 tx=0x7fbdbc00ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a873ebf80597c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdad7fa640 1 -- 172.21.15.143:0/648373907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc01f060 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdad7fa640 1 -- 172.21.15.143:0/648373907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbdbc00ee20 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb40b85f0 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdad7fa640 1 -- 172.21.15.143:0/648373907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbdbc01a3f0 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 10 client.17310 fetch_fsmap learned FSMap version 40 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55603a55b950 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdad7fa640 1 -- 172.21.15.143:0/648373907 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbdb40b85f0 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdad7fa640 1 -- 172.21.15.143:0/648373907 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7fbdbc019070 con 0x55603a4e1b80 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 10 client.17310 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 10 client.17310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 20 client.17310 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.864+0000 7fbdce63c740 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a55c100 0x55603a55e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 4 client.17310 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55603a43bd70 con 0x55603a55c100 2024-09-10T10:53:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a55c100 0x55603a55e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdad7fa640 10 client.17310 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdc9fff640 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a55c100 0x55603a55e520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1efa4019035f73c server_cookie=3f639a98cae7bf8c in_seq=0 out_seq=0 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdad7fa640 1 -- 172.21.15.143:0/648373907 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55603a43bd70 con 0x55603a55c100 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdad7fa640 10 client.17310 handle_command_reply: tid=0 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 client.17310 shutdown 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 msgr2=0x55603a553e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a553e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00fa40 tx=0x7fbdbc00ff00 comp rx=0 tx=0).stop 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 shutdown_connections 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x55603a55c100 0x55603a55e520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55603a4e1b80 0x55603a553e80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55603a4d8350 0x55603a4e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdad7fa640 0 client.17310 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:40.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 --2- 172.21.15.143:0/648373907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55603a4d6de0 0x55603a4e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.865+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 >> 172.21.15.143:0/648373907 conn(0x55603a497310 msgr2=0x55603a4e0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 shutdown_connections 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdce63c740 1 -- 172.21.15.143:0/648373907 wait complete. 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdce63c740 20 client.17310 trim_cache size 0 max 16384 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbdb00a2b80 msgr2=0x7fbdb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbdb00a2b80 0x7fbdb00a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41051e0 tx=0x7fbdc0005f90 comp rx=0 tx=0).stop 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 msgr2=0x7fbdc4103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4103750 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006010 tx=0x7fbdb4004270 comp rx=0 tx=0).stop 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 shutdown_connections 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fbdb00a2b80 0x7fbdb00a5040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc4102ef0 0x7fbdc41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc4102570 0x7fbdc4103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 --2- 172.21.15.143:0/2774427416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdc4100e20 0x7fbdc4103750 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 >> 172.21.15.143:0/2774427416 conn(0x7fbdc40fc7d0 msgr2=0x7fbdc406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.866+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 shutdown_connections 2024-09-10T10:53:40.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:40.867+0000 7fbdcc263640 1 -- 172.21.15.143:0/2774427416 wait complete. 2024-09-10T10:53:41.023 DEBUG:teuthology.parallel:result is None 2024-09-10T10:53:41.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:53:41.060 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:53:41.060 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T10:53:41.071 INFO:tasks.quiescer:joining Quiescers 2024-09-10T10:53:41.071 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T10:53:41.072 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:53:42.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:53:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:42.540 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:42 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:42.540 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:42 smithi143 ceph-mon[31057]: pgmap v610: 129 pgs: 129 active+clean; 187 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 499 op/s 2024-09-10T10:53:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:42 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:42.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:42 smithi178 ceph-mon[34813]: pgmap v610: 129 pgs: 129 active+clean; 187 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 499 op/s 2024-09-10T10:53:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:42 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:42.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:42 smithi144 ceph-mon[34230]: pgmap v610: 129 pgs: 129 active+clean; 187 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 499 op/s 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2108614504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 msgr2=0x7f2c38112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2108614504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c38112650 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009900 tx=0x7f2c2802a480 comp rx=0 tx=0).stop 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2108614504 shutdown_connections 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2108614504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 0x7f2c38114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2108614504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c38112650 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2108614504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c38074590 0x7f2c38074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2108614504 >> 172.21.15.143:0/2108614504 conn(0x7f2c380fd8c0 msgr2=0x7f2c380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2108614504 shutdown_connections 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.914+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2108614504 wait complete. 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 Processor -- start 2024-09-10T10:53:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 -- start start 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c38074590 0x7f2c381a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c381a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 0x7f2c3819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2c381184a0 con 0x7f2c38074eb0 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c38118320 con 0x7f2c38112b90 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3fb6c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c38118620 con 0x7f2c38074590 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.915+0000 7f2c3d107640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c381a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d908640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c38074590 0x7f2c381a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c381a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59110/0 (socket says 172.21.15.143:59110) 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 -- 172.21.15.143:0/2365594275 learned_addr learned my addr 172.21.15.143:0/2365594275 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3e109640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 0x7f2c3819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 -- 172.21.15.143:0/2365594275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c38074590 msgr2=0x7f2c381a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c38074590 0x7f2c381a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 -- 172.21.15.143:0/2365594275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 msgr2=0x7f2c3819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 0x7f2c3819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 -- 172.21.15.143:0/2365594275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3819f040 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3e109640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 0x7f2c3819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3d107640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c381a4c90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2c28005940 tx=0x7f2c28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2803f030 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c28003aa0 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28035070 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3819f2d0 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.916+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c381ab620 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.917+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c38074970 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.918+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2c28042030 con 0x7f2c38074eb0 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.919+0000 7f2c36ffd640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2c240a2c00 0x7f2c240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.919+0000 7f2c3d908640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2c240a2c00 0x7f2c240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.919+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2c280efb10 con 0x7f2c38074eb0 2024-09-10T10:53:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.920+0000 7f2c3d908640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2c240a2c00 0x7f2c240a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c3819fa50 tx=0x7f2c3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:42.924+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c280bc710 con 0x7f2c38074eb0 2024-09-10T10:53:43.058 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:43.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.133+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c38105030 con 0x7f2c38074eb0 2024-09-10T10:53:43.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.133+0000 7f2c36ffd640 1 -- 172.21.15.143:0/2365594275 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12012 (secure 0 0 0) 0x7f2c280bc010 con 0x7f2c38074eb0 2024-09-10T10:53:43.136 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:53:43.136 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":41,"btime":"2024-09-10T10:53:41:467528+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:53:41.417383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24475,24481,24485,24502]},"id":1}]} 2024-09-10T10:53:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 41 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2c240a2c00 msgr2=0x7f2c240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2c240a2c00 0x7f2c240a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c3819fa50 tx=0x7f2c3000b390 comp rx=0 tx=0).stop 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 msgr2=0x7f2c381a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c381a4c90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2c28005940 tx=0x7f2c28004270 comp rx=0 tx=0).stop 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 shutdown_connections 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2c240a2c00 0x7f2c240a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c38112b90 0x7f2c3819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c38074eb0 0x7f2c381a4c90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 --2- 172.21.15.143:0/2365594275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c38074590 0x7f2c381a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 >> 172.21.15.143:0/2365594275 conn(0x7f2c380fd8c0 msgr2=0x7f2c380ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 shutdown_connections 2024-09-10T10:53:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:43.136+0000 7f2c3fb6c640 1 -- 172.21.15.143:0/2365594275 wait complete. 2024-09-10T10:53:43.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:43 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2365594275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:43.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:43 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2365594275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:43.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:43 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2365594275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 -- 172.21.15.143:0/3705254997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48106f20 msgr2=0x7f9b4810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3705254997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48106f20 0x7f9b4810eaa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 -- 172.21.15.143:0/3705254997 shutdown_connections 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3705254997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48106f20 0x7f9b4810eaa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3705254997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b481027a0 0x7f9b481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3705254997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b48101df0 0x7f9b481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 -- 172.21.15.143:0/3705254997 >> 172.21.15.143:0/3705254997 conn(0x7f9b480fd8b0 msgr2=0x7f9b480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 -- 172.21.15.143:0/3705254997 shutdown_connections 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.365+0000 7f9b4e151640 1 -- 172.21.15.143:0/3705254997 wait complete. 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 Processor -- start 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 -- start start 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 0x7f9b481982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b481027a0 0x7f9b481987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b48106f20 0x7f9b4819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48111e30 con 0x7f9b48101df0 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48111cb0 con 0x7f9b481027a0 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b4e151640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48111fb0 con 0x7f9b48106f20 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 0x7f9b481982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 0x7f9b481982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59118/0 (socket says 172.21.15.143:59118) 2024-09-10T10:53:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b477fe640 1 -- 172.21.15.143:0/3722140461 learned_addr learned my addr 172.21.15.143:0/3722140461 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.366+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b481027a0 0x7f9b481987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b48106f20 0x7f9b4819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b477fe640 1 -- 172.21.15.143:0/3722140461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b48106f20 msgr2=0x7f9b4819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b477fe640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b48106f20 0x7f9b4819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b477fe640 1 -- 172.21.15.143:0/3722140461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b481027a0 msgr2=0x7f9b481987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b477fe640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b481027a0 0x7f9b481987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b477fe640 1 -- 172.21.15.143:0/3722140461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4819d260 con 0x7f9b48101df0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b48106f20 0x7f9b4819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b477fe640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 0x7f9b481982b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800d950 tx=0x7f9b3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b44ff9640 1 -- 172.21.15.143:0/3722140461 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38017070 con 0x7f9b48101df0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b44ff9640 1 -- 172.21.15.143:0/3722140461 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b38009c30 con 0x7f9b48101df0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b44ff9640 1 -- 172.21.15.143:0/3722140461 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800f620 con 0x7f9b48101df0 2024-09-10T10:53:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4819d550 con 0x7f9b48101df0 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.367+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a4e30 con 0x7f9b48101df0 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.369+0000 7f9b44ff9640 1 -- 172.21.15.143:0/3722140461 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9b380109a0 con 0x7f9b48101df0 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 Processor -- start 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 -- start start 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556d1f9e22f0 con 0x556d1fa7db40 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d1f989cf0 con 0x556d1fa87370 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b5052a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556d1f9de2b0 con 0x556d1fa7c5d0 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b44ff9640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9b300a2c00 0x7f9b300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59122/0 (socket says 172.21.15.143:59122) 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 -- 172.21.15.143:0/3397013512 learned_addr learned my addr 172.21.15.143:0/3397013512 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b44ff9640 1 -- 172.21.15.143:0/3722140461 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9b380cefc0 con 0x7f9b48101df0 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9b300a2c00 0x7f9b300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b477fe640 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa89760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 -- 172.21.15.143:0/3397013512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 msgr2=0x556d1fa7ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa7ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 -- 172.21.15.143:0/3397013512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 msgr2=0x556d1fa89760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.370+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa89760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b47fff640 1 -- 172.21.15.143:0/3397013512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1f96e3a0 con 0x556d1fa7db40 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7c090 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9b40005e30 tx=0x7f9b40032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c09fc10570468e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b47fff640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9b300a2c00 0x7f9b300a50c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009a60 tx=0x7f9b40043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b2dffb640 1 -- 172.21.15.143:0/3397013512 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4004f070 con 0x556d1fa7db40 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b2dffb640 1 -- 172.21.15.143:0/3397013512 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b40036040 con 0x556d1fa7db40 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.371+0000 7f9b2dffb640 1 -- 172.21.15.143:0/3397013512 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400307c0 con 0x556d1fa7db40 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 -- 172.21.15.143:0/3397013512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 msgr2=0x556d1fa7c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7c090 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9b40005e30 tx=0x7f9b40032c90 comp rx=0 tx=0).stop 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 -- 172.21.15.143:0/3397013512 shutdown_connections 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 --2- 172.21.15.143:0/3397013512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7c090 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 -- 172.21.15.143:0/3397013512 >> 172.21.15.143:0/3397013512 conn(0x556d1fa45310 msgr2=0x556d1fa78c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 -- 172.21.15.143:0/3397013512 shutdown_connections 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 -- 172.21.15.143:0/3397013512 wait complete. 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 -- start start 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.372+0000 7f9b5052a740 1 Processor -- start 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7c5d0 0x556d1fa840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7db40 0x556d1fa84630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 0x556d1fa84b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556d1fa8a010 con 0x556d1fa87370 2024-09-10T10:53:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d1f99d060 con 0x556d1fa7c5d0 2024-09-10T10:53:44.409 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:44.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556d1fa8a190 con 0x556d1fa7db40 2024-09-10T10:53:44.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 10 client.0 _ll_register_callbacks cb 0x556d1f6f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 0x556d1fa84b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7c5d0 0x556d1fa840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 0x556d1fa84b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59128/0 (socket says 172.21.15.143:59128) 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 -- 172.21.15.143:0/1959317668 learned_addr learned my addr 172.21.15.143:0/1959317668 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7db40 0x556d1fa84630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 -- 172.21.15.143:0/1959317668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7db40 msgr2=0x556d1fa84630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7db40 0x556d1fa84630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 -- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7c5d0 msgr2=0x556d1fa840f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7c5d0 0x556d1fa840f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 -- 172.21.15.143:0/1959317668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1f7533c0 con 0x556d1fa87370 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b47fff640 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 0x556d1fa84b70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9b4002a900 tx=0x7f9b40004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a680c9f40c592b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:44.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b26ffd640 1 -- 172.21.15.143:0/1959317668 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40058020 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b26ffd640 1 -- 172.21.15.143:0/1959317668 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9b400307c0 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.373+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b40036040 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b26ffd640 1 -- 172.21.15.143:0/1959317668 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9b40063050 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 10 client.17328 fetch_fsmap learned FSMap version 41 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556d1fa8a010 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b26ffd640 1 -- 172.21.15.143:0/1959317668 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b40036040 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b26ffd640 1 -- 172.21.15.143:0/1959317668 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 41) ==== 8406+0+0 (secure 0 0 0) 0x7f9b4004b8a0 con 0x556d1fa87370 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 10 client.17328 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 10 client.17328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 20 client.17328 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fa96d20 0x556d1fa99110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 4 client.17328 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d1f9ef860 con 0x556d1fa96d20 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.374+0000 7f9b477fe640 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fa96d20 0x556d1fa99110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.375+0000 7f9b477fe640 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fa96d20 0x556d1fa99110 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c95ebc25afafc49 server_cookie=280c2de3cf015aa3 in_seq=0 out_seq=0 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.375+0000 7f9b26ffd640 10 client.17328 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b26ffd640 1 -- 172.21.15.143:0/1959317668 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d1f9ef860 con 0x556d1fa96d20 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b26ffd640 10 client.17328 handle_command_reply: tid=0 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 client.17328 shutdown 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 msgr2=0x556d1fa84b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 0x556d1fa84b70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9b4002a900 tx=0x7f9b40004640 comp rx=0 tx=0).stop 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 shutdown_connections 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa87370 0x556d1fa84b70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7db40 0x556d1fa84630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fa96d20 0x556d1fa99110 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 --2- 172.21.15.143:0/1959317668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7c5d0 0x556d1fa840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b26ffd640 0 client.17328 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 >> 172.21.15.143:0/1959317668 conn(0x556d1fa45310 msgr2=0x556d1fa7e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 shutdown_connections 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 1 -- 172.21.15.143:0/1959317668 wait complete. 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.376+0000 7f9b5052a740 20 client.17328 trim_cache size 0 max 16384 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 Processor -- start 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 -- start start 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa7f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481aeb00 con 0x556d1fa7db40 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b0760 con 0x556d1fa87370 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b5052a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481af1d0 con 0x556d1fa7c5d0 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59140/0 (socket says 172.21.15.143:59140) 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 -- 172.21.15.143:0/2648033395 learned_addr learned my addr 172.21.15.143:0/2648033395 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b477fe640 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 -- 172.21.15.143:0/2648033395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 msgr2=0x556d1fa77fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa77fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b47fff640 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa7f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 -- 172.21.15.143:0/2648033395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 msgr2=0x556d1fa7f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa7f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 -- 172.21.15.143:0/2648033395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1fa99770 con 0x556d1fa7db40 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b47fff640 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.389+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7f120 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c004550 tx=0x7f9b3c00ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f20c6c82762f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b2d7fa640 1 -- 172.21.15.143:0/2648033395 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0038d0 con 0x556d1fa7db40 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b2d7fa640 1 -- 172.21.15.143:0/2648033395 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b3c005020 con 0x556d1fa7db40 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b2d7fa640 1 -- 172.21.15.143:0/2648033395 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c004b70 con 0x556d1fa7db40 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 -- 172.21.15.143:0/2648033395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 msgr2=0x556d1fa7f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7f120 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c004550 tx=0x7f9b3c00ba60 comp rx=0 tx=0).stop 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 -- 172.21.15.143:0/2648033395 shutdown_connections 2024-09-10T10:53:44.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa87370 0x556d1fa7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7db40 0x556d1fa7f120 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2648033395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa7c5d0 0x556d1fa77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 -- 172.21.15.143:0/2648033395 >> 172.21.15.143:0/2648033395 conn(0x556d1fa45310 msgr2=0x556d1fa83110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 -- 172.21.15.143:0/2648033395 shutdown_connections 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.390+0000 7f9b5052a740 1 -- 172.21.15.143:0/2648033395 wait complete. 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 -- start start 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 Processor -- start 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 0x556d1fa8fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7db40 0x556d1fa80d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa87370 0x556d1fa812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481affd0 con 0x556d1fa7c5d0 2024-09-10T10:53:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481af8d0 con 0x556d1fa7db40 2024-09-10T10:53:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481acae0 con 0x556d1fa87370 2024-09-10T10:53:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b5052a740 10 client.0 _ll_register_callbacks cb 0x7f9b3800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:44.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 0x556d1fa8fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 0x556d1fa8fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59156/0 (socket says 172.21.15.143:59156) 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 -- 172.21.15.143:0/2744704988 learned_addr learned my addr 172.21.15.143:0/2744704988 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7db40 0x556d1fa80d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b47fff640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa87370 0x556d1fa812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 -- 172.21.15.143:0/2744704988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa87370 msgr2=0x556d1fa812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa87370 0x556d1fa812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 -- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7db40 msgr2=0x556d1fa80d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7db40 0x556d1fa80d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.391+0000 7f9b477fe640 1 -- 172.21.15.143:0/2744704988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1fa819b0 con 0x556d1fa7c5d0 2024-09-10T10:53:44.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b477fe640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 0x556d1fa8fa60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9b38097e30 tx=0x7f9b380ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d37b673f4787927 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b26ffd640 1 -- 172.21.15.143:0/2744704988 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380b3020 con 0x556d1fa7c5d0 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b26ffd640 1 -- 172.21.15.143:0/2744704988 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f9b380cc810 con 0x556d1fa7c5d0 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b3c005020 con 0x556d1fa7c5d0 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b26ffd640 1 -- 172.21.15.143:0/2744704988 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9b38020040 con 0x556d1fa7c5d0 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 10 client.17334 fetch_fsmap learned FSMap version 41 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556d1fb017e0 con 0x556d1fa7c5d0 2024-09-10T10:53:44.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b26ffd640 1 -- 172.21.15.143:0/2744704988 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b3c005020 con 0x556d1fa7c5d0 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b26ffd640 1 -- 172.21.15.143:0/2744704988 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 41) ==== 8406+0+0 (secure 0 0 0) 0x7f9b38097ad0 con 0x556d1fa7c5d0 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 10 client.17334 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 10 client.17334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 20 client.17334 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fb01f90 0x556d1fb043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 4 client.17334 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.392+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d1f9e9d70 con 0x556d1fb01f90 2024-09-10T10:53:44.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fb01f90 0x556d1fb043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b26ffd640 10 client.17334 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b46ffd640 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fb01f90 0x556d1fb043b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a00c2c85fe4fe072 server_cookie=bdc28694651bb443 in_seq=0 out_seq=0 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b26ffd640 1 -- 172.21.15.143:0/2744704988 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d1f9e9d70 con 0x556d1fb01f90 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b26ffd640 10 client.17334 handle_command_reply: tid=0 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 client.17334 shutdown 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 msgr2=0x556d1fa8fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 0x556d1fa8fa60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9b38097e30 tx=0x7f9b380ce040 comp rx=0 tx=0).stop 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 shutdown_connections 2024-09-10T10:53:44.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d1fa87370 0x556d1fa812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x556d1fb01f90 0x556d1fb043b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d1fa7db40 0x556d1fa80d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 --2- 172.21.15.143:0/2744704988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d1fa7c5d0 0x556d1fa8fa60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b26ffd640 0 client.17334 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 >> 172.21.15.143:0/2744704988 conn(0x556d1fa45310 msgr2=0x556d1fa922a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 shutdown_connections 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.393+0000 7f9b5052a740 1 -- 172.21.15.143:0/2744704988 wait complete. 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b5052a740 20 client.17334 trim_cache size 0 max 16384 2024-09-10T10:53:44.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9b300a2c00 msgr2=0x7f9b300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9b300a2c00 0x7f9b300a50c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009a60 tx=0x7f9b40043000 comp rx=0 tx=0).stop 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 msgr2=0x7f9b481982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 0x7f9b481982b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800d950 tx=0x7f9b3800de10 comp rx=0 tx=0).stop 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 shutdown_connections 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f9b300a2c00 0x7f9b300a50c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b48106f20 0x7f9b4819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b481027a0 0x7f9b481987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 --2- 172.21.15.143:0/3722140461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b48101df0 0x7f9b481982b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 >> 172.21.15.143:0/3722140461 conn(0x7f9b480fd8b0 msgr2=0x7f9b480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 shutdown_connections 2024-09-10T10:53:44.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:44.394+0000 7f9b4e151640 1 -- 172.21.15.143:0/3722140461 wait complete. 2024-09-10T10:53:44.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:44 smithi143 ceph-mon[31057]: pgmap v611: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 552 op/s 2024-09-10T10:53:44.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:44 smithi178 ceph-mon[34813]: pgmap v611: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 552 op/s 2024-09-10T10:53:44.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:44 smithi144 ceph-mon[34230]: pgmap v611: 129 pgs: 129 active+clean; 173 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 552 op/s 2024-09-10T10:53:45.127 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:53:45.566 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --cancel --all' 2024-09-10T10:53:45.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --cancel --all 2024-09-10T10:53:45.586 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:45.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:45 smithi143 ceph-mon[31057]: pgmap v612: 129 pgs: 129 active+clean; 172 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 851 KiB/s wr, 441 op/s 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 -- 172.21.15.143:0/1240322287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 msgr2=0x7fe4e410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/1240322287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e410d950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009ca0 tx=0x7fe4dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 -- 172.21.15.143:0/1240322287 shutdown_connections 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/1240322287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e410d950 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/1240322287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4e4100a80 0x7fe4e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/1240322287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4e4106a80 0x7fe4e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 -- 172.21.15.143:0/1240322287 >> 172.21.15.143:0/1240322287 conn(0x7fe4e40fc7a0 msgr2=0x7fe4e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 -- 172.21.15.143:0/1240322287 shutdown_connections 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.116+0000 7fe4e90f9640 1 -- 172.21.15.143:0/1240322287 wait complete. 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 Processor -- start 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 -- start start 2024-09-10T10:53:46.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4e4100a80 0x7fe4e4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4e4106a80 0x7fe4e419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4111e10 con 0x7fe4e4101420 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4111c90 con 0x7fe4e4106a80 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.117+0000 7fe4e90f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4111f90 con 0x7fe4e4100a80 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e4198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59164/0 (socket says 172.21.15.143:59164) 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 -- 172.21.15.143:0/3732889772 learned_addr learned my addr 172.21.15.143:0/3732889772 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4e4106a80 0x7fe4e419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4e4100a80 0x7fe4e4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 -- 172.21.15.143:0/3732889772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4e4100a80 msgr2=0x7fe4e4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4e4100a80 0x7fe4e4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 -- 172.21.15.143:0/3732889772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4e4106a80 msgr2=0x7fe4e419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4e4106a80 0x7fe4e419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 -- 172.21.15.143:0/3732889772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e419d2e0 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e4198870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800d770 tx=0x7fe4d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4cbfff640 1 -- 172.21.15.143:0/3732889772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8017070 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4cbfff640 1 -- 172.21.15.143:0/3732889772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d80054d0 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e419d5d0 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4cbfff640 1 -- 172.21.15.143:0/3732889772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8004cf0 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.118+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41a4eb0 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4cbfff640 1 -- 172.21.15.143:0/3732889772 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe4d8004e90 con 0x7fe4e4101420 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 Processor -- start 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 -- start start 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a41760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55701699a2f0 con 0x557016a3f370 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557016941cf0 con 0x557016a35b40 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5570169962b0 con 0x557016a345d0 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4cbfff640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4cc0a2b80 0x7fe4cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a41760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a41760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59166/0 (socket says 172.21.15.143:59166) 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4e259c640 1 -- 172.21.15.143:0/3595718089 learned_addr learned my addr 172.21.15.143:0/3595718089 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.121+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4cc0a2b80 0x7fe4cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4cbfff640 1 -- 172.21.15.143:0/3732889772 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe4d80cee10 con 0x7fe4e4101420 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e2d9d640 1 -- 172.21.15.143:0/3595718089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 msgr2=0x557016a34090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a34090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e2d9d640 1 -- 172.21.15.143:0/3595718089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 msgr2=0x557016a41760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a41760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e2d9d640 1 -- 172.21.15.143:0/3595718089 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570169263a0 con 0x557016a345d0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4cc0a2b80 0x7fe4cc0a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc004af0 tx=0x7fe4dc005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a34a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009af0 tx=0x7fe4d40113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91846aaad2deb73d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4c97fa640 1 -- 172.21.15.143:0/3595718089 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4011e10 con 0x557016a345d0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4c97fa640 1 -- 172.21.15.143:0/3595718089 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d4016070 con 0x557016a345d0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.122+0000 7fe4c97fa640 1 -- 172.21.15.143:0/3595718089 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4012450 con 0x557016a345d0 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3595718089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 msgr2=0x557016a34a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a34a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009af0 tx=0x7fe4d40113e0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3595718089 shutdown_connections 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a34a10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3595718089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a34090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3595718089 >> 172.21.15.143:0/3595718089 conn(0x5570169fd310 msgr2=0x557016a30c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3595718089 shutdown_connections 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.123+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3595718089 wait complete. 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 -- start start 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 Processor -- start 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a3ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a3bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a3c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x557016a42070 con 0x557016a3f370 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557016955060 con 0x557016a35b40 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557016a421f0 con 0x557016a345d0 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4eb4d2740 10 client.0 _ll_register_callbacks cb 0x5570166a88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a3c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a3c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59176/0 (socket says 172.21.15.143:59176) 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 -- 172.21.15.143:0/3924041746 learned_addr learned my addr 172.21.15.143:0/3924041746 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a3ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a3bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 -- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 msgr2=0x557016a3ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a3ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 -- 172.21.15.143:0/3924041746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 msgr2=0x557016a3bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a3bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.124+0000 7fe4e359e640 1 -- 172.21.15.143:0/3924041746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701670b3c0 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a3c530 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009430 tx=0x7fe4dc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0592f089cdcbaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/3924041746 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc004240 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/3924041746 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe4dc0043e0 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4016070 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/3924041746 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc05d050 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 10 client.17346 fetch_fsmap learned FSMap version 41 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557016a4dad0 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/3924041746 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d4016070 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/3924041746 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 41) ==== 8406+0+0 (secure 0 0 0) 0x7fe4dc030c30 con 0x557016a3f370 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 10 client.17346 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:53:46.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 20 client.17346 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016a4eee0 0x557016a51380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 4 client.17346 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.125+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570169a7860 con 0x557016a4eee0 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.126+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016a4eee0 0x557016a51380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.126+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016a4eee0 0x557016a51380 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83fb3933b50db43 server_cookie=8b8015e2de3c9c4 in_seq=0 out_seq=0 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.126+0000 7fe4c6ff5640 10 client.17346 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/3924041746 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570169a7860 con 0x557016a4eee0 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4c6ff5640 10 client.17346 handle_command_reply: tid=0 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 client.17346 shutdown 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 msgr2=0x557016a3c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a3c530 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009430 tx=0x7fe4dc009360 comp rx=0 tx=0).stop 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 shutdown_connections 2024-09-10T10:53:46.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a3c530 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a3bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016a4eee0 0x557016a51380 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3924041746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a3ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4c6ff5640 0 client.17346 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 >> 172.21.15.143:0/3924041746 conn(0x5570169fd310 msgr2=0x557016a36960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 shutdown_connections 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3924041746 wait complete. 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.127+0000 7fe4eb4d2740 20 client.17346 trim_cache size 0 max 16384 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 Processor -- start 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 -- start start 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a4b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 0x557016a3b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a3f370 0x557016a56c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41aeb40 con 0x557016a35b40 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41b07a0 con 0x557016a3f370 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41af210 con 0x557016a345d0 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 0x557016a3b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 0x557016a3b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59182/0 (socket says 172.21.15.143:59182) 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 -- 172.21.15.143:0/3754334338 learned_addr learned my addr 172.21.15.143:0/3754334338 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a4b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 -- 172.21.15.143:0/3754334338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 msgr2=0x557016a4b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e359e640 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a3f370 0x557016a56c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a4b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 -- 172.21.15.143:0/3754334338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a3f370 msgr2=0x557016a56c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a3f370 0x557016a56c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e259c640 1 -- 172.21.15.143:0/3754334338 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557016a3a580 con 0x557016a35b40 2024-09-10T10:53:46.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.140+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a4b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4e259c640 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 0x557016a3b010 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe4d809b2a0 tx=0x7fe4d80b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a1caa229798ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4c8ff9640 1 -- 172.21.15.143:0/3754334338 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8020070 con 0x557016a35b40 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4c8ff9640 1 -- 172.21.15.143:0/3754334338 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4d8009c30 con 0x557016a35b40 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4c8ff9640 1 -- 172.21.15.143:0/3754334338 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8099610 con 0x557016a35b40 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3754334338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 msgr2=0x557016a3b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 0x557016a3b010 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe4d809b2a0 tx=0x7fe4d80b4040 comp rx=0 tx=0).stop 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3754334338 shutdown_connections 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a3f370 0x557016a56c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a35b40 0x557016a3b010 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/3754334338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a4b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3754334338 >> 172.21.15.143:0/3754334338 conn(0x5570169fd310 msgr2=0x557016a3a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3754334338 shutdown_connections 2024-09-10T10:53:46.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.141+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/3754334338 wait complete. 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 -- start start 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 Processor -- start 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a44480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41b0010 con 0x557016a3f370 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41af910 con 0x557016a35b40 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41acb20 con 0x557016a345d0 2024-09-10T10:53:46.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4eb4d2740 10 client.0 _ll_register_callbacks cb 0x7fe4dc031780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a44480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a44480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59198/0 (socket says 172.21.15.143:59198) 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 -- 172.21.15.143:0/2297690456 learned_addr learned my addr 172.21.15.143:0/2297690456 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e259c640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 -- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 msgr2=0x557016a474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 -- 172.21.15.143:0/2297690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 msgr2=0x557016a479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.142+0000 7fe4e359e640 1 -- 172.21.15.143:0/2297690456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557016a48060 con 0x557016a3f370 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4e359e640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a44480 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0319e0 tx=0x7fe4dc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb31038610fea90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/2297690456 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc066060 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570169a1d70 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/2297690456 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe4dc04ca10 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/2297690456 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc06d050 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 10 client.17352 fetch_fsmap learned FSMap version 41 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557016aba150 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/2297690456 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5570169a1d70 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/2297690456 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 41) ==== 8406+0+0 (secure 0 0 0) 0x7fe4dc0044f0 con 0x557016a3f370 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 10 client.17352 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 20 client.17352 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016aba8a0 0x557016abccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 4 client.17352 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.143+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55701691e170 con 0x557016aba8a0 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016aba8a0 0x557016abccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4c6ff5640 10 client.17352 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4e2d9d640 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016aba8a0 0x557016abccc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4454ace11658ea server_cookie=d826e320e17af8bf in_seq=0 out_seq=0 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4c6ff5640 1 -- 172.21.15.143:0/2297690456 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55701691e170 con 0x557016aba8a0 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4c6ff5640 10 client.17352 handle_command_reply: tid=0 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 client.17352 shutdown 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 msgr2=0x557016a44480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a44480 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0319e0 tx=0x7fe4dc004b40 comp rx=0 tx=0).stop 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 shutdown_connections 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x557016a3f370 0x557016a44480 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557016a35b40 0x557016a479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x557016aba8a0 0x557016abccc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 --2- 172.21.15.143:0/2297690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557016a345d0 0x557016a474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4c6ff5640 0 client.17352 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 >> 172.21.15.143:0/2297690456 conn(0x5570169fd310 msgr2=0x557016a3b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 shutdown_connections 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 1 -- 172.21.15.143:0/2297690456 wait complete. 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.144+0000 7fe4eb4d2740 20 client.17352 trim_cache size 0 max 16384 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4cc0a2b80 msgr2=0x7fe4cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4cc0a2b80 0x7fe4cc0a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc004af0 tx=0x7fe4dc005ef0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 msgr2=0x7fe4e4198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e4198870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800d770 tx=0x7fe4d800dc30 comp rx=0 tx=0).stop 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 shutdown_connections 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4cc0a2b80 0x7fe4cc0a5040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4e4106a80 0x7fe4e419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4e4101420 0x7fe4e4198870 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 --2- 172.21.15.143:0/3732889772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4e4100a80 0x7fe4e4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 >> 172.21.15.143:0/3732889772 conn(0x7fe4e40fc7a0 msgr2=0x7fe4e40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 shutdown_connections 2024-09-10T10:53:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:46.145+0000 7fe4e90f9640 1 -- 172.21.15.143:0/3732889772 wait complete. 2024-09-10T10:53:46.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:45 smithi178 ceph-mon[34813]: pgmap v612: 129 pgs: 129 active+clean; 172 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 851 KiB/s wr, 441 op/s 2024-09-10T10:53:46.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:45 smithi144 ceph-mon[34230]: pgmap v612: 129 pgs: 129 active+clean; 172 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 851 KiB/s wr, 441 op/s 2024-09-10T10:53:46.833 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 -- 172.21.15.143:0/1020865281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa81021b0 msgr2=0x7f3aa81056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/1020865281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa81021b0 0x7f3aa81056f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009930 tx=0x7f3a9002a420 comp rx=0 tx=0).stop 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 -- 172.21.15.143:0/1020865281 shutdown_connections 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/1020865281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aa8105e00 0x7f3aa810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/1020865281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa81021b0 0x7f3aa81056f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/1020865281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8101800 0x7f3aa8101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 -- 172.21.15.143:0/1020865281 >> 172.21.15.143:0/1020865281 conn(0x7f3aa80fba90 msgr2=0x7f3aa80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 -- 172.21.15.143:0/1020865281 shutdown_connections 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.371+0000 7f3aadbdd640 1 -- 172.21.15.143:0/1020865281 wait complete. 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 Processor -- start 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 -- start start 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 0x7f3aa8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aa81021b0 0x7f3aa8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8105e00 0x7f3aa819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8110cc0 con 0x7f3aa8101800 2024-09-10T10:53:47.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8110b40 con 0x7f3aa8105e00 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.372+0000 7f3aadbdd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8110e40 con 0x7f3aa81021b0 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 0x7f3aa8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 0x7f3aa8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53064/0 (socket says 172.21.15.143:53064) 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 -- 172.21.15.143:0/25071684 learned_addr learned my addr 172.21.15.143:0/25071684 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3a9effd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aa81021b0 0x7f3aa8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8105e00 0x7f3aa819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 -- 172.21.15.143:0/25071684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aa81021b0 msgr2=0x7f3aa8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aa81021b0 0x7f3aa8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 -- 172.21.15.143:0/25071684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8105e00 msgr2=0x7f3aa819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8105e00 0x7f3aa819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 -- 172.21.15.143:0/25071684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa819d3c0 con 0x7f3aa8101800 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8105e00 0x7f3aa819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 0x7f3aa8198410 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c890 tx=0x7f3a9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa57fa640 1 -- 172.21.15.143:0/25071684 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98004220 con 0x7f3aa8101800 2024-09-10T10:53:47.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa819d6b0 con 0x7f3aa8101800 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa57fa640 1 -- 172.21.15.143:0/25071684 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a980043c0 con 0x7f3aa8101800 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aa57fa640 1 -- 172.21.15.143:0/25071684 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98004a00 con 0x7f3aa8101800 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.373+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa81a4f90 con 0x7f3aa8101800 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aa57fa640 1 -- 172.21.15.143:0/25071684 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3a98004ba0 con 0x7f3aa8101800 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 Processor -- start 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 -- start start 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aa57fa640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3a940a2b80 0x7f3a940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef7fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f86eee4200 con 0x55f86ef7f8e0 2024-09-10T10:53:47.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f86ee8adc0 con 0x55f86ef80200 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.376+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f86eee01c0 con 0x55f86ef8eff0 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3a940a2b80 0x7f3a940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef7fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa57fa640 1 -- 172.21.15.143:0/25071684 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3a980cf0c0 con 0x7f3aa8101800 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef7fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53066/0 (socket says 172.21.15.143:53066) 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 -- 172.21.15.143:0/354629293 learned_addr learned my addr 172.21.15.143:0/354629293 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3a9effd640 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 -- 172.21.15.143:0/354629293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 msgr2=0x55f86ef913e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef913e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 -- 172.21.15.143:0/354629293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 msgr2=0x55f86ef8eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 -- 172.21.15.143:0/354629293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f86ec55620 con 0x55f86ef7f8e0 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3a940a2b80 0x7f3a940a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009fd0 tx=0x7f3aa0014b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef7fcc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3a9809b4f0 tx=0x7f3a98097920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfb12ec4542eeae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3a9dffb640 1 -- 172.21.15.143:0/354629293 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801d070 con 0x55f86ef7f8e0 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3a9dffb640 1 -- 172.21.15.143:0/354629293 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a9809b560 con 0x55f86ef7f8e0 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.377+0000 7f3a9dffb640 1 -- 172.21.15.143:0/354629293 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980cc910 con 0x55f86ef7f8e0 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 -- 172.21.15.143:0/354629293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 msgr2=0x55f86ef7fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef7fcc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3a9809b4f0 tx=0x7f3a98097920 comp rx=0 tx=0).stop 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 -- 172.21.15.143:0/354629293 shutdown_connections 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef913e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/354629293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef7fcc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 -- 172.21.15.143:0/354629293 >> 172.21.15.143:0/354629293 conn(0x55f86ef7bf90 msgr2=0x55f86ef7c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 -- 172.21.15.143:0/354629293 shutdown_connections 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 -- 172.21.15.143:0/354629293 wait complete. 2024-09-10T10:53:47.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.378+0000 7f3aaffb6740 1 -- start start 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 Processor -- start 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef8e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86ef8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86ef8c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f86ef91bc0 con 0x55f86ef7f8e0 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f86ee7ec50 con 0x55f86ef8eff0 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f86ef91d40 con 0x55f86ef80200 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aaffb6740 10 client.0 _ll_register_callbacks cb 0x55f86ee90ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef8e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef8e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53074/0 (socket says 172.21.15.143:53074) 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 -- 172.21.15.143:0/1328406186 learned_addr learned my addr 172.21.15.143:0/1328406186 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3a9effd640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86ef8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86ef8c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 -- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 msgr2=0x55f86ef8bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86ef8bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 -- 172.21.15.143:0/1328406186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 msgr2=0x55f86ef8c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86ef8c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.379+0000 7f3aa77fe640 1 -- 172.21.15.143:0/1328406186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f86ef8c9e0 con 0x55f86ef7f8e0 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef8e930 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3a980979c0 tx=0x7f3a98097870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769ebc1a04256f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3a86ffd640 1 -- 172.21.15.143:0/1328406186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801d040 con 0x55f86ef7f8e0 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3a86ffd640 1 -- 172.21.15.143:0/1328406186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3a9807d0b0 con 0x55f86ef7f8e0 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9809b560 con 0x55f86ef7f8e0 2024-09-10T10:53:47.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3a86ffd640 1 -- 172.21.15.143:0/1328406186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3a980a41d0 con 0x55f86ef7f8e0 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 10 client.17364 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f86ef9d7c0 con 0x55f86ef7f8e0 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3a86ffd640 1 -- 172.21.15.143:0/1328406186 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a9809b560 con 0x55f86ef7f8e0 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3a86ffd640 1 -- 172.21.15.143:0/1328406186 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f3a98081540 con 0x55f86ef7f8e0 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 10 client.17364 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 10 client.17364 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 20 client.17364 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86ef9eb10 0x55f86efa0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 4 client.17364 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.380+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f86eef1770 con 0x55f86ef9eb10 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.381+0000 7f3a9effd640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86ef9eb10 0x55f86efa0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.381+0000 7f3a9effd640 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86ef9eb10 0x55f86efa0f00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780593b80c359e8b server_cookie=dd1169084fcb2935 in_seq=0 out_seq=0 2024-09-10T10:53:47.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.381+0000 7f3a86ffd640 10 client.17364 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3a86ffd640 1 -- 172.21.15.143:0/1328406186 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f86eef1770 con 0x55f86ef9eb10 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3a86ffd640 10 client.17364 handle_command_reply: tid=0 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 client.17364 shutdown 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 msgr2=0x55f86ef8e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef8e930 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3a980979c0 tx=0x7f3a98097870 comp rx=0 tx=0).stop 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 shutdown_connections 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86ef8c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86ef9eb10 0x55f86efa0f00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86ef8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/1328406186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef8e930 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 >> 172.21.15.143:0/1328406186 conn(0x55f86ef7bf90 msgr2=0x55f86ef81cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3a86ffd640 0 client.17364 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:47.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 shutdown_connections 2024-09-10T10:53:47.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 1 -- 172.21.15.143:0/1328406186 wait complete. 2024-09-10T10:53:47.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.382+0000 7f3aaffb6740 20 client.17364 trim_cache size 0 max 16384 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 Processor -- start 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 -- start start 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef9a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81aca90 con 0x55f86ef7f8e0 2024-09-10T10:53:47.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ae6f0 con 0x55f86ef80200 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ad160 con 0x55f86ef8eff0 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef9a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef9a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53090/0 (socket says 172.21.15.143:53090) 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3aa77fe640 1 -- 172.21.15.143:0/2959140856 learned_addr learned my addr 172.21.15.143:0/2959140856 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.399+0000 7f3a9effd640 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa77fe640 1 -- 172.21.15.143:0/2959140856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 msgr2=0x55f86ef88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef88e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa77fe640 1 -- 172.21.15.143:0/2959140856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 msgr2=0x55f86ef8e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa77fe640 1 -- 172.21.15.143:0/2959140856 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f86ef8c520 con 0x55f86ef7f8e0 2024-09-10T10:53:47.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3a9effd640 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef9a6e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3a98097ee0 tx=0x7f3a98005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995098ce8f803fff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3a9d7fa640 1 -- 172.21.15.143:0/2959140856 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801d040 con 0x55f86ef7f8e0 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3a9d7fa640 1 -- 172.21.15.143:0/2959140856 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a98084af0 con 0x55f86ef7f8e0 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.400+0000 7f3a9d7fa640 1 -- 172.21.15.143:0/2959140856 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98081460 con 0x55f86ef7f8e0 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 -- 172.21.15.143:0/2959140856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 msgr2=0x55f86ef9a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef9a6e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3a98097ee0 tx=0x7f3a98005380 comp rx=0 tx=0).stop 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 -- 172.21.15.143:0/2959140856 shutdown_connections 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef8eff0 0x55f86ef88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef80200 0x55f86ef8e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/2959140856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86ef7f8e0 0x55f86ef9a6e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 -- 172.21.15.143:0/2959140856 >> 172.21.15.143:0/2959140856 conn(0x55f86ef7bf90 msgr2=0x55f86ef99fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 -- 172.21.15.143:0/2959140856 shutdown_connections 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 -- 172.21.15.143:0/2959140856 wait complete. 2024-09-10T10:53:47.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.401+0000 7f3aaffb6740 1 -- start start 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 Processor -- start 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86efa4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86efa13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 0x55f86efa1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81adf60 con 0x55f86efa1910 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ad860 con 0x55f86ef8eff0 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81aaa70 con 0x55f86ef80200 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aaffb6740 10 client.0 _ll_register_callbacks cb 0x7f3a9809bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 0x55f86efa1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3a9effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86efa13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 0x55f86efa1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53106/0 (socket says 172.21.15.143:53106) 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 -- 172.21.15.143:0/505154710 learned_addr learned my addr 172.21.15.143:0/505154710 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:47.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86efa4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 -- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 msgr2=0x55f86efa4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86efa4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 -- 172.21.15.143:0/505154710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 msgr2=0x55f86efa13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86efa13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa7fff640 1 -- 172.21.15.143:0/505154710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f86f008450 con 0x55f86efa1910 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.402+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86efa4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3aa7fff640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 0x55f86efa1d90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005c40 tx=0x7f3aa0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba162e83243ff1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3a86ffd640 1 -- 172.21.15.143:0/505154710 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0010e30 con 0x55f86efa1910 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3a86ffd640 1 -- 172.21.15.143:0/505154710 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3aa002b410 con 0x55f86efa1910 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a98084af0 con 0x55f86efa1910 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3a86ffd640 1 -- 172.21.15.143:0/505154710 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0014020 con 0x55f86efa1910 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3aaffb6740 10 client.17370 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:47.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f86f0096c0 con 0x55f86efa1910 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.403+0000 7f3a86ffd640 1 -- 172.21.15.143:0/505154710 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a98084af0 con 0x55f86efa1910 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3a86ffd640 1 -- 172.21.15.143:0/505154710 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f3aa0013420 con 0x55f86efa1910 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 10 client.17370 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 10 client.17370 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 20 client.17370 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86f009f00 0x55f86f00c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 4 client.17370 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f86eeebc80 con 0x55f86f009f00 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86f009f00 0x55f86f00c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3a86ffd640 10 client.17370 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aa77fe640 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86f009f00 0x55f86f00c320 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df514e2728840f1 server_cookie=6607e6a40950484f in_seq=0 out_seq=0 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3a86ffd640 1 -- 172.21.15.143:0/505154710 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f86eeebc80 con 0x55f86f009f00 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3a86ffd640 10 client.17370 handle_command_reply: tid=0 2024-09-10T10:53:47.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.404+0000 7f3aaffb6740 1 client.17370 shutdown 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 msgr2=0x55f86efa1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 0x55f86efa1d90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005c40 tx=0x7f3aa0004ab0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 shutdown_connections 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f86efa1910 0x55f86efa1d90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f86ef8eff0 0x55f86efa13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55f86f009f00 0x55f86f00c320 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 --2- 172.21.15.143:0/505154710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f86ef80200 0x55f86efa4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 >> 172.21.15.143:0/505154710 conn(0x55f86ef7bf90 msgr2=0x55f86ef9a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3a86ffd640 0 client.17370 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 shutdown_connections 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 1 -- 172.21.15.143:0/505154710 wait complete. 2024-09-10T10:53:47.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.405+0000 7f3aaffb6740 20 client.17370 trim_cache size 0 max 16384 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 42, 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stdout: "leader": 24502, 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stdout: "set_version": 24, 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stdout: "sets": {} 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3a940a2b80 msgr2=0x7f3a940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3a940a2b80 0x7f3a940a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009fd0 tx=0x7f3aa0014b30 comp rx=0 tx=0).stop 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 msgr2=0x7f3aa8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 0x7f3aa8198410 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c890 tx=0x7f3a9800cd50 comp rx=0 tx=0).stop 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 shutdown_connections 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f3a940a2b80 0x7f3a940a5040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa8105e00 0x7f3aa819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aa81021b0 0x7f3aa8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 --2- 172.21.15.143:0/25071684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa8101800 0x7f3aa8198410 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 >> 172.21.15.143:0/25071684 conn(0x7f3aa80fba90 msgr2=0x7f3aa81007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 shutdown_connections 2024-09-10T10:53:47.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:47.406+0000 7f3aadbdd640 1 -- 172.21.15.143:0/25071684 wait complete. 2024-09-10T10:53:47.862 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:53:47.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:53:48.013 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:47 smithi143 ceph-mon[31057]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:48.013 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:47 smithi143 ceph-mon[31057]: pgmap v613: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 586 KiB/s wr, 432 op/s 2024-09-10T10:53:48.175 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T10:53:48.175 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T10:53:48.186 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T10:53:48.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:47 smithi144 ceph-mon[34230]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:48.187 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:47 smithi144 ceph-mon[34230]: pgmap v613: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 586 KiB/s wr, 432 op/s 2024-09-10T10:53:48.187 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:47 smithi178 ceph-mon[34813]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:53:48.187 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:47 smithi178 ceph-mon[34813]: pgmap v613: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 586 KiB/s wr, 432 op/s 2024-09-10T10:53:48.219 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:49.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:49 smithi143 ceph-mon[31057]: scrub queued for path: / 2024-09-10T10:53:49.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:49 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:49.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:49 smithi143 ceph-mon[31057]: scrub summary: active paths [/] 2024-09-10T10:53:49.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:49 smithi143 ceph-mon[31057]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:49.551 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:49 smithi143 ceph-mon[31057]: scrub summary: idle 2024-09-10T10:53:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:49 smithi178 ceph-mon[34813]: scrub queued for path: / 2024-09-10T10:53:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:49 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:49 smithi178 ceph-mon[34813]: scrub summary: active paths [/] 2024-09-10T10:53:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:49 smithi178 ceph-mon[34813]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:49.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:49 smithi178 ceph-mon[34813]: scrub summary: idle 2024-09-10T10:53:49.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:49 smithi144 ceph-mon[34230]: scrub queued for path: / 2024-09-10T10:53:49.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:49 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:49.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:49 smithi144 ceph-mon[34230]: scrub summary: active paths [/] 2024-09-10T10:53:49.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:49 smithi144 ceph-mon[34230]: scrub summary: idle+waiting paths [/] 2024-09-10T10:53:49.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:49 smithi144 ceph-mon[34230]: scrub summary: idle 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 -- 172.21.15.143:0/919018573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 msgr2=0x7f0340112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 --2- 172.21.15.143:0/919018573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f0340112650 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402a480 comp rx=0 tx=0).stop 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 -- 172.21.15.143:0/919018573 shutdown_connections 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 --2- 172.21.15.143:0/919018573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340112b90 0x7f0340114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 --2- 172.21.15.143:0/919018573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f0340112650 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 --2- 172.21.15.143:0/919018573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 0x7f0340074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 -- 172.21.15.143:0/919018573 >> 172.21.15.143:0/919018573 conn(0x7f03400fd8c0 msgr2=0x7f03400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 -- 172.21.15.143:0/919018573 shutdown_connections 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.872+0000 7f034726e640 1 -- 172.21.15.143:0/919018573 wait complete. 2024-09-10T10:53:49.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 Processor -- start 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 -- start start 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 0x7f03401a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f03401a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340112b90 0x7f034019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f03401183f0 con 0x7f0340074eb0 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0340118270 con 0x7f0340112b90 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.873+0000 7f034726e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0340118570 con 0x7f0340074590 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f034500a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 0x7f03401a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f03401a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f034500a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 0x7f03401a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:57252/0 (socket says 172.21.15.143:57252) 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f034500a640 1 -- 172.21.15.143:0/1578221675 learned_addr learned my addr 172.21.15.143:0/1578221675 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f034580b640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340112b90 0x7f034019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 -- 172.21.15.143:0/1578221675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 msgr2=0x7f03401a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 0x7f03401a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 -- 172.21.15.143:0/1578221675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340112b90 msgr2=0x7f034019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340112b90 0x7f034019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 -- 172.21.15.143:0/1578221675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034019ef20 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f0344809640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f03401a4b40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f033402a9a0 tx=0x7f0334004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f03327fc640 1 -- 172.21.15.143:0/1578221675 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033403f030 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f03327fc640 1 -- 172.21.15.143:0/1578221675 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0334003aa0 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034019f1b0 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.874+0000 7f03327fc640 1 -- 172.21.15.143:0/1578221675 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334035070 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.875+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03401ab460 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f03327fc640 1 -- 172.21.15.143:0/1578221675 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0334042030 con 0x7f0340074eb0 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 Processor -- start 2024-09-10T10:53:49.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 -- start start 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9873b60 0x55bbe9873f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe987e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9881130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bbe97d9280 con 0x55bbe987ed40 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bbe9780b10 con 0x55bbe9873b60 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.877+0000 7f0349647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbe97d5240 con 0x55bbe9874480 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9881130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9881130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53138/0 (socket says 172.21.15.143:53138) 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 -- 172.21.15.143:0/3672464148 learned_addr learned my addr 172.21.15.143:0/3672464148 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f03327fc640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f032c0a2b80 0x7f032c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f034500a640 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe987e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f034580b640 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9873b60 0x55bbe9873f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f034580b640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f032c0a2b80 0x7f032c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f03327fc640 1 -- 172.21.15.143:0/1578221675 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f03340ef870 con 0x7f0340074eb0 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 -- 172.21.15.143:0/3672464148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 msgr2=0x55bbe987e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe987e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 -- 172.21.15.143:0/3672464148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9873b60 msgr2=0x55bbe9873f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9873b60 0x55bbe9873f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 -- 172.21.15.143:0/3672464148 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe978d1f0 con 0x55bbe987ed40 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f0344809640 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9881130 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f03340b87f0 tx=0x7f033402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fab9fd33a6e256 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f034580b640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f032c0a2b80 0x7f032c0a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0338005fd0 tx=0x7f033800a4f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.878+0000 7f032b7fe640 1 -- 172.21.15.143:0/3672464148 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334049070 con 0x55bbe987ed40 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f032b7fe640 1 -- 172.21.15.143:0/3672464148 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03340c0120 con 0x55bbe987ed40 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f032b7fe640 1 -- 172.21.15.143:0/3672464148 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340b8490 con 0x55bbe987ed40 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 -- 172.21.15.143:0/3672464148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 msgr2=0x55bbe9881130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9881130 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f03340b87f0 tx=0x7f033402a480 comp rx=0 tx=0).stop 2024-09-10T10:53:49.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 -- 172.21.15.143:0/3672464148 shutdown_connections 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9881130 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe987e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 --2- 172.21.15.143:0/3672464148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9873b60 0x55bbe9873f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 -- 172.21.15.143:0/3672464148 >> 172.21.15.143:0/3672464148 conn(0x55bbe98700f0 msgr2=0x55bbe98704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 -- 172.21.15.143:0/3672464148 shutdown_connections 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.879+0000 7f0349647740 1 -- 172.21.15.143:0/3672464148 wait complete. 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 -- start start 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 Processor -- start 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 0x55bbe987c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe987c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bbe98816d0 con 0x55bbe9874480 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bbe9793f10 con 0x55bbe987ed40 2024-09-10T10:53:49.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbe9881850 con 0x55bbe9873b60 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0349647740 10 client.0 _ll_register_callbacks cb 0x55bbe975ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.880+0000 7f0344809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 0x55bbe987c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 0x55bbe987c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53146/0 (socket says 172.21.15.143:53146) 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 -- 172.21.15.143:0/3146425103 learned_addr learned my addr 172.21.15.143:0/3146425103 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f034500a640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 -- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 msgr2=0x55bbe987bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f034580b640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe987c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 -- 172.21.15.143:0/3146425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 msgr2=0x55bbe987c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe987c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 -- 172.21.15.143:0/3146425103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe954a6a0 con 0x55bbe9874480 2024-09-10T10:53:49.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f034580b640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe987c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0344809640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 0x55bbe987c120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f03340b68c0 tx=0x7f0334009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a18605ae45cc480 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0328ff9640 1 -- 172.21.15.143:0/3146425103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334049050 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0328ff9640 1 -- 172.21.15.143:0/3146425103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f03340b8490 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03340c0120 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0328ff9640 1 -- 172.21.15.143:0/3146425103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f03340bc080 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0349647740 10 client.17382 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bbe98816d0 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.881+0000 7f0328ff9640 1 -- 172.21.15.143:0/3146425103 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03340c0120 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0328ff9640 1 -- 172.21.15.143:0/3146425103 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f033403bbb0 con 0x55bbe9874480 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0349647740 10 client.17382 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0349647740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0349647740 20 client.17382 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:49.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0349647740 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe988e5a0 0x55bbe98909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0349647740 4 client.17382 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe97e67f0 con 0x55bbe988e5a0 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f034500a640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe988e5a0 0x55bbe98909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f0328ff9640 10 client.17382 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.882+0000 7f034500a640 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe988e5a0 0x55bbe98909c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78fdf060eb942cd server_cookie=a9c8372e081f38a4 in_seq=0 out_seq=0 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0328ff9640 1 -- 172.21.15.143:0/3146425103 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbe97e67f0 con 0x55bbe988e5a0 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0328ff9640 10 client.17382 handle_command_reply: tid=0 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 client.17382 shutdown 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 msgr2=0x55bbe987c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 0x55bbe987c120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f03340b68c0 tx=0x7f0334009680 comp rx=0 tx=0).stop 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 shutdown_connections 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe987c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9874480 0x55bbe987c120 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe988e5a0 0x55bbe98909c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 --2- 172.21.15.143:0/3146425103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0328ff9640 0 client.17382 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 >> 172.21.15.143:0/3146425103 conn(0x55bbe98700f0 msgr2=0x55bbe9875dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 shutdown_connections 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.883+0000 7f0349647740 1 -- 172.21.15.143:0/3146425103 wait complete. 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.884+0000 7f0349647740 20 client.17382 trim_cache size 0 max 16384 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 Processor -- start 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 -- start start 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9874480 0x55bbe987da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9896270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b5170 con 0x55bbe987ed40 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b6dd0 con 0x55bbe9874480 2024-09-10T10:53:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0349647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b5840 con 0x55bbe9873b60 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9896270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9896270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53154/0 (socket says 172.21.15.143:53154) 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 -- 172.21.15.143:0/817709755 learned_addr learned my addr 172.21.15.143:0/817709755 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034500a640 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f0344809640 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9874480 0x55bbe987da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 -- 172.21.15.143:0/817709755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 msgr2=0x55bbe987c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 -- 172.21.15.143:0/817709755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9874480 msgr2=0x55bbe987da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9874480 0x55bbe987da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.901+0000 7f034580b640 1 -- 172.21.15.143:0/817709755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe987a130 con 0x55bbe987ed40 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f034580b640 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9896270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f03380149d0 tx=0x7f033800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc10a5b781a1faf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f032affd640 1 -- 172.21.15.143:0/817709755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338025070 con 0x55bbe987ed40 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f032affd640 1 -- 172.21.15.143:0/817709755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0338011e60 con 0x55bbe987ed40 2024-09-10T10:53:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f032affd640 1 -- 172.21.15.143:0/817709755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338021b30 con 0x55bbe987ed40 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 -- 172.21.15.143:0/817709755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 msgr2=0x55bbe9896270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9896270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f03380149d0 tx=0x7f033800a8f0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 -- 172.21.15.143:0/817709755 shutdown_connections 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe987ed40 0x55bbe9896270 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe9874480 0x55bbe987da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 --2- 172.21.15.143:0/817709755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9873b60 0x55bbe987c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 -- 172.21.15.143:0/817709755 >> 172.21.15.143:0/817709755 conn(0x55bbe98700f0 msgr2=0x55bbe987d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.902+0000 7f0349647740 1 -- 172.21.15.143:0/817709755 shutdown_connections 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 -- 172.21.15.143:0/817709755 wait complete. 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 -- start start 2024-09-10T10:53:49.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 Processor -- start 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9873b60 0x55bbe98827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe9882cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe9883230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b6640 con 0x55bbe9873b60 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b5f40 con 0x55bbe987ed40 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.903+0000 7f0349647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b3150 con 0x55bbe9874480 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0349647740 10 client.0 _ll_register_callbacks cb 0x7f033800adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe9882cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe9882cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.143:57288/0 (socket says 172.21.15.143:57288) 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 -- 172.21.15.143:0/237876923 learned_addr learned my addr 172.21.15.143:0/237876923 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f034500a640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9873b60 0x55bbe98827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f034580b640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe9883230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 -- 172.21.15.143:0/237876923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 msgr2=0x55bbe9883230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe9883230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 -- 172.21.15.143:0/237876923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9873b60 msgr2=0x55bbe98827b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9873b60 0x55bbe98827b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 -- 172.21.15.143:0/237876923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe98f8290 con 0x55bbe9874480 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f034500a640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9873b60 0x55bbe98827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0344809640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe9882cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f03340b8280 tx=0x7f03340bb2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f494d0385994e6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0328ff9640 1 -- 172.21.15.143:0/237876923 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334049040 con 0x55bbe9874480 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0338011e60 con 0x55bbe9874480 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.904+0000 7f0328ff9640 1 -- 172.21.15.143:0/237876923 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0334003c50 con 0x55bbe9874480 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0328ff9640 1 -- 172.21.15.143:0/237876923 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f03340a5050 con 0x55bbe9874480 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 10 client.26764 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bbe98f8290 con 0x55bbe9874480 2024-09-10T10:53:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0328ff9640 1 -- 172.21.15.143:0/237876923 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0338011e60 con 0x55bbe9874480 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0328ff9640 1 -- 172.21.15.143:0/237876923 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f033403bbb0 con 0x55bbe9874480 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 10 client.26764 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 20 client.26764 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe98f9990 0x55bbe98fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 4 client.26764 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bbe97e0d00 con 0x55bbe98f9990 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.905+0000 7f034500a640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe98f9990 0x55bbe98fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.906+0000 7f034500a640 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe98f9990 0x55bbe98fbdb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763ba8f934b84625 server_cookie=24d42c4b37ffd7c1 in_seq=0 out_seq=0 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.906+0000 7f0328ff9640 10 client.26764 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0328ff9640 1 -- 172.21.15.143:0/237876923 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bbe97e0d00 con 0x55bbe98f9990 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0328ff9640 10 client.26764 handle_command_reply: tid=0 2024-09-10T10:53:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 client.26764 shutdown 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 msgr2=0x55bbe9882cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe9882cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f03340b8280 tx=0x7f03340bb2f0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stdout: "return_code": 0, 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_tag": "7b25569a-c950-4f9d-9651-0da6a30803e9", 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": "asynchronous" 2024-09-10T10:53:49.757 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 shutdown_connections 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bbe987ed40 0x55bbe9883230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55bbe98f9990 0x55bbe98fbdb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbe9874480 0x55bbe9882cf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 --2- 172.21.15.143:0/237876923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bbe9873b60 0x55bbe98827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0328ff9640 0 client.26764 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 >> 172.21.15.143:0/237876923 conn(0x55bbe98700f0 msgr2=0x55bbe986f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 shutdown_connections 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 1 -- 172.21.15.143:0/237876923 wait complete. 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.907+0000 7f0349647740 20 client.26764 trim_cache size 0 max 16384 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f032c0a2b80 msgr2=0x7f032c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f032c0a2b80 0x7f032c0a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0338005fd0 tx=0x7f033800a4f0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 msgr2=0x7f03401a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f03401a4b40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f033402a9a0 tx=0x7f0334004270 comp rx=0 tx=0).stop 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 shutdown_connections 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f032c0a2b80 0x7f032c0a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340112b90 0x7f034019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0340074eb0 0x7f03401a4b40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 --2- 172.21.15.143:0/1578221675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0340074590 0x7f03401a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 >> 172.21.15.143:0/1578221675 conn(0x7f03400fd8c0 msgr2=0x7f03400ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 shutdown_connections 2024-09-10T10:53:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:48.908+0000 7f034726e640 1 -- 172.21.15.143:0/1578221675 wait complete. 2024-09-10T10:53:50.550 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:50 smithi143 ceph-mon[31057]: pgmap v614: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 546 KiB/s wr, 298 op/s 2024-09-10T10:53:50.675 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:50 smithi178 ceph-mon[34813]: pgmap v614: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 546 KiB/s wr, 298 op/s 2024-09-10T10:53:50.686 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:50 smithi144 ceph-mon[34230]: pgmap v614: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 546 KiB/s wr, 298 op/s 2024-09-10T10:53:50.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:53:51.135 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 -- 172.21.15.143:0/3142505799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198106f10 msgr2=0x7f819810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 --2- 172.21.15.143:0/3142505799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198106f10 0x7f819810ea90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8190009ca0 tx=0x7f819002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 -- 172.21.15.143:0/3142505799 shutdown_connections 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 --2- 172.21.15.143:0/3142505799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198106f10 0x7f819810ea90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 --2- 172.21.15.143:0/3142505799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8198102790 0x7f81981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 --2- 172.21.15.143:0/3142505799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198101de0 0x7f81981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 -- 172.21.15.143:0/3142505799 >> 172.21.15.143:0/3142505799 conn(0x7f81980fd8c0 msgr2=0x7f81980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 -- 172.21.15.143:0/3142505799 shutdown_connections 2024-09-10T10:53:51.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.621+0000 7f819d03f640 1 -- 172.21.15.143:0/3142505799 wait complete. 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 Processor -- start 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 -- start start 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 0x7f8198198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8198102790 0x7f8198198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198106f10 0x7f819819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111e40 con 0x7f8198101de0 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111cc0 con 0x7f8198106f10 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819d03f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111fc0 con 0x7f8198102790 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f8196d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 0x7f8198198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f8196d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 0x7f8198198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53182/0 (socket says 172.21.15.143:53182) 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f8196d9d640 1 -- 172.21.15.143:0/996256483 learned_addr learned my addr 172.21.15.143:0/996256483 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819759e640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198106f10 0x7f819819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.622+0000 7f819659c640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8198102790 0x7f8198198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f8196d9d640 1 -- 172.21.15.143:0/996256483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8198102790 msgr2=0x7f8198198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f8196d9d640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8198102790 0x7f8198198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f8196d9d640 1 -- 172.21.15.143:0/996256483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198106f10 msgr2=0x7f819819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f8196d9d640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198106f10 0x7f819819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f8196d9d640 1 -- 172.21.15.143:0/996256483 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819819d3b0 con 0x7f8198101de0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f819759e640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198106f10 0x7f819819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f8196d9d640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 0x7f8198198400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f818800d950 tx=0x7f818800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f817ffff640 1 -- 172.21.15.143:0/996256483 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188017070 con 0x7f8198101de0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f817ffff640 1 -- 172.21.15.143:0/996256483 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818800de80 con 0x7f8198101de0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819819d6a0 con 0x7f8198101de0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f817ffff640 1 -- 172.21.15.143:0/996256483 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818800f620 con 0x7f8198101de0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.623+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81981a4f80 con 0x7f8198101de0 2024-09-10T10:53:51.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.625+0000 7f817ffff640 1 -- 172.21.15.143:0/996256483 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8188009c30 con 0x7f8198101de0 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 Processor -- start 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 -- start start 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f817ffff640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81800a2b80 0x7f81800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b0ec6102e0 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0ec5b7ce0 con 0x55b0ec6aadc0 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819f418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0ec60c2a0 con 0x55b0ec6b5680 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819759e640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81800a2b80 0x7f81800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f817ffff640 1 -- 172.21.15.143:0/996256483 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f81880cf060 con 0x7f8198101de0 2024-09-10T10:53:51.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53196/0 (socket says 172.21.15.143:53196) 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 -- 172.21.15.143:0/654654579 learned_addr learned my addr 172.21.15.143:0/654654579 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819759e640 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 -- 172.21.15.143:0/654654579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 msgr2=0x55b0ec6b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 -- 172.21.15.143:0/654654579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 msgr2=0x55b0ec6b5140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 -- 172.21.15.143:0/654654579 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ec59c440 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f819759e640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81800a2b80 0x7f81800a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8190031040 tx=0x7f8190031b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.626+0000 7f8196d9d640 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aa880 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f818809b200 tx=0x7f818809ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee716eeaf894f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f817d7fa640 1 -- 172.21.15.143:0/654654579 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188020070 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f817d7fa640 1 -- 172.21.15.143:0/654654579 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818809bc00 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f817d7fa640 1 -- 172.21.15.143:0/654654579 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81880a4170 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 -- 172.21.15.143:0/654654579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 msgr2=0x55b0ec6aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aa880 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f818809b200 tx=0x7f818809ae60 comp rx=0 tx=0).stop 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 -- 172.21.15.143:0/654654579 shutdown_connections 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 --2- 172.21.15.143:0/654654579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aa880 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 -- 172.21.15.143:0/654654579 >> 172.21.15.143:0/654654579 conn(0x55b0ec6a6a30 msgr2=0x55b0ec6a6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 -- 172.21.15.143:0/654654579 shutdown_connections 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.627+0000 7f819f418740 1 -- 172.21.15.143:0/654654579 wait complete. 2024-09-10T10:53:51.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 -- start start 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 Processor -- start 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b0ec6b8250 con 0x55b0ec6aadc0 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0ec5cb050 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b0ec6b83d0 con 0x55b0ec6b5680 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819f418740 10 client.0 _ll_register_callbacks cb 0x55b0ec60b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53202/0 (socket says 172.21.15.143:53202) 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 -- 172.21.15.143:0/3408233942 learned_addr learned my addr 172.21.15.143:0/3408233942 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:51.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f8196d9d640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 -- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 msgr2=0x55b0ec6b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 -- 172.21.15.143:0/3408233942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 msgr2=0x55b0ec6b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f819659c640 1 -- 172.21.15.143:0/3408233942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ec3813c0 con 0x55b0ec6aadc0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.628+0000 7f8196d9d640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819659c640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b2160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f818c00e480 tx=0x7f818c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c083fad52f84d688 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f817aff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f817aff5640 1 -- 172.21.15.143:0/3408233942 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c017070 con 0x55b0ec6aadc0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f817aff5640 1 -- 172.21.15.143:0/3408233942 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f818c013680 con 0x55b0ec6aadc0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818809bc00 con 0x55b0ec6aadc0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f817aff5640 1 -- 172.21.15.143:0/3408233942 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f818c023050 con 0x55b0ec6aadc0 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 10 client.17400 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b0ec6c3bb0 con 0x55b0ec6aadc0 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f817aff5640 1 -- 172.21.15.143:0/3408233942 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818809bc00 con 0x55b0ec6aadc0 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f817aff5640 1 -- 172.21.15.143:0/3408233942 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f818c012900 con 0x55b0ec6aadc0 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 10 client.17400 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 20 client.17400 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6c5080 0x55b0ec6c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 4 client.17400 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.629+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ec61d850 con 0x55b0ec6c5080 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.630+0000 7f8196d9d640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6c5080 0x55b0ec6c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.630+0000 7f817aff5640 10 client.17400 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.630+0000 7f8196d9d640 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6c5080 0x55b0ec6c7540 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d98450d797c181 server_cookie=bc496b77ff3c544 in_seq=0 out_seq=0 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f817aff5640 1 -- 172.21.15.143:0/3408233942 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0ec61d850 con 0x55b0ec6c5080 2024-09-10T10:53:51.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f817aff5640 10 client.17400 handle_command_reply: tid=0 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 client.17400 shutdown 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 msgr2=0x55b0ec6b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b2160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f818c00e480 tx=0x7f818c00e940 comp rx=0 tx=0).stop 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 shutdown_connections 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6c5080 0x55b0ec6c7540 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b2160 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 --2- 172.21.15.143:0/3408233942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f817aff5640 0 client.17400 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 >> 172.21.15.143:0/3408233942 conn(0x55b0ec6a6a30 msgr2=0x55b0ec6ac760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 shutdown_connections 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 1 -- 172.21.15.143:0/3408233942 wait complete. 2024-09-10T10:53:51.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.631+0000 7f819f418740 20 client.17400 trim_cache size 0 max 16384 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.648+0000 7f819f418740 1 Processor -- start 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.648+0000 7f819f418740 1 -- start start 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819f418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819f418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819f418740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 0x55b0ec6cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819f418740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f81981aeb30 con 0x55b0ec6b5680 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819f418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81981b0790 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819f418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81981af200 con 0x55b0ec6aadc0 2024-09-10T10:53:51.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 0x55b0ec6cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 0x55b0ec6cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53216/0 (socket says 172.21.15.143:53216) 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 -- 172.21.15.143:0/1275910717 learned_addr learned my addr 172.21.15.143:0/1275910717 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f8196d9d640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 -- 172.21.15.143:0/1275910717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6aadc0 msgr2=0x55b0ec6b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819659c640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 -- 172.21.15.143:0/1275910717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 msgr2=0x55b0ec6ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 -- 172.21.15.143:0/1275910717 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ec6a6860 con 0x55b0ec6b5680 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f8196d9d640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819659c640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f819759e640 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 0x55b0ec6cce50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8190035000 tx=0x7f8190041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ad5e1c65463310 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f817cff9640 1 -- 172.21.15.143:0/1275910717 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190031e90 con 0x55b0ec6b5680 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f817cff9640 1 -- 172.21.15.143:0/1275910717 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f819004e070 con 0x55b0ec6b5680 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.649+0000 7f817cff9640 1 -- 172.21.15.143:0/1275910717 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190030be0 con 0x55b0ec6b5680 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 -- 172.21.15.143:0/1275910717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 msgr2=0x55b0ec6cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 0x55b0ec6cce50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8190035000 tx=0x7f8190041040 comp rx=0 tx=0).stop 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 -- 172.21.15.143:0/1275910717 shutdown_connections 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6b5680 0x55b0ec6cce50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 --2- 172.21.15.143:0/1275910717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 -- 172.21.15.143:0/1275910717 >> 172.21.15.143:0/1275910717 conn(0x55b0ec6a6a30 msgr2=0x55b0ec6b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 -- 172.21.15.143:0/1275910717 shutdown_connections 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.650+0000 7f819f418740 1 -- 172.21.15.143:0/1275910717 wait complete. 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 -- start start 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 Processor -- start 2024-09-10T10:53:51.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6bc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f81981b0000 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81981af900 con 0x55b0ec6aadc0 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81981acb10 con 0x55b0ec6b5680 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819f418740 10 client.0 _ll_register_callbacks cb 0x7f818c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53222/0 (socket says 172.21.15.143:53222) 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 -- 172.21.15.143:0/2190928199 learned_addr learned my addr 172.21.15.143:0/2190928199 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819659c640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 -- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 msgr2=0x55b0ec6bc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6bc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 -- 172.21.15.143:0/2190928199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 msgr2=0x55b0ec6af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f8196d9d640 1 -- 172.21.15.143:0/2190928199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ec6aba20 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.651+0000 7f819659c640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f8196d9d640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aee70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f818809b490 tx=0x7f8188000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac2bd898348d568 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f817aff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f817aff5640 1 -- 172.21.15.143:0/2190928199 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188020040 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f817aff5640 1 -- 172.21.15.143:0/2190928199 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f81880cc8b0 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819004e070 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f817aff5640 1 -- 172.21.15.143:0/2190928199 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f818809bc00 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f819f418740 10 client.17406 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b0ec6bdc60 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f817aff5640 1 -- 172.21.15.143:0/2190928199 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f819004e070 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.652+0000 7f817aff5640 1 -- 172.21.15.143:0/2190928199 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f818807a370 con 0x55b0ec6aa4a0 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819f418740 10 client.17406 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819f418740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819f418740 20 client.17406 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819f418740 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6bdfa0 0x55b0ec73aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819f418740 4 client.17406 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0ec6b2a30 con 0x55b0ec6bdfa0 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819659c640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6bdfa0 0x55b0ec73aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f817aff5640 10 client.17406 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.653+0000 7f819659c640 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6bdfa0 0x55b0ec73aa30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb757933cd19663 server_cookie=9553b5f1004f4f61 in_seq=0 out_seq=0 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f817aff5640 1 -- 172.21.15.143:0/2190928199 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0ec6b2a30 con 0x55b0ec6bdfa0 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f817aff5640 10 client.17406 handle_command_reply: tid=0 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 client.17406 shutdown 2024-09-10T10:53:51.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 msgr2=0x55b0ec6aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aee70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f818809b490 tx=0x7f8188000bc0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 shutdown_connections 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55b0ec6bdfa0 0x55b0ec73aa30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f817aff5640 0 client.17406 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b0ec6b5680 0x55b0ec6bc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0ec6aadc0 0x55b0ec6af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 --2- 172.21.15.143:0/2190928199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b0ec6aa4a0 0x55b0ec6aee70 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 >> 172.21.15.143:0/2190928199 conn(0x55b0ec6a6a30 msgr2=0x55b0ec6b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 shutdown_connections 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 1 -- 172.21.15.143:0/2190928199 wait complete. 2024-09-10T10:53:51.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.654+0000 7f819f418740 20 client.17406 trim_cache size 0 max 16384 2024-09-10T10:53:51.663 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:53:51.663 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "no active scrubs running", 2024-09-10T10:53:51.663 INFO:teuthology.orchestra.run.smithi143.stdout: "scrubs": {} 2024-09-10T10:53:51.663 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:53:51.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81800a2b80 msgr2=0x7f81800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81800a2b80 0x7f81800a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8190031040 tx=0x7f8190031b40 comp rx=0 tx=0).stop 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 msgr2=0x7f8198198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 0x7f8198198400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f818800d950 tx=0x7f818800dd50 comp rx=0 tx=0).stop 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 shutdown_connections 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f81800a2b80 0x7f81800a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8198106f10 0x7f819819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8198102790 0x7f8198198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 --2- 172.21.15.143:0/996256483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8198101de0 0x7f8198198400 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 >> 172.21.15.143:0/996256483 conn(0x7f81980fd8c0 msgr2=0x7f81980fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 shutdown_connections 2024-09-10T10:53:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:51.655+0000 7f819d03f640 1 -- 172.21.15.143:0/996256483 wait complete. 2024-09-10T10:53:52.028 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:51 smithi143 ceph-mon[31057]: pgmap v615: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 470 KiB/s wr, 239 op/s 2024-09-10T10:53:52.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:51 smithi178 ceph-mon[34813]: pgmap v615: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 470 KiB/s wr, 239 op/s 2024-09-10T10:53:52.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:51 smithi144 ceph-mon[34230]: pgmap v615: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 470 KiB/s wr, 239 op/s 2024-09-10T10:53:52.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:53:52.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:53:52.626 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 -- 172.21.15.143:0/3825632882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 msgr2=0x7fa9cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/3825632882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc106e60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009930 tx=0x7fa9bc02a420 comp rx=0 tx=0).stop 2024-09-10T10:53:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 -- 172.21.15.143:0/3825632882 shutdown_connections 2024-09-10T10:53:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/3825632882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc101420 0x7fa9cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/3825632882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc100a80 0x7fa9cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/3825632882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc106e60 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 -- 172.21.15.143:0/3825632882 >> 172.21.15.143:0/3825632882 conn(0x7fa9cc0fc7a0 msgr2=0x7fa9cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 -- 172.21.15.143:0/3825632882 shutdown_connections 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.378+0000 7fa9d27d4640 1 -- 172.21.15.143:0/3825632882 wait complete. 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 Processor -- start 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 -- start start 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc100a80 0x7fa9cc0fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc101420 0x7fa9cc0fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc0fd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc110d50 con 0x7fa9cc106a80 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc110bd0 con 0x7fa9cc101420 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d27d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc110ed0 con 0x7fa9cc100a80 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc0fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc0fd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53248/0 (socket says 172.21.15.143:53248) 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 -- 172.21.15.143:0/770261487 learned_addr learned my addr 172.21.15.143:0/770261487 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9cbfff640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc100a80 0x7fa9cc0fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9cb7fe640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc101420 0x7fa9cc0fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 -- 172.21.15.143:0/770261487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc100a80 msgr2=0x7fa9cc0fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc100a80 0x7fa9cc0fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 -- 172.21.15.143:0/770261487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc101420 msgr2=0x7fa9cc0fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc101420 0x7fa9cc0fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9d0d71640 1 -- 172.21.15.143:0/770261487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9cc1a67f0 con 0x7fa9cc106a80 2024-09-10T10:53:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.379+0000 7fa9cbfff640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc100a80 0x7fa9cc0fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9cb7fe640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc101420 0x7fa9cc0fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9d0d71640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc0fd820 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400b970 tx=0x7fa9c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4019070 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9c4014440 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9cc1a6ae0 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40075d0 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.380+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9cc1a6fd0 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.381+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9cc100ee0 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.382+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa9c401c070 con 0x7fa9cc106a80 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.383+0000 7fa9c97fa640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b40a2b80 0x7fa9b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.383+0000 7fa9cbfff640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b40a2b80 0x7fa9b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.383+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa9c40cea20 con 0x7fa9cc106a80 2024-09-10T10:53:53.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.383+0000 7fa9cbfff640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b40a2b80 0x7fa9b40a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc000c00 tx=0x7fa9bc031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.387+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9c409b620 con 0x7fa9cc106a80 2024-09-10T10:53:53.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.589+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9cc10a000 con 0x7fa9cc106a80 2024-09-10T10:53:53.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.590+0000 7fa9c97fa640 1 -- 172.21.15.143:0/770261487 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12014 (secure 0 0 0) 0x7fa9c409af20 con 0x7fa9cc106a80 2024-09-10T10:53:53.591 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:53:53.591 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":42,"btime":"2024-09-10T10:53:46:702117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:53:46.416985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:53:53.592 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 42 2024-09-10T10:53:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b40a2b80 msgr2=0x7fa9b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b40a2b80 0x7fa9b40a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc000c00 tx=0x7fa9bc031cf0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 msgr2=0x7fa9cc0fd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc0fd820 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400b970 tx=0x7fa9c400be30 comp rx=0 tx=0).stop 2024-09-10T10:53:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 shutdown_connections 2024-09-10T10:53:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fa9b40a2b80 0x7fa9b40a5040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc106a80 0x7fa9cc0fd820 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc101420 0x7fa9cc0fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 --2- 172.21.15.143:0/770261487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc100a80 0x7fa9cc0fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 >> 172.21.15.143:0/770261487 conn(0x7fa9cc0fc7a0 msgr2=0x7fa9cc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 shutdown_connections 2024-09-10T10:53:53.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:53.592+0000 7fa9d27d4640 1 -- 172.21.15.143:0/770261487 wait complete. 2024-09-10T10:53:53.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:53 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/770261487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:53.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:53 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/770261487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:54.025 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:53 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/770261487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:53:54.302 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:53:54.549 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:54.910 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:54 smithi143 ceph-mon[31057]: pgmap v616: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 156 KiB/s wr, 178 op/s 2024-09-10T10:53:54.924 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:54 smithi178 ceph-mon[34813]: pgmap v616: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 156 KiB/s wr, 178 op/s 2024-09-10T10:53:54.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:54 smithi144 ceph-mon[34230]: pgmap v616: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 156 KiB/s wr, 178 op/s 2024-09-10T10:53:55.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 -- 172.21.15.143:0/1398075416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4106f10 msgr2=0x7f15d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1398075416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4106f10 0x7f15d410ea90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009ca0 tx=0x7f15cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 -- 172.21.15.143:0/1398075416 shutdown_connections 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1398075416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4106f10 0x7f15d410ea90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1398075416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15d4102790 0x7f15d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1398075416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 0x7f15d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 -- 172.21.15.143:0/1398075416 >> 172.21.15.143:0/1398075416 conn(0x7f15d40fd8c0 msgr2=0x7f15d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 -- 172.21.15.143:0/1398075416 shutdown_connections 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 -- 172.21.15.143:0/1398075416 wait complete. 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 Processor -- start 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.991+0000 7f15daf2d640 1 -- start start 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 0x7f15d4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 0x7f15d4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15d4106f10 0x7f15d419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111e40 con 0x7f15d4102790 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111cc0 con 0x7f15d4101de0 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111fc0 con 0x7f15d4106f10 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 0x7f15d4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 0x7f15d4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53262/0 (socket says 172.21.15.143:53262) 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 -- 172.21.15.143:0/1296182254 learned_addr learned my addr 172.21.15.143:0/1296182254 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 0x7f15d4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d94ca640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15d4106f10 0x7f15d419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 -- 172.21.15.143:0/1296182254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15d4106f10 msgr2=0x7f15d419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15d4106f10 0x7f15d419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 -- 172.21.15.143:0/1296182254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 msgr2=0x7f15d4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 0x7f15d4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 -- 172.21.15.143:0/1296182254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d419d3b0 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 0x7f15d4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d3fff640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 0x7f15d4198940 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f15c000b4a0 tx=0x7f15c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d1ffb640 1 -- 172.21.15.143:0/1296182254 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0014070 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d419d6a0 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d1ffb640 1 -- 172.21.15.143:0/1296182254 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15c0010ac0 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.992+0000 7f15d1ffb640 1 -- 172.21.15.143:0/1296182254 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0017c30 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.993+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41a4f80 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15d1ffb640 1 -- 172.21.15.143:0/1296182254 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f15c0017070 con 0x7f15d4102790 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 Processor -- start 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 -- start start 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a7865c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865c5d0 0x558a7865ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a78669760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558a785c22f0 con 0x558a7865db40 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558a78569cf0 con 0x558a7865c5d0 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15dd306740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558a785be2b0 con 0x558a78667370 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.995+0000 7f15d1ffb640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15c40a2b80 0x7f15c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a7865c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a7865c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53268/0 (socket says 172.21.15.143:53268) 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 -- 172.21.15.143:0/2984248165 learned_addr learned my addr 172.21.15.143:0/2984248165 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15c40a2b80 0x7f15c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865c5d0 0x558a7865ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d3fff640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a78669760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d1ffb640 1 -- 172.21.15.143:0/1296182254 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15c00ce850 con 0x7f15d4102790 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 -- 172.21.15.143:0/2984248165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 msgr2=0x558a78669760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a78669760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 -- 172.21.15.143:0/2984248165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865c5d0 msgr2=0x558a7865ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865c5d0 0x558a7865ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 -- 172.21.15.143:0/2984248165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7854e3a0 con 0x558a7865db40 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d3fff640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a78669760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865c5d0 0x558a7865ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a7865c090 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005fc0 tx=0x7f15cc032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72aa2472b5d7201b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15d94ca640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15c40a2b80 0x7f15c40a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f15cc035000 tx=0x7f15cc043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15beffd640 1 -- 172.21.15.143:0/2984248165 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc04e070 con 0x558a7865db40 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15beffd640 1 -- 172.21.15.143:0/2984248165 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15cc036040 con 0x558a7865db40 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.996+0000 7f15beffd640 1 -- 172.21.15.143:0/2984248165 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0047e0 con 0x558a7865db40 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 -- 172.21.15.143:0/2984248165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 msgr2=0x558a7865c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a7865c090 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005fc0 tx=0x7f15cc032d70 comp rx=0 tx=0).stop 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 -- 172.21.15.143:0/2984248165 shutdown_connections 2024-09-10T10:53:55.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a78669760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865c5d0 0x558a7865ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 --2- 172.21.15.143:0/2984248165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a7865c090 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 -- 172.21.15.143:0/2984248165 >> 172.21.15.143:0/2984248165 conn(0x558a78625310 msgr2=0x558a78658c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 -- 172.21.15.143:0/2984248165 shutdown_connections 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 -- 172.21.15.143:0/2984248165 wait complete. 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.997+0000 7f15dd306740 1 -- start start 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 Processor -- start 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a78666d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a78664040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 0x558a786646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558a7866a010 con 0x558a78667370 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558a7857d060 con 0x558a7865db40 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558a7866a190 con 0x558a7865c5d0 2024-09-10T10:53:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15dd306740 10 client.0 _ll_register_callbacks cb 0x558a782d08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 0x558a786646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 0x558a786646a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53272/0 (socket says 172.21.15.143:53272) 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 -- 172.21.15.143:0/4060208356 learned_addr learned my addr 172.21.15.143:0/4060208356 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d3fff640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a78664040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a78666d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 -- 172.21.15.143:0/4060208356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 msgr2=0x558a78666d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a78666d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 -- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 msgr2=0x558a78664040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a78664040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d94ca640 1 -- 172.21.15.143:0/4060208356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a783333c0 con 0x558a78667370 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.998+0000 7f15d3fff640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a78664040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15d94ca640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 0x558a786646a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f15cc02a900 tx=0x7f15cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4e6bc219d2dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15b3fff640 1 -- 172.21.15.143:0/4060208356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc057070 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15b3fff640 1 -- 172.21.15.143:0/4060208356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15cc031c50 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc036040 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15b3fff640 1 -- 172.21.15.143:0/4060208356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f15cc063050 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 10 client.17424 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558a7866a010 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15b3fff640 1 -- 172.21.15.143:0/4060208356 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15cc036040 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15b3fff640 1 -- 172.21.15.143:0/4060208356 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f15cc032050 con 0x558a78667370 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 10 client.17424 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 10 client.17424 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 20 client.17424 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a78678fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 4 client.17424 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:54.999+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a785cf860 con 0x558a78676be0 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.000+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a78678fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.000+0000 7f15b3fff640 10 client.17424 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.000+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a78678fd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eaa8a56fa16ba391 server_cookie=11a46a213bd625ef in_seq=0 out_seq=0 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15b3fff640 1 -- 172.21.15.143:0/4060208356 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a785cf860 con 0x558a78676be0 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15b3fff640 10 client.17424 handle_command_reply: tid=0 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 client.17424 shutdown 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 msgr2=0x558a786646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 0x558a786646a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f15cc02a900 tx=0x7f15cc004750 comp rx=0 tx=0).stop 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 shutdown_connections 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a78678fd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a78667370 0x558a786646a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a78664040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15b3fff640 0 client.17424 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 --2- 172.21.15.143:0/4060208356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a78666d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 >> 172.21.15.143:0/4060208356 conn(0x558a78625310 msgr2=0x558a7865e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 shutdown_connections 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 1 -- 172.21.15.143:0/4060208356 wait complete. 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.001+0000 7f15dd306740 20 client.17424 trim_cache size 0 max 16384 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 Processor -- start 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 -- start start 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a786732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a78662910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a78667370 0x558a7867e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41aeb70 con 0x558a7865db40 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b07d0 con 0x558a78667370 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15dd306740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41af240 con 0x558a7865c5d0 2024-09-10T10:53:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a78662910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15d3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a78662910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53284/0 (socket says 172.21.15.143:53284) 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.014+0000 7f15d3fff640 1 -- 172.21.15.143:0/3704643192 learned_addr learned my addr 172.21.15.143:0/3704643192 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d3fff640 1 -- 172.21.15.143:0/3704643192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 msgr2=0x558a786732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d3fff640 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a786732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d94ca640 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a78667370 0x558a7867e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d3fff640 1 -- 172.21.15.143:0/3704643192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a78667370 msgr2=0x558a7867e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d3fff640 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a78667370 0x558a7867e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d3fff640 1 -- 172.21.15.143:0/3704643192 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a78664ca0 con 0x558a7865db40 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d94ca640 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a78667370 0x558a7867e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15d3fff640 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a78662910 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15c00971b0 tx=0x7f15c00134b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe864d264c50d7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15be7fc640 1 -- 172.21.15.143:0/3704643192 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c001e050 con 0x558a7865db40 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15be7fc640 1 -- 172.21.15.143:0/3704643192 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15c009b450 con 0x558a7865db40 2024-09-10T10:53:55.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15be7fc640 1 -- 172.21.15.143:0/3704643192 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00973c0 con 0x558a7865db40 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 -- 172.21.15.143:0/3704643192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 msgr2=0x558a78662910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a78662910 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15c00971b0 tx=0x7f15c00134b0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 -- 172.21.15.143:0/3704643192 shutdown_connections 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a78667370 0x558a7867e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865db40 0x558a78662910 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 --2- 172.21.15.143:0/3704643192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a7865c5d0 0x558a786732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 -- 172.21.15.143:0/3704643192 >> 172.21.15.143:0/3704643192 conn(0x558a78625310 msgr2=0x558a7865e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.015+0000 7f15dd306740 1 -- 172.21.15.143:0/3704643192 shutdown_connections 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 -- 172.21.15.143:0/3704643192 wait complete. 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 -- start start 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 Processor -- start 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865c5d0 0x558a7866b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a7866b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a7866bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b0040 con 0x558a7865c5d0 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41af940 con 0x558a7865db40 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41acb50 con 0x558a78667370 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15dd306740 10 client.0 _ll_register_callbacks cb 0x7f15cc032bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15d8cc9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865c5d0 0x558a7866b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15d8cc9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865c5d0 0x558a7866b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53292/0 (socket says 172.21.15.143:53292) 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.016+0000 7f15d8cc9640 1 -- 172.21.15.143:0/900996283 learned_addr learned my addr 172.21.15.143:0/900996283 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a7866b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d94ca640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a7866bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 -- 172.21.15.143:0/900996283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 msgr2=0x558a7866bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a7866bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 -- 172.21.15.143:0/900996283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865c5d0 msgr2=0x558a7866b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865c5d0 0x558a7866b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 -- 172.21.15.143:0/900996283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a786e8f70 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15d3fff640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a7866b740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15c0017f80 tx=0x7f15c00a3a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1cc9a3f9830009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15b3fff640 1 -- 172.21.15.143:0/900996283 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00973c0 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c009b450 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15b3fff640 1 -- 172.21.15.143:0/900996283 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15c0010c70 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15b3fff640 1 -- 172.21.15.143:0/900996283 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f15c001e070 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15dd306740 10 client.26648 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.017+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558a786ea210 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15b3fff640 1 -- 172.21.15.143:0/900996283 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15c009b450 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15b3fff640 1 -- 172.21.15.143:0/900996283 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f15c0062060 con 0x558a7865db40 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15dd306740 10 client.26648 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15dd306740 10 client.26648 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-10T10:53:55.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15dd306740 20 client.26648 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15dd306740 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a786ec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15dd306740 4 client.26648 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 --> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a785c9d70 con 0x558a78676be0 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.018+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a786ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15d8cc9640 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a786ec920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9011123e6147a0e3 server_cookie=9f4465dc89e7f7cc in_seq=0 out_seq=0 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15b3fff640 10 client.26648 ms_handle_connect on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15b3fff640 1 -- 172.21.15.143:0/900996283 <== mds.4 v2:172.21.15.144:6836/3174266316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a785c9d70 con 0x558a78676be0 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15b3fff640 10 client.26648 handle_command_reply: tid=0 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 client.26648 shutdown 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 msgr2=0x558a7866b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a7866b740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15c0017f80 tx=0x7f15c00a3a40 comp rx=0 tx=0).stop 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 shutdown_connections 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:6836/3174266316,v1:172.21.15.144:6837/3174266316] conn(0x558a78676be0 0x558a786ec920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a78667370 0x558a7866bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15b3fff640 0 client.26648 ms_handle_reset on v2:172.21.15.144:6836/3174266316 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a7865db40 0x558a7866b740 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 --2- 172.21.15.143:0/900996283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558a7865c5d0 0x558a7866b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 >> 172.21.15.143:0/900996283 conn(0x558a78625310 msgr2=0x558a7865e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 shutdown_connections 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 1 -- 172.21.15.143:0/900996283 wait complete. 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.019+0000 7f15dd306740 20 client.26648 trim_cache size 0 max 16384 2024-09-10T10:53:55.036 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15c40a2b80 msgr2=0x7f15c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15c40a2b80 0x7f15c40a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f15cc035000 tx=0x7f15cc043000 comp rx=0 tx=0).stop 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 msgr2=0x7f15d4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 0x7f15d4198940 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f15c000b4a0 tx=0x7f15c000b960 comp rx=0 tx=0).stop 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 shutdown_connections 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15c40a2b80 0x7f15c40a5040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15d4106f10 0x7f15d419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15d4102790 0x7f15d4198940 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 --2- 172.21.15.143:0/1296182254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15d4101de0 0x7f15d4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:55.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 >> 172.21.15.143:0/1296182254 conn(0x7f15d40fd8c0 msgr2=0x7f15d40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:55.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 shutdown_connections 2024-09-10T10:53:55.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:55.020+0000 7f15daf2d640 1 -- 172.21.15.143:0/1296182254 wait complete. 2024-09-10T10:53:55.710 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:53:55.955 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 -- 172.21.15.143:0/3358890644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 msgr2=0x7f196c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 --2- 172.21.15.143:0/3358890644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c100ee0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1958009930 tx=0x7f195802a420 comp rx=0 tx=0).stop 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 -- 172.21.15.143:0/3358890644 shutdown_connections 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 --2- 172.21.15.143:0/3358890644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c101420 0x7f196c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 --2- 172.21.15.143:0/3358890644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c100ee0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 --2- 172.21.15.143:0/3358890644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196c106a80 0x7f196c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 -- 172.21.15.143:0/3358890644 >> 172.21.15.143:0/3358890644 conn(0x7f196c0fc7a0 msgr2=0x7f196c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 -- 172.21.15.143:0/3358890644 shutdown_connections 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.440+0000 7f1972cd0640 1 -- 172.21.15.143:0/3358890644 wait complete. 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 Processor -- start 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 -- start start 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196c101420 0x7f196c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c106a80 0x7f196c1a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f196c110cf0 con 0x7f196c100a80 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f196c110b70 con 0x7f196c106a80 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1972cd0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f196c110e70 con 0x7f196c101420 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38020/0 (socket says 172.21.15.143:38020) 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 -- 172.21.15.143:0/4030936798 learned_addr learned my addr 172.21.15.143:0/4030936798 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f196bfff640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196c101420 0x7f196c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f197126d640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c106a80 0x7f196c1a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 -- 172.21.15.143:0/4030936798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196c101420 msgr2=0x7f196c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196c101420 0x7f196c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 -- 172.21.15.143:0/4030936798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c106a80 msgr2=0x7f196c1a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.441+0000 7f1970a6c640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c106a80 0x7f196c1a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1970a6c640 1 -- 172.21.15.143:0/4030936798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c1a4700 con 0x7f196c100a80 2024-09-10T10:53:56.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f197126d640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c106a80 0x7f196c1a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1970a6c640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c199140 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f196000c920 tx=0x7f196000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1969ffb640 1 -- 172.21.15.143:0/4030936798 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960007c70 con 0x7f196c100a80 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1969ffb640 1 -- 172.21.15.143:0/4030936798 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1960007e10 con 0x7f196c100a80 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c1a48d0 con 0x7f196c100a80 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1969ffb640 1 -- 172.21.15.143:0/4030936798 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960018820 con 0x7f196c100a80 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.442+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1a4de0 con 0x7f196c100a80 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.444+0000 7f1969ffb640 1 -- 172.21.15.143:0/4030936798 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f19600189c0 con 0x7f196c100a80 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 Processor -- start 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 -- start start 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f1969ffb640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f195c0a2b80 0x7f195c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dab9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dab9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f197126d640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f195c0a2b80 0x7f195c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f1969ffb640 1 -- 172.21.15.143:0/4030936798 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f19600cf560 con 0x7f196c100a80 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dab9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b22da1f2f0 con 0x55b22dab95d0 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b22d9c6cf0 con 0x55b22dabab40 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f19750a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b22da1b2b0 con 0x55b22dac4370 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dab9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38034/0 (socket says 172.21.15.143:38034) 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 -- 172.21.15.143:0/3304106622 learned_addr learned my addr 172.21.15.143:0/3304106622 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f1970a6c640 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dab9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 -- 172.21.15.143:0/3304106622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 msgr2=0x55b22dac6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f197126d640 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 -- 172.21.15.143:0/3304106622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 msgr2=0x55b22dab9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.445+0000 7f196bfff640 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dab9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f196bfff640 1 -- 172.21.15.143:0/3304106622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22d9ab3a0 con 0x55b22dab95d0 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f197126d640 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f197126d640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f195c0a2b80 0x7f195c0a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1964005fd0 tx=0x7f196400f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f196bfff640 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dab9a10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1958009930 tx=0x7f1958032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93cbf93019ad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f1956ffd640 1 -- 172.21.15.143:0/3304106622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195803f070 con 0x55b22dab95d0 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f1956ffd640 1 -- 172.21.15.143:0/3304106622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19580316a0 con 0x55b22dab95d0 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.446+0000 7f1956ffd640 1 -- 172.21.15.143:0/3304106622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958035070 con 0x55b22dab95d0 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- 172.21.15.143:0/3304106622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 msgr2=0x55b22dab9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dab9a10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1958009930 tx=0x7f1958032cc0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- 172.21.15.143:0/3304106622 shutdown_connections 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dab9a10 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- 172.21.15.143:0/3304106622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dab9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- 172.21.15.143:0/3304106622 >> 172.21.15.143:0/3304106622 conn(0x55b22da82310 msgr2=0x55b22dab5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- 172.21.15.143:0/3304106622 shutdown_connections 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- 172.21.15.143:0/3304106622 wait complete. 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- start start 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 Processor -- start 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dac3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dac1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b22dac7010 con 0x55b22dab95d0 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b22d9da060 con 0x55b22dabab40 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b22dac7190 con 0x55b22dac4370 2024-09-10T10:53:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.447+0000 7f19750a9740 10 client.0 _ll_register_callbacks cb 0x55b22d72d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dac3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dac3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38046/0 (socket says 172.21.15.143:38046) 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 -- 172.21.15.143:0/2119835904 learned_addr learned my addr 172.21.15.143:0/2119835904 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f197126d640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f196bfff640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dac1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 -- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 msgr2=0x55b22dac1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 -- 172.21.15.143:0/2119835904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 msgr2=0x55b22dac1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dac1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 -- 172.21.15.143:0/2119835904 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22d7903c0 con 0x55b22dab95d0 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f196bfff640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dac1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f1970a6c640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dac3d90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f196009b760 tx=0x7f196000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c558bb8f73a400e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:56.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f194bfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f194bfff640 1 -- 172.21.15.143:0/2119835904 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196009fe10 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f194bfff640 1 -- 172.21.15.143:0/2119835904 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f196009bcc0 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19580316a0 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f194bfff640 1 -- 172.21.15.143:0/2119835904 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f19600a46d0 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f19750a9740 10 client.17442 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b22dad2a50 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.448+0000 7f194bfff640 1 -- 172.21.15.143:0/2119835904 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19580316a0 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f194bfff640 1 -- 172.21.15.143:0/2119835904 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f19600977b0 con 0x55b22dab95d0 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f19750a9740 10 client.17442 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f19750a9740 10 client.17442 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f19750a9740 20 client.17442 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f19750a9740 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22dad3eb0 0x55b22dad6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f19750a9740 4 client.17442 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:56.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22da2c860 con 0x55b22dad3eb0 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f196bfff640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22dad3eb0 0x55b22dad6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f196bfff640 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22dad3eb0 0x55b22dad6300 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bac0f0bab70f283 server_cookie=ce5fe06ab790c276 in_seq=0 out_seq=0 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.449+0000 7f194bfff640 10 client.17442 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f194bfff640 1 -- 172.21.15.143:0/2119835904 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b22da2c860 con 0x55b22dad3eb0 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f194bfff640 10 client.17442 handle_command_reply: tid=0 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 client.17442 shutdown 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 msgr2=0x55b22dac3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dac3d90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f196009b760 tx=0x7f196000fb00 comp rx=0 tx=0).stop 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 shutdown_connections 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22dad3eb0 0x55b22dad6300 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22dac1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f194bfff640 0 client.17442 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dabab40 0x55b22dac1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 --2- 172.21.15.143:0/2119835904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dab95d0 0x55b22dac3d90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 >> 172.21.15.143:0/2119835904 conn(0x55b22da82310 msgr2=0x55b22dabb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 shutdown_connections 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.450+0000 7f19750a9740 1 -- 172.21.15.143:0/2119835904 wait complete. 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.451+0000 7f19750a9740 20 client.17442 trim_cache size 0 max 16384 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 Processor -- start 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 -- start start 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dab95d0 0x55b22dad02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dacc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22daccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1ae8e0 con 0x55b22dabab40 2024-09-10T10:53:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1b0540 con 0x55b22dab95d0 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.463+0000 7f19750a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1aefb0 con 0x55b22dac4370 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dacc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dacc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38062/0 (socket says 172.21.15.143:38062) 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 -- 172.21.15.143:0/1570473869 learned_addr learned my addr 172.21.15.143:0/1570473869 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f1970a6c640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dab95d0 0x55b22dad02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 -- 172.21.15.143:0/1570473869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 msgr2=0x55b22daccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f197126d640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22daccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22daccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 -- 172.21.15.143:0/1570473869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dab95d0 msgr2=0x55b22dad02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dab95d0 0x55b22dad02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 -- 172.21.15.143:0/1570473869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22dabf590 con 0x55b22dabab40 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f197126d640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22daccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:56.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f1970a6c640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dab95d0 0x55b22dad02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f196bfff640 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dacc800 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1958009930 tx=0x7f1958031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dcb235c1009178 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f19567fc640 1 -- 172.21.15.143:0/1570473869 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958051070 con 0x55b22dabab40 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f19567fc640 1 -- 172.21.15.143:0/1570473869 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19580035f0 con 0x55b22dabab40 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.464+0000 7f19567fc640 1 -- 172.21.15.143:0/1570473869 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195803f040 con 0x55b22dabab40 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 -- 172.21.15.143:0/1570473869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 msgr2=0x55b22dacc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dacc800 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1958009930 tx=0x7f1958031830 comp rx=0 tx=0).stop 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 -- 172.21.15.143:0/1570473869 shutdown_connections 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dac4370 0x55b22daccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dacc800 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 --2- 172.21.15.143:0/1570473869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dab95d0 0x55b22dad02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 -- 172.21.15.143:0/1570473869 >> 172.21.15.143:0/1570473869 conn(0x55b22da82310 msgr2=0x55b22dac6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 -- 172.21.15.143:0/1570473869 shutdown_connections 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 -- 172.21.15.143:0/1570473869 wait complete. 2024-09-10T10:53:56.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 -- start start 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.465+0000 7f19750a9740 1 Processor -- start 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dab95d0 0x55b22dadb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dadbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dac4370 0x55b22dadc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1afdb0 con 0x55b22dabab40 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1af6b0 con 0x55b22dac4370 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1ac8c0 con 0x55b22dab95d0 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f19750a9740 10 client.0 _ll_register_callbacks cb 0x7f196009b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dadbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dadbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38068/0 (socket says 172.21.15.143:38068) 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 -- 172.21.15.143:0/2686488033 learned_addr learned my addr 172.21.15.143:0/2686488033 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f197126d640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dac4370 0x55b22dadc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f1970a6c640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dab95d0 0x55b22dadb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 -- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dab95d0 msgr2=0x55b22dadb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dab95d0 0x55b22dadb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 -- 172.21.15.143:0/2686488033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dac4370 msgr2=0x55b22dadc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dac4370 0x55b22dadc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 -- 172.21.15.143:0/2686488033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22db41bd0 con 0x55b22dabab40 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f196bfff640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dadbe80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f195802a940 tx=0x7f1958000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546736f4b6f69eff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f194bfff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f194bfff640 1 -- 172.21.15.143:0/2686488033 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195803f050 con 0x55b22dabab40 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.466+0000 7f194bfff640 1 -- 172.21.15.143:0/2686488033 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f195804c5d0 con 0x55b22dabab40 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19580035f0 con 0x55b22dabab40 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f194bfff640 1 -- 172.21.15.143:0/2686488033 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1958062050 con 0x55b22dabab40 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 10 client.17448 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:56.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b22db42de0 con 0x55b22dabab40 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f194bfff640 1 -- 172.21.15.143:0/2686488033 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19580035f0 con 0x55b22dabab40 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f194bfff640 1 -- 172.21.15.143:0/2686488033 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f195803b880 con 0x55b22dabab40 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 10 client.17448 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 10 client.17448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 20 client.17448 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22db43110 0x55b22db45500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 4 client.17448 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 --> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b22da26d70 con 0x55b22db43110 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.467+0000 7f1970a6c640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22db43110 0x55b22db45500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f194bfff640 10 client.17448 ms_handle_connect on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f1970a6c640 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22db43110 0x55b22db45500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=428402e8c7a71b88 server_cookie=f5b4b4927ea6f804 in_seq=0 out_seq=0 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f194bfff640 1 -- 172.21.15.143:0/2686488033 <== mds.3 v2:172.21.15.178:6834/261851614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b22da26d70 con 0x55b22db43110 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f194bfff640 10 client.17448 handle_command_reply: tid=0 2024-09-10T10:53:56.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 client.17448 shutdown 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 msgr2=0x55b22dadbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dadbe80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f195802a940 tx=0x7f1958000bc0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 shutdown_connections 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:6834/261851614,v1:172.21.15.178:6835/261851614] conn(0x55b22db43110 0x55b22db45500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b22dac4370 0x55b22dadc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b22dabab40 0x55b22dadbe80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 --2- 172.21.15.143:0/2686488033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b22dab95d0 0x55b22dadb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 >> 172.21.15.143:0/2686488033 conn(0x55b22da82310 msgr2=0x55b22dabb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f194bfff640 0 client.17448 ms_handle_reset on v2:172.21.15.178:6834/261851614 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 shutdown_connections 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 1 -- 172.21.15.143:0/2686488033 wait complete. 2024-09-10T10:53:56.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.468+0000 7f19750a9740 20 client.17448 trim_cache size 0 max 16384 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f195c0a2b80 msgr2=0x7f195c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f195c0a2b80 0x7f195c0a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1964005fd0 tx=0x7f196400f730 comp rx=0 tx=0).stop 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 msgr2=0x7f196c199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c199140 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f196000c920 tx=0x7f196000cde0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 shutdown_connections 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f195c0a2b80 0x7f195c0a5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f196c106a80 0x7f196c1a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196c101420 0x7f196c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 --2- 172.21.15.143:0/4030936798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f196c100a80 0x7f196c199140 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:56.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 >> 172.21.15.143:0/4030936798 conn(0x7f196c0fc7a0 msgr2=0x7f196c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:56.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 shutdown_connections 2024-09-10T10:53:56.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:56.469+0000 7f1972cd0640 1 -- 172.21.15.143:0/4030936798 wait complete. 2024-09-10T10:53:56.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:56 smithi178 ceph-mon[34813]: pgmap v617: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 54 op/s 2024-09-10T10:53:56.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:56 smithi144 ceph-mon[34230]: pgmap v617: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 54 op/s 2024-09-10T10:53:57.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:56 smithi143 ceph-mon[31057]: pgmap v617: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 54 op/s 2024-09-10T10:53:57.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:53:57.385 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 -- 172.21.15.143:0/2072986663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8102790 msgr2=0x7f07b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 --2- 172.21.15.143:0/2072986663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8102790 0x7f07b81067e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009900 tx=0x7f07a802a410 comp rx=0 tx=0).stop 2024-09-10T10:53:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 -- 172.21.15.143:0/2072986663 shutdown_connections 2024-09-10T10:53:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 --2- 172.21.15.143:0/2072986663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07b8106f10 0x7f07b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 --2- 172.21.15.143:0/2072986663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8102790 0x7f07b81067e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 --2- 172.21.15.143:0/2072986663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8101de0 0x7f07b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 -- 172.21.15.143:0/2072986663 >> 172.21.15.143:0/2072986663 conn(0x7f07b80fd8c0 msgr2=0x7f07b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 -- 172.21.15.143:0/2072986663 shutdown_connections 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.873+0000 7f07bce96640 1 -- 172.21.15.143:0/2072986663 wait complete. 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 Processor -- start 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 -- start start 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07b8101de0 0x7f07b8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8102790 0x7f07b8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 0x7f07b819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111e40 con 0x7f07b8106f10 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111cc0 con 0x7f07b8102790 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07bce96640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111fc0 con 0x7f07b8101de0 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.874+0000 7f07b6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 0x7f07b819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 0x7f07b819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38094/0 (socket says 172.21.15.143:38094) 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1768769015 learned_addr learned my addr 172.21.15.143:0/1768769015 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8102790 0x7f07b8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1768769015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07b8101de0 msgr2=0x7f07b8198320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07b8101de0 0x7f07b8198320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1768769015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8102790 msgr2=0x7f07b8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8102790 0x7f07b8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1768769015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b819d2d0 con 0x7f07b8106f10 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 0x7f07b819cbf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b970 tx=0x7f07b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07af7fe640 1 -- 172.21.15.143:0/1768769015 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0019070 con 0x7f07b8106f10 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07af7fe640 1 -- 172.21.15.143:0/1768769015 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07b0014440 con 0x7f07b8106f10 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b819d5c0 con 0x7f07b8106f10 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.875+0000 7f07af7fe640 1 -- 172.21.15.143:0/1768769015 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00075d0 con 0x7f07b8106f10 2024-09-10T10:53:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.876+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a4ea0 con 0x7f07b8106f10 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.878+0000 7f07af7fe640 1 -- 172.21.15.143:0/1768769015 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f07b001c070 con 0x7f07b8106f10 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.878+0000 7f07bf26f740 1 Processor -- start 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 -- start start 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07af7fe640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07a40a2c00 0x7f07a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5649884162f0 con 0x5649884bf780 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5649883bdcf0 con 0x5649884b5f50 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07bf26f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649884122b0 con 0x5649884b49e0 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07af7fe640 1 -- 172.21.15.143:0/1768769015 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07b00ce8b0 con 0x7f07b8106f10 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07a40a2c00 0x7f07a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.143:49748/0 (socket says 172.21.15.143:49748) 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b659c640 1 -- 172.21.15.143:0/102496018 learned_addr learned my addr 172.21.15.143:0/102496018 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b659c640 1 -- 172.21.15.143:0/102496018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 msgr2=0x5649884b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.879+0000 7f07b659c640 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07b659c640 1 -- 172.21.15.143:0/102496018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 msgr2=0x5649884c1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07b659c640 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07b659c640 1 -- 172.21.15.143:0/102496018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649883a23a0 con 0x5649884b5f50 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07a40a2c00 0x7f07a40a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f07a8031440 tx=0x7f07a8009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07b659c640 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884b44a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f07a000aa20 tx=0x7f07a000aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0a052e9b8ea621 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07acff9640 1 -- 172.21.15.143:0/102496018 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a000ce40 con 0x5649884b5f50 2024-09-10T10:53:57.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07acff9640 1 -- 172.21.15.143:0/102496018 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07a000f900 con 0x5649884b5f50 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.880+0000 7f07acff9640 1 -- 172.21.15.143:0/102496018 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00189d0 con 0x5649884b5f50 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 -- 172.21.15.143:0/102496018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 msgr2=0x5649884b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884b44a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f07a000aa20 tx=0x7f07a000aee0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 -- 172.21.15.143:0/102496018 shutdown_connections 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 --2- 172.21.15.143:0/102496018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884b44a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 -- 172.21.15.143:0/102496018 >> 172.21.15.143:0/102496018 conn(0x564988479310 msgr2=0x5649884b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 -- 172.21.15.143:0/102496018 shutdown_connections 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 -- 172.21.15.143:0/102496018 wait complete. 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 -- start start 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.881+0000 7f07bf26f740 1 Processor -- start 2024-09-10T10:53:57.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5649884c2480 con 0x5649884bf780 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5649883d1060 con 0x5649884b5f50 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649884c2600 con 0x5649884b49e0 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 10 client.0 _ll_register_callbacks cb 0x5649881248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.914 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884bc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38112/0 (socket says 172.21.15.143:38112) 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1941647404 learned_addr learned my addr 172.21.15.143:0/1941647404 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b659c640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 msgr2=0x5649884bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1941647404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 msgr2=0x5649884bc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884bc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 -- 172.21.15.143:0/1941647404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649881873c0 con 0x5649884bf780 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b659c640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:57.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884bc9e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f07b00cc050 tx=0x7f07b009afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c468cbea66a70998 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f079dffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f079dffb640 1 -- 172.21.15.143:0/1941647404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a3790 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a000f900 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.882+0000 7f079dffb640 1 -- 172.21.15.143:0/1941647404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07b00b9550 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f079dffb640 1 -- 172.21.15.143:0/1941647404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f07b0099220 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 10 client.17460 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5649884cdf90 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f079dffb640 1 -- 172.21.15.143:0/1941647404 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07a000f900 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f079dffb640 1 -- 172.21.15.143:0/1941647404 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f07b0096bb0 con 0x5649884bf780 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 10 client.17460 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 10 client.17460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 20 client.17460 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:57.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5649884cf3a0 0x5649884d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 4 client.17460 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.883+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564988423860 con 0x5649884cf3a0 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.884+0000 7f07b659c640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5649884cf3a0 0x5649884d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.884+0000 7f079dffb640 10 client.17460 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.884+0000 7f07b659c640 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5649884cf3a0 0x5649884d1790 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=908a76b222c5c542 server_cookie=9b492e7955f20673 in_seq=0 out_seq=0 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f079dffb640 1 -- 172.21.15.143:0/1941647404 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564988423860 con 0x5649884cf3a0 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f079dffb640 10 client.17460 handle_command_reply: tid=0 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 client.17460 shutdown 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 msgr2=0x5649884bc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884bc9e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f07b00cc050 tx=0x7f07b009afa0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 shutdown_connections 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x5649884cf3a0 0x5649884d1790 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884bc9e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f079dffb640 0 client.17460 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1941647404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 >> 172.21.15.143:0/1941647404 conn(0x564988479310 msgr2=0x5649884b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 shutdown_connections 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 1 -- 172.21.15.143:0/1941647404 wait complete. 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.885+0000 7f07bf26f740 20 client.17460 trim_cache size 0 max 16384 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 Processor -- start 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 -- start start 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 0x5649884be360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884bf780 0x5649884b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81ac920 con 0x5649884b5f50 2024-09-10T10:53:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81ae580 con 0x5649884bf780 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07bf26f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81acff0 con 0x5649884b49e0 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.898+0000 7f07b5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 0x5649884be360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 0x5649884be360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38128/0 (socket says 172.21.15.143:38128) 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 -- 172.21.15.143:0/1573067484 learned_addr learned my addr 172.21.15.143:0/1573067484 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b659c640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884bf780 0x5649884b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 -- 172.21.15.143:0/1573067484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 msgr2=0x5649884bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 -- 172.21.15.143:0/1573067484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884bf780 msgr2=0x5649884b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884bf780 0x5649884b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 -- 172.21.15.143:0/1573067484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649884bb3a0 con 0x5649884b5f50 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b659c640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884bf780 0x5649884b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 0x5649884be360 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f07a802a930 tx=0x7f07a8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d465d2c781db25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f079ffff640 1 -- 172.21.15.143:0/1573067484 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8055070 con 0x5649884b5f50 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f079ffff640 1 -- 172.21.15.143:0/1573067484 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07a80043c0 con 0x5649884b5f50 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.899+0000 7f079ffff640 1 -- 172.21.15.143:0/1573067484 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8035040 con 0x5649884b5f50 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 -- 172.21.15.143:0/1573067484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 msgr2=0x5649884be360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 0x5649884be360 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f07a802a930 tx=0x7f07a8045000 comp rx=0 tx=0).stop 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 -- 172.21.15.143:0/1573067484 shutdown_connections 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884bf780 0x5649884b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884b5f50 0x5649884be360 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 --2- 172.21.15.143:0/1573067484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 -- 172.21.15.143:0/1573067484 >> 172.21.15.143:0/1573067484 conn(0x564988479310 msgr2=0x5649884bdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 -- 172.21.15.143:0/1573067484 shutdown_connections 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 -- 172.21.15.143:0/1573067484 wait complete. 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.900+0000 7f07bf26f740 1 -- start start 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 Processor -- start 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81addf0 con 0x5649884bf780 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81ad6f0 con 0x5649884b5f50 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81aa900 con 0x5649884b49e0 2024-09-10T10:53:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07bf26f740 10 client.0 _ll_register_callbacks cb 0x7f07b0014a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38142/0 (socket says 172.21.15.143:38142) 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 -- 172.21.15.143:0/3277697972 learned_addr learned my addr 172.21.15.143:0/3277697972 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b5d9b640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b659c640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 -- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 msgr2=0x5649884c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 -- 172.21.15.143:0/3277697972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 msgr2=0x5649884c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 -- 172.21.15.143:0/3277697972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649884c39d0 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b659c640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f07b6d9d640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c31a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f07b0097830 tx=0x7f07b009a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462449f158550135 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.901+0000 7f079dffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f079dffb640 1 -- 172.21.15.143:0/3277697972 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00b9750 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f079dffb640 1 -- 172.21.15.143:0/3277697972 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f07b00a3590 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a80043c0 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f079dffb640 1 -- 172.21.15.143:0/3277697972 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f07b0096dc0 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 10 client.17466 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564988539c70 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f079dffb640 1 -- 172.21.15.143:0/3277697972 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07a80043c0 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f079dffb640 1 -- 172.21.15.143:0/3277697972 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f07b0099d20 con 0x5649884bf780 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 10 client.17466 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 10 client.17466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 20 client.17466 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x56498853a330 0x56498853c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 4 client.17466 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.902+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 --> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56498841dd70 con 0x56498853a330 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07b659c640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x56498853a330 0x56498853c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07b659c640 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x56498853a330 0x56498853c750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66165403ef74061e server_cookie=1cf51b24cbbe270d in_seq=0 out_seq=0 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f079dffb640 10 client.17466 ms_handle_connect on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f079dffb640 1 -- 172.21.15.143:0/3277697972 <== mds.2 v2:172.21.15.178:6836/2330516637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56498841dd70 con 0x56498853a330 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f079dffb640 10 client.17466 handle_command_reply: tid=0 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 client.17466 shutdown 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 msgr2=0x5649884c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c31a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f07b0097830 tx=0x7f07b009a9a0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 shutdown_connections 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:6836/2330516637,v1:172.21.15.178:6837/2330516637] conn(0x56498853a330 0x56498853c750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5649884bf780 0x5649884c31a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f079dffb640 0 client.17466 ms_handle_reset on v2:172.21.15.178:6836/2330516637 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649884b5f50 0x5649884c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 --2- 172.21.15.143:0/3277697972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649884b49e0 0x5649884c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 >> 172.21.15.143:0/3277697972 conn(0x564988479310 msgr2=0x5649884ab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.903+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 shutdown_connections 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bf26f740 1 -- 172.21.15.143:0/3277697972 wait complete. 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bf26f740 20 client.17466 trim_cache size 0 max 16384 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07a40a2c00 msgr2=0x7f07a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07a40a2c00 0x7f07a40a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f07a8031440 tx=0x7f07a8009560 comp rx=0 tx=0).stop 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 msgr2=0x7f07b819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 0x7f07b819cbf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b970 tx=0x7f07b000be30 comp rx=0 tx=0).stop 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 shutdown_connections 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f07a40a2c00 0x7f07a40a50c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07b8106f10 0x7f07b819cbf0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8102790 0x7f07b8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 --2- 172.21.15.143:0/1768769015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07b8101de0 0x7f07b8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 >> 172.21.15.143:0/1768769015 conn(0x7f07b80fd8c0 msgr2=0x7f07b80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 shutdown_connections 2024-09-10T10:53:57.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:57.904+0000 7f07bce96640 1 -- 172.21.15.143:0/1768769015 wait complete. 2024-09-10T10:53:58.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:53:58.800 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:53:58 smithi143 ceph-mon[31057]: pgmap v618: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 39 op/s 2024-09-10T10:53:58.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:53:58 smithi178 ceph-mon[34813]: pgmap v618: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 39 op/s 2024-09-10T10:53:58.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:53:58 smithi144 ceph-mon[34230]: pgmap v618: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 39 op/s 2024-09-10T10:53:59.029 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:53:59.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.541+0000 7fec0bf46640 1 -- 172.21.15.143:0/2651088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102ef0 msgr2=0x7fec0410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.541+0000 7fec0bf46640 1 --2- 172.21.15.143:0/2651088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102ef0 0x7fec0410da40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7febfc009ca0 tx=0x7febfc02a950 comp rx=0 tx=0).stop 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 -- 172.21.15.143:0/2651088906 shutdown_connections 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 --2- 172.21.15.143:0/2651088906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102ef0 0x7fec0410da40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 --2- 172.21.15.143:0/2651088906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102570 0x7fec041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 --2- 172.21.15.143:0/2651088906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec04100e20 0x7fec04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 -- 172.21.15.143:0/2651088906 >> 172.21.15.143:0/2651088906 conn(0x7fec040fc7d0 msgr2=0x7fec040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 -- 172.21.15.143:0/2651088906 shutdown_connections 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 -- 172.21.15.143:0/2651088906 wait complete. 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 Processor -- start 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.542+0000 7fec0bf46640 1 -- start start 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec04100e20 0x7fec041972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 0x7fec041977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102ef0 0x7fec0419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec04110e30 con 0x7fec04102570 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec04110cb0 con 0x7fec04102ef0 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec04110fb0 con 0x7fec04100e20 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 0x7fec041977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 0x7fec041977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38154/0 (socket says 172.21.15.143:38154) 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 -- 172.21.15.143:0/1924409507 learned_addr learned my addr 172.21.15.143:0/1924409507 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec09ce2640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec04100e20 0x7fec041972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102ef0 0x7fec0419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 -- 172.21.15.143:0/1924409507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec04100e20 msgr2=0x7fec041972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec04100e20 0x7fec041972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 -- 172.21.15.143:0/1924409507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102ef0 msgr2=0x7fec0419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102ef0 0x7fec0419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 -- 172.21.15.143:0/1924409507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0419c250 con 0x7fec04102570 2024-09-10T10:53:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102ef0 0x7fec0419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec094e1640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 0x7fec041977e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7febf4009970 tx=0x7febf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7febfaffd640 1 -- 172.21.15.143:0/1924409507 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf400d020 con 0x7fec04102570 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7febfaffd640 1 -- 172.21.15.143:0/1924409507 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febf400d1c0 con 0x7fec04102570 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0419c540 con 0x7fec04102570 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.543+0000 7febfaffd640 1 -- 172.21.15.143:0/1924409507 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf400cc50 con 0x7fec04102570 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.544+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec04069f70 con 0x7fec04102570 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.546+0000 7febfaffd640 1 -- 172.21.15.143:0/1924409507 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7febf400c460 con 0x7fec04102570 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.546+0000 7fec0e31f740 1 Processor -- start 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.546+0000 7fec0e31f740 1 -- start start 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b98b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0e31f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a004aed2f0 con 0x55a004b96780 2024-09-10T10:53:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0e31f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a004a94cf0 con 0x55a004b875d0 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0e31f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a004ae92b0 con 0x55a004b88b40 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7febfaffd640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7febf00a2b80 0x7febf00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec094e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b98b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec09ce2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec094e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b98b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38168/0 (socket says 172.21.15.143:38168) 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec094e1640 1 -- 172.21.15.143:0/4203270535 learned_addr learned my addr 172.21.15.143:0/4203270535 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7febf00a2b80 0x7febf00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7febfaffd640 1 -- 172.21.15.143:0/1924409507 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7febf40d0850 con 0x7fec04102570 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 -- 172.21.15.143:0/4203270535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 msgr2=0x55a004b87a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b87a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 -- 172.21.15.143:0/4203270535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 msgr2=0x55a004b98b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b98b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.547+0000 7fec0a4e3640 1 -- 172.21.15.143:0/4203270535 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a004a793a0 con 0x55a004b88b40 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7febf00a2b80 0x7febf00a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7febfc009ca0 tx=0x7febfc042000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b87090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc02f660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dd4ea3453655366 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7febeffff640 1 -- 172.21.15.143:0/4203270535 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc04c070 con 0x55a004b88b40 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7febeffff640 1 -- 172.21.15.143:0/4203270535 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc004630 con 0x55a004b88b40 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7febeffff640 1 -- 172.21.15.143:0/4203270535 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc034040 con 0x55a004b88b40 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7fec0e31f740 1 -- 172.21.15.143:0/4203270535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 msgr2=0x55a004b87090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.548+0000 7fec0e31f740 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b87090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc02f660 comp rx=0 tx=0).stop 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- 172.21.15.143:0/4203270535 shutdown_connections 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b98b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 --2- 172.21.15.143:0/4203270535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b87090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- 172.21.15.143:0/4203270535 >> 172.21.15.143:0/4203270535 conn(0x55a004b50310 msgr2=0x55a004b83c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- 172.21.15.143:0/4203270535 shutdown_connections 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- 172.21.15.143:0/4203270535 wait complete. 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- start start 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 Processor -- start 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 0x55a004b93460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b96780 0x55a004b93bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a004b99420 con 0x55a004b88b40 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a004aa8060 con 0x55a004b96780 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a004b995a0 con 0x55a004b875d0 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.549+0000 7fec0e31f740 10 client.0 _ll_register_callbacks cb 0x55a0047fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 0x55a004b93460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 0x55a004b93460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38182/0 (socket says 172.21.15.143:38182) 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 -- 172.21.15.143:0/3791989347 learned_addr learned my addr 172.21.15.143:0/3791989347 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec09ce2640 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 -- 172.21.15.143:0/3791989347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 msgr2=0x55a004b961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 -- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b96780 msgr2=0x55a004b93bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b96780 0x55a004b93bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 -- 172.21.15.143:0/3791989347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00485e3c0 con 0x55a004b88b40 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec094e1640 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 0x55a004b93460 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7febf40991b0 tx=0x7febf409c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee6bca2613c0c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7febed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7febed7fa640 1 -- 172.21.15.143:0/3791989347 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf409cd50 con 0x55a004b88b40 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc004630 con 0x55a004b88b40 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7febed7fa640 1 -- 172.21.15.143:0/3791989347 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7febf40993c0 con 0x55a004b88b40 2024-09-10T10:53:59.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7febed7fa640 1 -- 172.21.15.143:0/3791989347 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7febf40ba050 con 0x55a004b88b40 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec0e31f740 10 client.17478 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a004ba4f50 con 0x55a004b88b40 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.550+0000 7febed7fa640 1 -- 172.21.15.143:0/3791989347 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc004630 con 0x55a004b88b40 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7febed7fa640 1 -- 172.21.15.143:0/3791989347 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7febf4064060 con 0x55a004b88b40 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec0e31f740 10 client.17478 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec0e31f740 10 client.17478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec0e31f740 20 client.17478 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec0e31f740 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004ba63b0 0x55a004ba8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec0e31f740 4 client.17478 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a004afa860 con 0x55a004ba63b0 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec09ce2640 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004ba63b0 0x55a004ba8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7febed7fa640 10 client.17478 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.551+0000 7fec09ce2640 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004ba63b0 0x55a004ba8810 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=950ef525bdc7b36d server_cookie=82b83e8f114793f7 in_seq=0 out_seq=0 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7febed7fa640 1 -- 172.21.15.143:0/3791989347 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a004afa860 con 0x55a004ba63b0 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7febed7fa640 10 client.17478 handle_command_reply: tid=0 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 client.17478 shutdown 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 msgr2=0x55a004b93460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 0x55a004b93460 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7febf40991b0 tx=0x7febf409c770 comp rx=0 tx=0).stop 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 shutdown_connections 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004ba63b0 0x55a004ba8810 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b96780 0x55a004b93bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b88b40 0x55a004b93460 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7febed7fa640 0 client.17478 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 --2- 172.21.15.143:0/3791989347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.552+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 >> 172.21.15.143:0/3791989347 conn(0x55a004b50310 msgr2=0x55a004b89960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.553+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 shutdown_connections 2024-09-10T10:53:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.553+0000 7fec0e31f740 1 -- 172.21.15.143:0/3791989347 wait complete. 2024-09-10T10:53:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.553+0000 7fec0e31f740 20 client.17478 trim_cache size 0 max 16384 2024-09-10T10:53:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 Processor -- start 2024-09-10T10:53:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 -- start start 2024-09-10T10:53:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b88300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b83700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b83c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec041ac840 con 0x55a004b96780 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec041ae4a0 con 0x55a004b875d0 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0e31f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec041acf10 con 0x55a004b88b40 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b83c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b83c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38194/0 (socket says 172.21.15.143:38194) 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 -- 172.21.15.143:0/499944479 learned_addr learned my addr 172.21.15.143:0/499944479 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec09ce2640 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b88300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec094e1640 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b83700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 -- 172.21.15.143:0/499944479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 msgr2=0x55a004b83700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b83700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 -- 172.21.15.143:0/499944479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 msgr2=0x55a004b88300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b88300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 -- 172.21.15.143:0/499944479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a004b834f0 con 0x55a004b96780 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec094e1640 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b83700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.566+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b83c40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7febfc009420 tx=0x7febfc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2936bb1aeb93a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7febef7fe640 1 -- 172.21.15.143:0/499944479 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc058070 con 0x55a004b96780 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7febef7fe640 1 -- 172.21.15.143:0/499944479 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc02fdc0 con 0x55a004b96780 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7febef7fe640 1 -- 172.21.15.143:0/499944479 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc04c040 con 0x55a004b96780 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 -- 172.21.15.143:0/499944479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 msgr2=0x55a004b83c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b83c40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7febfc009420 tx=0x7febfc02a950 comp rx=0 tx=0).stop 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 -- 172.21.15.143:0/499944479 shutdown_connections 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b83c40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b88b40 0x55a004b83700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 --2- 172.21.15.143:0/499944479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b875d0 0x55a004b88300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 -- 172.21.15.143:0/499944479 >> 172.21.15.143:0/499944479 conn(0x55a004b50310 msgr2=0x55a004b7ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 -- 172.21.15.143:0/499944479 shutdown_connections 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.567+0000 7fec0e31f740 1 -- 172.21.15.143:0/499944479 wait complete. 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 -- start start 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 Processor -- start 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b88b40 0x55a004b908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b90e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec041add10 con 0x55a004b96780 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec041ad610 con 0x55a004b88b40 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec041aa820 con 0x55a004b875d0 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0e31f740 10 client.0 _ll_register_callbacks cb 0x7febf4007f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0a4e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b90e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0a4e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b90e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38196/0 (socket says 172.21.15.143:38196) 2024-09-10T10:53:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec0a4e3640 1 -- 172.21.15.143:0/765355476 learned_addr learned my addr 172.21.15.143:0/765355476 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec094e1640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b88b40 0x55a004b908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.568+0000 7fec09ce2640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0a4e3640 1 -- 172.21.15.143:0/765355476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 msgr2=0x55a004b903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b903b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0a4e3640 1 -- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b88b40 msgr2=0x55a004b908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b88b40 0x55a004b908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0a4e3640 1 -- 172.21.15.143:0/765355476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a004c10060 con 0x55a004b96780 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec09ce2640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0a4e3640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b90e30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7febfc0043b0 tx=0x7febfc0043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127687a0fdaaec21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7febed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7febed7fa640 1 -- 172.21.15.143:0/765355476 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc064070 con 0x55a004b96780 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7febed7fa640 1 -- 172.21.15.143:0/765355476 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7febfc004710 con 0x55a004b96780 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc02fdc0 con 0x55a004b96780 2024-09-10T10:53:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7febed7fa640 1 -- 172.21.15.143:0/765355476 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7febfc058040 con 0x55a004b96780 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0e31f740 10 client.17484 fetch_fsmap learned FSMap version 42 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a004c11210 con 0x55a004b96780 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7febed7fa640 1 -- 172.21.15.143:0/765355476 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc02fdc0 con 0x55a004b96780 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.569+0000 7febed7fa640 1 -- 172.21.15.143:0/765355476 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7febfc05d4e0 con 0x55a004b96780 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 10 client.17484 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 10 client.17484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 20 client.17484 populate_metadata read hostname 'smithi143' 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004c11490 0x55a004c13880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 4 client.17484 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 --> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a004af4d70 con 0x55a004c11490 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec09ce2640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004c11490 0x55a004c13880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7febed7fa640 10 client.17484 ms_handle_connect on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec09ce2640 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004c11490 0x55a004c13880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a1120824c9896cd server_cookie=65b6bb08e8891a88 in_seq=0 out_seq=0 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7febed7fa640 1 -- 172.21.15.143:0/765355476 <== mds.1 v2:172.21.15.144:6838/1994626034 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a004af4d70 con 0x55a004c11490 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7febed7fa640 10 client.17484 handle_command_reply: tid=0 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.570+0000 7fec0e31f740 1 client.17484 shutdown 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 msgr2=0x55a004b90e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b90e30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7febfc0043b0 tx=0x7febfc0043e0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 shutdown_connections 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a004b96780 0x55a004b90e30 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:6838/1994626034,v1:172.21.15.144:6839/1994626034] conn(0x55a004c11490 0x55a004c13880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a004b88b40 0x55a004b908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 --2- 172.21.15.143:0/765355476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a004b875d0 0x55a004b903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7febed7fa640 0 client.17484 ms_handle_reset on v2:172.21.15.144:6838/1994626034 2024-09-10T10:53:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 >> 172.21.15.143:0/765355476 conn(0x55a004b50310 msgr2=0x55a004b88170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 shutdown_connections 2024-09-10T10:53:59.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 1 -- 172.21.15.143:0/765355476 wait complete. 2024-09-10T10:53:59.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0e31f740 20 client.17484 trim_cache size 0 max 16384 2024-09-10T10:53:59.582 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7febf00a2b80 msgr2=0x7febf00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7febf00a2b80 0x7febf00a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7febfc009ca0 tx=0x7febfc042000 comp rx=0 tx=0).stop 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 msgr2=0x7fec041977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 0x7fec041977e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7febf4009970 tx=0x7febf4009e30 comp rx=0 tx=0).stop 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 shutdown_connections 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7febf00a2b80 0x7febf00a5040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04102ef0 0x7fec0419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec04102570 0x7fec041977e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 --2- 172.21.15.143:0/1924409507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec04100e20 0x7fec041972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.571+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 >> 172.21.15.143:0/1924409507 conn(0x7fec040fc7d0 msgr2=0x7fec040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.572+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 shutdown_connections 2024-09-10T10:53:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:53:59.572+0000 7fec0bf46640 1 -- 172.21.15.143:0/1924409507 wait complete. 2024-09-10T10:54:00.329 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:54:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:00.904 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:00 smithi143 ceph-mon[31057]: pgmap v619: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-09-10T10:54:00.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:00 smithi178 ceph-mon[34813]: pgmap v619: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-09-10T10:54:00.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:00 smithi144 ceph-mon[34230]: pgmap v619: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 -- 172.21.15.143:0/604398573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 msgr2=0x7f0d9c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 --2- 172.21.15.143:0/604398573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c100ed0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009930 tx=0x7f0d9002a420 comp rx=0 tx=0).stop 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 -- 172.21.15.143:0/604398573 shutdown_connections 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 --2- 172.21.15.143:0/604398573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c101410 0x7f0d9c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 --2- 172.21.15.143:0/604398573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c100ed0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 --2- 172.21.15.143:0/604398573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c106a70 0x7f0d9c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 -- 172.21.15.143:0/604398573 >> 172.21.15.143:0/604398573 conn(0x7f0d9c0fc790 msgr2=0x7f0d9c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 -- 172.21.15.143:0/604398573 shutdown_connections 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 -- 172.21.15.143:0/604398573 wait complete. 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 Processor -- start 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.109+0000 7f0da17c9640 1 -- start start 2024-09-10T10:54:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c101410 0x7f0d9c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c106a70 0x7f0d9c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c110d20 con 0x7f0d9c100a70 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c110ba0 con 0x7f0d9c101410 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c110ea0 con 0x7f0d9c106a70 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38208/0 (socket says 172.21.15.143:38208) 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 -- 172.21.15.143:0/2681656151 learned_addr learned my addr 172.21.15.143:0/2681656151 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c101410 0x7f0d9c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c106a70 0x7f0d9c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 -- 172.21.15.143:0/2681656151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c106a70 msgr2=0x7f0d9c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c106a70 0x7f0d9c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 -- 172.21.15.143:0/2681656151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c101410 msgr2=0x7f0d9c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c101410 0x7f0d9c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 -- 172.21.15.143:0/2681656151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c19d320 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c198370 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d770 tx=0x7f0d8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d83fff640 1 -- 172.21.15.143:0/2681656151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c017070 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d83fff640 1 -- 172.21.15.143:0/2681656151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d8c0054d0 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c19d610 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0d83fff640 1 -- 172.21.15.143:0/2681656151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c004cf0 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.110+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a4ef0 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0d83fff640 1 -- 172.21.15.143:0/2681656151 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0d8c004e90 con 0x7f0d9c100a70 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 Processor -- start 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 -- start start 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d790090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d790a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d79f780 0x55959d7a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55959d6f62f0 con 0x55959d791b40 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55959d69dcf0 con 0x55959d79f780 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55959d6f22b0 con 0x55959d7905d0 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0d83fff640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0d840a2b80 0x7f0d840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d790090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d790090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38210/0 (socket says 172.21.15.143:38210) 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.113+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/2088468757 learned_addr learned my addr 172.21.15.143:0/2088468757 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0d840a2b80 0x7f0d840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d83fff640 1 -- 172.21.15.143:0/2681656151 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0d8c0cecc0 con 0x7f0d9c100a70 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d790a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d79f780 0x55959d7a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/2088468757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 msgr2=0x55959d790a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d790a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/2088468757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d79f780 msgr2=0x55959d7a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d79f780 0x55959d7a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/2088468757 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959d6823a0 con 0x55959d791b40 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d790090 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0d94012d80 tx=0x7f0d94010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3d1126ff6f3e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0d840a2b80 0x7f0d840a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d94010c40 tx=0x7f0d9400c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d817fa640 1 -- 172.21.15.143:0/2088468757 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94024070 con 0x55959d791b40 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d817fa640 1 -- 172.21.15.143:0/2088468757 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d940125a0 con 0x55959d791b40 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.114+0000 7f0d817fa640 1 -- 172.21.15.143:0/2088468757 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94005280 con 0x55959d791b40 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2088468757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 msgr2=0x55959d790090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d790090 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0d94012d80 tx=0x7f0d94010740 comp rx=0 tx=0).stop 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2088468757 shutdown_connections 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d79f780 0x55959d7a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d790a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2088468757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d790090 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2088468757 >> 172.21.15.143:0/2088468757 conn(0x55959d759310 msgr2=0x55959d78cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2088468757 shutdown_connections 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2088468757 wait complete. 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.115+0000 7f0da3ba2740 1 -- start start 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 Processor -- start 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d79f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d79c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d79f780 0x55959d79cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55959d7a2420 con 0x55959d791b40 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55959d6b1060 con 0x55959d7905d0 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55959d7a25a0 con 0x55959d79f780 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0da3ba2740 10 client.0 _ll_register_callbacks cb 0x55959d4048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d79c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d79c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38214/0 (socket says 172.21.15.143:38214) 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 -- 172.21.15.143:0/2196838136 learned_addr learned my addr 172.21.15.143:0/2196838136 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d79f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 -- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d79f780 msgr2=0x55959d79cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d79f780 0x55959d79cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 -- 172.21.15.143:0/2196838136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 msgr2=0x55959d79f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d79f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 -- 172.21.15.143:0/2196838136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959d4673c0 con 0x55959d791b40 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d79c460 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0d9002a940 tx=0x7f0d900314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c7af42446809be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.116+0000 7f0d7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0d7eff5640 1 -- 172.21.15.143:0/2196838136 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90035040 con 0x55959d791b40 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0d7eff5640 1 -- 172.21.15.143:0/2196838136 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0d90031810 con 0x55959d791b40 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d940125a0 con 0x55959d791b40 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0d7eff5640 1 -- 172.21.15.143:0/2196838136 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0d90004040 con 0x55959d791b40 2024-09-10T10:54:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 10 client.17496 fetch_fsmap learned FSMap version 42 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55959d7adf00 con 0x55959d791b40 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0d7eff5640 1 -- 172.21.15.143:0/2196838136 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d940125a0 con 0x55959d791b40 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0d7eff5640 1 -- 172.21.15.143:0/2196838136 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f0d90046480 con 0x55959d791b40 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 10 client.17496 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 20 client.17496 populate_metadata read hostname 'smithi143' 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d7af460 0x55959d7b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 4 client.17496 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55959d703860 con 0x55959d7af460 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.117+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d7af460 0x55959d7b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0d9affd640 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d7af460 0x55959d7b18b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92ef3c0ee51fa1d server_cookie=4d2120c63f5a210 in_seq=0 out_seq=0 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0d7eff5640 10 client.17496 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:54:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0d7eff5640 1 -- 172.21.15.143:0/2196838136 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55959d703860 con 0x55959d7af460 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0d7eff5640 10 client.17496 handle_command_reply: tid=0 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0da3ba2740 1 client.17496 shutdown 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 msgr2=0x55959d79c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.118+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d79c460 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0d9002a940 tx=0x7f0d900314a0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 shutdown_connections 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d7af460 0x55959d7b18b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d79f780 0x55959d79cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d791b40 0x55959d79c460 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0d7eff5640 0 client.17496 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/2196838136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d79f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 >> 172.21.15.143:0/2196838136 conn(0x55959d759310 msgr2=0x55959d792960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 shutdown_connections 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 1 -- 172.21.15.143:0/2196838136 wait complete. 2024-09-10T10:54:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.119+0000 7f0da3ba2740 20 client.17496 trim_cache size 0 max 16384 2024-09-10T10:54:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 Processor -- start 2024-09-10T10:54:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 -- start start 2024-09-10T10:54:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d78c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d791b40 0x55959d78cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d7b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ac860 con 0x55959d79f780 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ae4c0 con 0x55959d7905d0 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1acf30 con 0x55959d791b40 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d7b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d7b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38220/0 (socket says 172.21.15.143:38220) 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/3635442477 learned_addr learned my addr 172.21.15.143:0/3635442477 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9affd640 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d78c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/3635442477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d791b40 msgr2=0x55959d78cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d791b40 0x55959d78cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d791b40 0x55959d78cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/3635442477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 msgr2=0x55959d78c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d78c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/3635442477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959d7a1ee0 con 0x55959d79f780 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.132+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d791b40 0x55959d78cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d7b71c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005f40 tx=0x7f0d94005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc1b537e5bdde90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0d80ff9640 1 -- 172.21.15.143:0/3635442477 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9402c070 con 0x55959d79f780 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0d80ff9640 1 -- 172.21.15.143:0/3635442477 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d940125a0 con 0x55959d79f780 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0d80ff9640 1 -- 172.21.15.143:0/3635442477 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94024070 con 0x55959d79f780 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 -- 172.21.15.143:0/3635442477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 msgr2=0x55959d7b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d7b71c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005f40 tx=0x7f0d94005bd0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 -- 172.21.15.143:0/3635442477 shutdown_connections 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d7b71c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d791b40 0x55959d78cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/3635442477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d7905d0 0x55959d78c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 -- 172.21.15.143:0/3635442477 >> 172.21.15.143:0/3635442477 conn(0x55959d759310 msgr2=0x55959d787a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 -- 172.21.15.143:0/3635442477 shutdown_connections 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.133+0000 7f0da3ba2740 1 -- 172.21.15.143:0/3635442477 wait complete. 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 -- start start 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 Processor -- start 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d7995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d791b40 0x55959d799b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d79a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1add30 con 0x55959d79f780 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ad630 con 0x55959d791b40 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1aa840 con 0x55959d7905d0 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d79a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0da3ba2740 10 client.0 _ll_register_callbacks cb 0x7f0d90009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d79a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38228/0 (socket says 172.21.15.143:38228) 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/1811265300 learned_addr learned my addr 172.21.15.143:0/1811265300 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d791b40 0x55959d799b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9affd640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d7995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 msgr2=0x55959d7995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d7995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/1811265300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d791b40 msgr2=0x55959d799b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d791b40 0x55959d799b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.134+0000 7f0d9b7fe640 1 -- 172.21.15.143:0/1811265300 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959d8192f0 con 0x55959d79f780 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d9a7fc640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d791b40 0x55959d799b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d9affd640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d7995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d9b7fe640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d79a050 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0d94012d80 tx=0x7f0d94009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943cd210f915cf83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d7eff5640 1 -- 172.21.15.143:0/1811265300 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94035070 con 0x55959d79f780 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d7eff5640 1 -- 172.21.15.143:0/1811265300 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0d94004a00 con 0x55959d79f780 2024-09-10T10:54:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d940125a0 con 0x55959d79f780 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d7eff5640 1 -- 172.21.15.143:0/1811265300 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0d94041050 con 0x55959d79f780 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 10 client.17502 fetch_fsmap learned FSMap version 42 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55959d81a4e0 con 0x55959d79f780 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d7eff5640 1 -- 172.21.15.143:0/1811265300 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d940125a0 con 0x55959d79f780 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0d7eff5640 1 -- 172.21.15.143:0/1811265300 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 42) ==== 8410+0+0 (secure 0 0 0) 0x7f0d9400f040 con 0x55959d79f780 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 10 client.17502 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 20 client.17502 populate_metadata read hostname 'smithi143' 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d81ab40 0x55959d81cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 4 client.17502 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.135+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 --> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55959d6fdd70 con 0x55959d81ab40 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.136+0000 7f0d9affd640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d81ab40 0x55959d81cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.136+0000 7f0d7eff5640 10 client.17502 ms_handle_connect on v2:172.21.15.178:6838/774166191 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.136+0000 7f0d9affd640 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d81ab40 0x55959d81cf60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fae9df6c34fd305 server_cookie=14c5aaf9a7a16f94 in_seq=0 out_seq=0 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.136+0000 7f0d7eff5640 1 -- 172.21.15.143:0/1811265300 <== mds.0 v2:172.21.15.178:6838/774166191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55959d6fdd70 con 0x55959d81ab40 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.136+0000 7f0d7eff5640 10 client.17502 handle_command_reply: tid=0 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 client.17502 shutdown 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 msgr2=0x55959d79a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d79a050 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0d94012d80 tx=0x7f0d94009ee0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 shutdown_connections 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55959d79f780 0x55959d79a050 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55959d791b40 0x55959d799b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:6838/774166191,v1:172.21.15.178:6839/774166191] conn(0x55959d81ab40 0x55959d81cf60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 --2- 172.21.15.143:0/1811265300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55959d7905d0 0x55959d7995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 >> 172.21.15.143:0/1811265300 conn(0x55959d759310 msgr2=0x55959d79a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0d7eff5640 0 client.17502 ms_handle_reset on v2:172.21.15.178:6838/774166191 2024-09-10T10:54:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 shutdown_connections 2024-09-10T10:54:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 1 -- 172.21.15.143:0/1811265300 wait complete. 2024-09-10T10:54:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da3ba2740 20 client.17502 trim_cache size 0 max 16384 2024-09-10T10:54:01.150 INFO:teuthology.orchestra.run.smithi143.stdout:[] 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0d840a2b80 msgr2=0x7f0d840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0d840a2b80 0x7f0d840a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d94010c40 tx=0x7f0d9400c000 comp rx=0 tx=0).stop 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 msgr2=0x7f0d9c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c198370 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00d770 tx=0x7f0d8c00dc30 comp rx=0 tx=0).stop 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 shutdown_connections 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0d840a2b80 0x7f0d840a5040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c106a70 0x7f0d9c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d9c101410 0x7f0d9c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 --2- 172.21.15.143:0/2681656151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d9c100a70 0x7f0d9c198370 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.137+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 >> 172.21.15.143:0/2681656151 conn(0x7f0d9c0fc790 msgr2=0x7f0d9c06aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.138+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 shutdown_connections 2024-09-10T10:54:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.138+0000 7f0da17c9640 1 -- 172.21.15.143:0/2681656151 wait complete. 2024-09-10T10:54:01.872 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T10:54:01.873 INFO:tasks.fwd_scrub:done joining 2024-09-10T10:54:01.873 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T10:54:01.884 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 -- 172.21.15.143:0/4050540745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114102410 msgr2=0x7fc114108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 --2- 172.21.15.143:0/4050540745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114102410 0x7fc114108ca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009ca0 tx=0x7fc10c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 -- 172.21.15.143:0/4050540745 shutdown_connections 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 --2- 172.21.15.143:0/4050540745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114102410 0x7fc114108ca0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 --2- 172.21.15.143:0/4050540745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc114101a50 0x7fc114101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 --2- 172.21.15.143:0/4050540745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114100860 0x7fc114100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 -- 172.21.15.143:0/4050540745 >> 172.21.15.143:0/4050540745 conn(0x7fc1140fc000 msgr2=0x7fc1140fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 -- 172.21.15.143:0/4050540745 shutdown_connections 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 -- 172.21.15.143:0/4050540745 wait complete. 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 Processor -- start 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.985+0000 7fc11a99f640 1 -- start start 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc114100860 0x7fc114069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 0x7fc11419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114102410 0x7fc11419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410c370 con 0x7fc114101a50 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410c1f0 con 0x7fc114102410 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410c4f0 con 0x7fc114100860 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 0x7fc11419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 0x7fc11419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38238/0 (socket says 172.21.15.143:38238) 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 -- 172.21.15.143:0/812127604 learned_addr learned my addr 172.21.15.143:0/812127604 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a19e640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114102410 0x7fc11419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11999d640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc114100860 0x7fc114069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 -- 172.21.15.143:0/812127604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc114100860 msgr2=0x7fc114069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc114100860 0x7fc114069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 -- 172.21.15.143:0/812127604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114102410 msgr2=0x7fc11419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114102410 0x7fc11419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 -- 172.21.15.143:0/812127604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419f7c0 con 0x7fc114101a50 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11919c640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 0x7fc11419c890 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc10800d950 tx=0x7fc10800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108017070 con 0x7fc114101a50 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc108009c30 con 0x7fc114101a50 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419fa50 con 0x7fc114101a50 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10800f620 con 0x7fc114101a50 2024-09-10T10:54:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.986+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11419ffa0 con 0x7fc114101a50 2024-09-10T10:54:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.987+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc114107d10 con 0x7fc114101a50 2024-09-10T10:54:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.988+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc1080109a0 con 0x7fc114101a50 2024-09-10T10:54:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.989+0000 7fc106ffd640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1000a2b90 0x7fc1000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.989+0000 7fc11999d640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1000a2b90 0x7fc1000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:01.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.989+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1080cef40 con 0x7fc114101a50 2024-09-10T10:54:01.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.989+0000 7fc11999d640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1000a2b90 0x7fc1000a5050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc1141018b0 tx=0x7fc11000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:01.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:01.993+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc108009de0 con 0x7fc114101a50 2024-09-10T10:54:02.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:01 smithi143 ceph-mon[31057]: pgmap v620: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T10:54:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.163+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc114062df0 con 0x7fc114101a50 2024-09-10T10:54:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.164+0000 7fc106ffd640 1 -- 172.21.15.143:0/812127604 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7fc10809b4f0 con 0x7fc114101a50 2024-09-10T10:54:02.166 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:54:02.166 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1000a2b90 msgr2=0x7fc1000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1000a2b90 0x7fc1000a5050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc1141018b0 tx=0x7fc11000a2d0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 msgr2=0x7fc11419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 0x7fc11419c890 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc10800d950 tx=0x7fc10800de10 comp rx=0 tx=0).stop 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 shutdown_connections 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc1000a2b90 0x7fc1000a5050 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114102410 0x7fc11419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc114101a50 0x7fc11419c890 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 --2- 172.21.15.143:0/812127604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc114100860 0x7fc114069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 >> 172.21.15.143:0/812127604 conn(0x7fc1140fc000 msgr2=0x7fc114104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:02.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 shutdown_connections 2024-09-10T10:54:02.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.166+0000 7fc11a99f640 1 -- 172.21.15.143:0/812127604 wait complete. 2024-09-10T10:54:02.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:01 smithi178 ceph-mon[34813]: pgmap v620: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T10:54:02.179 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:38:11.637482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:42:49.489312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:42:52.085668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:42:58.507118+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T10:54:02.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:54:02.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:01 smithi144 ceph-mon[34230]: pgmap v620: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 -- 172.21.15.143:0/806028553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c100a30 msgr2=0x7f2f5c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 --2- 172.21.15.143:0/806028553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c100a30 0x7f2f5c100e30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009900 tx=0x7f2f4c02a410 comp rx=0 tx=0).stop 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 -- 172.21.15.143:0/806028553 shutdown_connections 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 --2- 172.21.15.143:0/806028553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 --2- 172.21.15.143:0/806028553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c101c20 0x7f2f5c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 --2- 172.21.15.143:0/806028553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c100a30 0x7f2f5c100e30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 -- 172.21.15.143:0/806028553 >> 172.21.15.143:0/806028553 conn(0x7f2f5c0fc1b0 msgr2=0x7f2f5c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:02.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 -- 172.21.15.143:0/806028553 shutdown_connections 2024-09-10T10:54:02.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.281+0000 7f2f6242c640 1 -- 172.21.15.143:0/806028553 wait complete. 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.282+0000 7f2f6242c640 1 Processor -- start 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.282+0000 7f2f6242c640 1 -- start start 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f6242c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c100a30 0x7f2f5c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f6242c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f5c101c20 0x7f2f5c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f6242c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f6242c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c10c560 con 0x7f2f5c1025e0 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f6242c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c10c3e0 con 0x7f2f5c100a30 2024-09-10T10:54:02.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f6242c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c10c6e0 con 0x7f2f5c101c20 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c19e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38246/0 (socket says 172.21.15.143:38246) 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 -- 172.21.15.143:0/2001257618 learned_addr learned my addr 172.21.15.143:0/2001257618 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f5b7fe640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f5c101c20 0x7f2f5c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f5bfff640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c100a30 0x7f2f5c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 -- 172.21.15.143:0/2001257618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f5c101c20 msgr2=0x7f2f5c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f5c101c20 0x7f2f5c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 -- 172.21.15.143:0/2001257618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c100a30 msgr2=0x7f2f5c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c100a30 0x7f2f5c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 -- 172.21.15.143:0/2001257618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c19e960 con 0x7f2f5c1025e0 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f5bfff640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c100a30 0x7f2f5c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:02.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f609c9640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c19e2c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400eb10 tx=0x7f2f5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.283+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54004b20 con 0x7f2f5c1025e0 2024-09-10T10:54:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.284+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f54004cc0 con 0x7f2f5c1025e0 2024-09-10T10:54:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.284+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54010690 con 0x7f2f5c1025e0 2024-09-10T10:54:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.284+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5c19ec50 con 0x7f2f5c1025e0 2024-09-10T10:54:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.284+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5c19f1a0 con 0x7f2f5c1025e0 2024-09-10T10:54:02.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.285+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5c107fb0 con 0x7f2f5c1025e0 2024-09-10T10:54:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.287+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2f54005020 con 0x7f2f5c1025e0 2024-09-10T10:54:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.287+0000 7f2f597fa640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f440a2c10 0x7f2f440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.288+0000 7f2f5bfff640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f440a2c10 0x7f2f440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.288+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2f540cfb20 con 0x7f2f5c1025e0 2024-09-10T10:54:02.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.288+0000 7f2f5bfff640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f440a2c10 0x7f2f440a50d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009230 tx=0x7f2f4c009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:02.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.292+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f5409c720 con 0x7f2f5c1025e0 2024-09-10T10:54:02.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.461+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f5c062e90 con 0x7f2f5c1025e0 2024-09-10T10:54:02.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.461+0000 7f2f597fa640 1 -- 172.21.15.143:0/2001257618 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f2f5409c020 con 0x7f2f5c1025e0 2024-09-10T10:54:02.463 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.463+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f440a2c10 msgr2=0x7f2f440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.463+0000 7f2f6242c640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f440a2c10 0x7f2f440a50d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009230 tx=0x7f2f4c009610 comp rx=0 tx=0).stop 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 msgr2=0x7f2f5c19e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c19e2c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400eb10 tx=0x7f2f5400c2d0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 shutdown_connections 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f2f440a2c10 0x7f2f440a50d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1025e0 0x7f2f5c19e2c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f5c101c20 0x7f2f5c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 --2- 172.21.15.143:0/2001257618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f5c100a30 0x7f2f5c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 >> 172.21.15.143:0/2001257618 conn(0x7f2f5c0fc1b0 msgr2=0x7f2f5c105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:02.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 shutdown_connections 2024-09-10T10:54:02.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.464+0000 7f2f6242c640 1 -- 172.21.15.143:0/2001257618 wait complete. 2024-09-10T10:54:02.476 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 -- 172.21.15.143:0/228690245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 msgr2=0x7f0e001001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 --2- 172.21.15.143:0/228690245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 0x7f0e001001f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df002a410 comp rx=0 tx=0).stop 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 -- 172.21.15.143:0/228690245 shutdown_connections 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 --2- 172.21.15.143:0/228690245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0010a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 --2- 172.21.15.143:0/228690245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 0x7f0e00103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 --2- 172.21.15.143:0/228690245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 0x7f0e001001f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 -- 172.21.15.143:0/228690245 >> 172.21.15.143:0/228690245 conn(0x7f0e000fb910 msgr2=0x7f0e000fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:02.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 -- 172.21.15.143:0/228690245 shutdown_connections 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.577+0000 7f0e06282640 1 -- 172.21.15.143:0/228690245 wait complete. 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 Processor -- start 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 -- start start 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 0x7f0e00197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 0x7f0e00198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0010d650 con 0x7f0e00103780 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0010d4d0 con 0x7f0e00102dc0 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e06282640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0010d7d0 con 0x7f0e000ffdf0 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e0481f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e0481f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38262/0 (socket says 172.21.15.143:38262) 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0e0481f640 1 -- 172.21.15.143:0/1273502531 learned_addr learned my addr 172.21.15.143:0/1273502531 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0dfffff640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 0x7f0e00197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.578+0000 7f0df7fff640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 0x7f0e00198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e0481f640 1 -- 172.21.15.143:0/1273502531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 msgr2=0x7f0e00197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e0481f640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 0x7f0e00197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e0481f640 1 -- 172.21.15.143:0/1273502531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 msgr2=0x7f0e00198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e0481f640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 0x7f0e00198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e0481f640 1 -- 172.21.15.143:0/1273502531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0019fa10 con 0x7f0e00103780 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0df7fff640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 0x7f0e00198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e0481f640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0019f3a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0df800efd0 tx=0x7f0df800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8003b70 con 0x7f0e00103780 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0019fd00 con 0x7f0e00103780 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df8003d10 con 0x7f0e00103780 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8010920 con 0x7f0e00103780 2024-09-10T10:54:02.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.579+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e001a01a0 con 0x7f0e00103780 2024-09-10T10:54:02.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.580+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e001090b0 con 0x7f0e00103780 2024-09-10T10:54:02.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.581+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0df8004000 con 0x7f0e00103780 2024-09-10T10:54:02.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.582+0000 7f0dfdffb640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dec0a2b90 0x7f0dec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.582+0000 7f0dfffff640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dec0a2b90 0x7f0dec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.582+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0df80cfc20 con 0x7f0e00103780 2024-09-10T10:54:02.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.582+0000 7f0dfffff640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dec0a2b90 0x7f0dec0a5050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0df0004770 tx=0x7f0df0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:02.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.586+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0df809c820 con 0x7f0e00103780 2024-09-10T10:54:02.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.754+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e00062e90 con 0x7f0e00103780 2024-09-10T10:54:02.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.754+0000 7f0dfdffb640 1 -- 172.21.15.143:0/1273502531 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0df809c120 con 0x7f0e00103780 2024-09-10T10:54:02.755 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dec0a2b90 msgr2=0x7f0dec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dec0a2b90 0x7f0dec0a5050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0df0004770 tx=0x7f0df0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 msgr2=0x7f0e0019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0019f3a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0df800efd0 tx=0x7f0df800c6a0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 shutdown_connections 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f0dec0a2b90 0x7f0dec0a5050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e00103780 0x7f0e0019f3a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e00102dc0 0x7f0e00198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 --2- 172.21.15.143:0/1273502531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e000ffdf0 0x7f0e00197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 >> 172.21.15.143:0/1273502531 conn(0x7f0e000fb910 msgr2=0x7f0e00106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:02.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 shutdown_connections 2024-09-10T10:54:02.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.756+0000 7f0e06282640 1 -- 172.21.15.143:0/1273502531 wait complete. 2024-09-10T10:54:02.767 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:02 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/812127604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:54:02.767 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:02 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2001257618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:54:02.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 -- 172.21.15.143:0/2581464462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 msgr2=0x7f7578109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 --2- 172.21.15.143:0/2581464462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f7578109f50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7570009ca0 tx=0x7f757002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 -- 172.21.15.143:0/2581464462 shutdown_connections 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 --2- 172.21.15.143:0/2581464462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f7578109f50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 --2- 172.21.15.143:0/2581464462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 0x7f7578103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 --2- 172.21.15.143:0/2581464462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7578101b10 0x7f7578101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 -- 172.21.15.143:0/2581464462 >> 172.21.15.143:0/2581464462 conn(0x7f75780fd2b0 msgr2=0x7f75780ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 -- 172.21.15.143:0/2581464462 shutdown_connections 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.870+0000 7f757f70a640 1 -- 172.21.15.143:0/2581464462 wait complete. 2024-09-10T10:54:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 Processor -- start 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 -- start start 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7578101b10 0x7f7578197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 0x7f75781981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f757819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f757810d640 con 0x7f75781036c0 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f757810d4c0 con 0x7f7578102d00 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757f70a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f757810d7c0 con 0x7f7578101b10 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757dca7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f757819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757dca7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f757819f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38280/0 (socket says 172.21.15.143:38280) 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757dca7640 1 -- 172.21.15.143:0/3652500895 learned_addr learned my addr 172.21.15.143:0/3652500895 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.871+0000 7f757d4a6640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7578101b10 0x7f7578197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757cca5640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 0x7f75781981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757dca7640 1 -- 172.21.15.143:0/3652500895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7578101b10 msgr2=0x7f7578197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757dca7640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7578101b10 0x7f7578197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757dca7640 1 -- 172.21.15.143:0/3652500895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 msgr2=0x7f75781981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757dca7640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 0x7f75781981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757dca7640 1 -- 172.21.15.143:0/3652500895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757819f8e0 con 0x7f75781036c0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757cca5640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 0x7f75781981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757dca7640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f757819f240 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7570009dc0 tx=0x7f7570003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570032ec0 con 0x7f75781036c0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75700044f0 con 0x7f75781036c0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757819fb70 con 0x7f75781036c0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570030d40 con 0x7f75781036c0 2024-09-10T10:54:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.872+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781a00c0 con 0x7f75781036c0 2024-09-10T10:54:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.873+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578108f00 con 0x7f75781036c0 2024-09-10T10:54:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.874+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7570004040 con 0x7f75781036c0 2024-09-10T10:54:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.875+0000 7f756e7fc640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75640a2d30 0x7f75640a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.875+0000 7f757d4a6640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75640a2d30 0x7f75640a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.875+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f75700f09b0 con 0x7f75781036c0 2024-09-10T10:54:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.875+0000 7f757d4a6640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75640a2d30 0x7f75640a51f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7578198b70 tx=0x7f757400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:02.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:02.879+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75700bd5b0 con 0x7f75781036c0 2024-09-10T10:54:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.044+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7578062df0 con 0x7f75781036c0 2024-09-10T10:54:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.044+0000 7f756e7fc640 1 -- 172.21.15.143:0/3652500895 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f75700bceb0 con 0x7f75781036c0 2024-09-10T10:54:03.046 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75640a2d30 msgr2=0x7f75640a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75640a2d30 0x7f75640a51f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7578198b70 tx=0x7f757400b390 comp rx=0 tx=0).stop 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 msgr2=0x7f757819f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f757819f240 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7570009dc0 tx=0x7f7570003ba0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 shutdown_connections 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f75640a2d30 0x7f75640a51f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75781036c0 0x7f757819f240 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7578102d00 0x7f75781981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 --2- 172.21.15.143:0/3652500895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7578101b10 0x7f7578197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.046+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 >> 172.21.15.143:0/3652500895 conn(0x7f75780fd2b0 msgr2=0x7f75781060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.047+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 shutdown_connections 2024-09-10T10:54:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.047+0000 7f757f70a640 1 -- 172.21.15.143:0/3652500895 wait complete. 2024-09-10T10:54:03.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 -- 172.21.15.143:0/3744338111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 msgr2=0x7fe46c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 --2- 172.21.15.143:0/3744338111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c109f50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe464009ca0 tx=0x7fe46402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 -- 172.21.15.143:0/3744338111 shutdown_connections 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 --2- 172.21.15.143:0/3744338111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c109f50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 --2- 172.21.15.143:0/3744338111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46c102d00 0x7fe46c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 --2- 172.21.15.143:0/3744338111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe46c101b10 0x7fe46c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 -- 172.21.15.143:0/3744338111 >> 172.21.15.143:0/3744338111 conn(0x7fe46c0fd2b0 msgr2=0x7fe46c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 -- 172.21.15.143:0/3744338111 shutdown_connections 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.160+0000 7fe4721d8640 1 -- 172.21.15.143:0/3744338111 wait complete. 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 Processor -- start 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 -- start start 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe46c101b10 0x7fe46c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46c102d00 0x7fe46c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10d620 con 0x7fe46c1036c0 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10d4a0 con 0x7fe46c102d00 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe4721d8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10d7a0 con 0x7fe46c101b10 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38296/0 (socket says 172.21.15.143:38296) 2024-09-10T10:54:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe46bfff640 1 -- 172.21.15.143:0/457937715 learned_addr learned my addr 172.21.15.143:0/457937715 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe46b7fe640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe46c101b10 0x7fe46c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.161+0000 7fe46affd640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46c102d00 0x7fe46c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe46bfff640 1 -- 172.21.15.143:0/457937715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe46c101b10 msgr2=0x7fe46c075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe46bfff640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe46c101b10 0x7fe46c075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe46bfff640 1 -- 172.21.15.143:0/457937715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46c102d00 msgr2=0x7fe46c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe46bfff640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46c102d00 0x7fe46c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe46bfff640 1 -- 172.21.15.143:0/457937715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c0753b0 con 0x7fe46c1036c0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe46bfff640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c074b20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe464009dc0 tx=0x7fe464003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464032ec0 con 0x7fe46c1036c0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4640044f0 con 0x7fe46c1036c0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c1a1e30 con 0x7fe46c1036c0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464030d40 con 0x7fe46c1036c0 2024-09-10T10:54:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.162+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a2290 con 0x7fe46c1036c0 2024-09-10T10:54:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.163+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c108ff0 con 0x7fe46c1036c0 2024-09-10T10:54:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.164+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe464004040 con 0x7fe46c1036c0 2024-09-10T10:54:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.165+0000 7fe468ff9640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4580a2d30 0x7fe4580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.165+0000 7fe46b7fe640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4580a2d30 0x7fe4580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.165+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe4640f0ae0 con 0x7fe46c1036c0 2024-09-10T10:54:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.165+0000 7fe46b7fe640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4580a2d30 0x7fe4580a51f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe45c005d00 tx=0x7fe45c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.168+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4640bd6e0 con 0x7fe46c1036c0 2024-09-10T10:54:03.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:02 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/812127604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:54:03.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:02 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2001257618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:54:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:02 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/812127604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:54:03.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:02 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2001257618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:54:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.332+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe46c062e90 con 0x7fe46c1036c0 2024-09-10T10:54:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.333+0000 7fe468ff9640 1 -- 172.21.15.143:0/457937715 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fe4640bcfe0 con 0x7fe46c1036c0 2024-09-10T10:54:03.334 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-09-10T10:54:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4580a2d30 msgr2=0x7fe4580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4580a2d30 0x7fe4580a51f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe45c005d00 tx=0x7fe45c005c50 comp rx=0 tx=0).stop 2024-09-10T10:54:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 msgr2=0x7fe46c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c074b20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe464009dc0 tx=0x7fe464003ba0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 shutdown_connections 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fe4580a2d30 0x7fe4580a51f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe46c1036c0 0x7fe46c074b20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46c102d00 0x7fe46c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 --2- 172.21.15.143:0/457937715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe46c101b10 0x7fe46c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 >> 172.21.15.143:0/457937715 conn(0x7fe46c0fd2b0 msgr2=0x7fe46c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 shutdown_connections 2024-09-10T10:54:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.335+0000 7fe4721d8640 1 -- 172.21.15.143:0/457937715 wait complete. 2024-09-10T10:54:03.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T10:54:03.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 -- 172.21.15.143:0/1336690339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2540102d20 msgr2=0x7f25401031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 --2- 172.21.15.143:0/1336690339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2540102d20 0x7f25401031a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2530009900 tx=0x7f253002a410 comp rx=0 tx=0).stop 2024-09-10T10:54:03.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 -- 172.21.15.143:0/1336690339 shutdown_connections 2024-09-10T10:54:03.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 --2- 172.21.15.143:0/1336690339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25401036e0 0x7f2540109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 --2- 172.21.15.143:0/1336690339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2540102d20 0x7f25401031a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 --2- 172.21.15.143:0/1336690339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 0x7f2540101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 -- 172.21.15.143:0/1336690339 >> 172.21.15.143:0/1336690339 conn(0x7f25400fd2b0 msgr2=0x7f25400ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 -- 172.21.15.143:0/1336690339 shutdown_connections 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.447+0000 7f25478f8640 1 -- 172.21.15.143:0/1336690339 wait complete. 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 Processor -- start 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 -- start start 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 0x7f2540197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2540102d20 0x7f2540198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 0x7f254019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f254010d6c0 con 0x7f25401036e0 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f254010d540 con 0x7f2540101b30 2024-09-10T10:54:03.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f25478f8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f254010d840 con 0x7f2540102d20 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 0x7f254019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 0x7f254019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38314/0 (socket says 172.21.15.143:38314) 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 -- 172.21.15.143:0/3739738096 learned_addr learned my addr 172.21.15.143:0/3739738096 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2544e93640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2540102d20 0x7f2540198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545694640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 0x7f2540197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 -- 172.21.15.143:0/3739738096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2540102d20 msgr2=0x7f2540198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2540102d20 0x7f2540198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 -- 172.21.15.143:0/3739738096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 msgr2=0x7f2540197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 0x7f2540197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 -- 172.21.15.143:0/3739738096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254019f9f0 con 0x7f25401036e0 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545694640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 0x7f2540197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:03.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.448+0000 7f2545e95640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 0x7f254019f380 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f253800b970 tx=0x7f253800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.449+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538019070 con 0x7f25401036e0 2024-09-10T10:54:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.449+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2538014440 con 0x7f25401036e0 2024-09-10T10:54:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.449+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254019fce0 con 0x7f25401036e0 2024-09-10T10:54:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.449+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25380075d0 con 0x7f25401036e0 2024-09-10T10:54:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.449+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25401a0200 con 0x7f25401036e0 2024-09-10T10:54:03.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.450+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2540108f70 con 0x7f25401036e0 2024-09-10T10:54:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.451+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f253801c070 con 0x7f25401036e0 2024-09-10T10:54:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.451+0000 7f25367fc640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f252c0a2b90 0x7f252c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.451+0000 7f2545694640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f252c0a2b90 0x7f252c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.452+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f25380ceec0 con 0x7f25401036e0 2024-09-10T10:54:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.452+0000 7f2545694640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f252c0a2b90 0x7f252c0a5050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2540102b80 tx=0x7f253c005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:03.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.454+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f253809bac0 con 0x7f25401036e0 2024-09-10T10:54:03.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.666+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2540062df0 con 0x7f25401036e0 2024-09-10T10:54:03.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.673+0000 7f25367fc640 1 -- 172.21.15.143:0/3739738096 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740378 (secure 0 0 0) 0x7f253809b3c0 con 0x7f25401036e0 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 20, 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "active_gid": 14150, 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "active_name": "x", 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "active_addrs": { 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:6800", 2024-09-10T10:54:03.686 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1095035473 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:6801", 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1095035473 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "active_addr": "172.21.15.143:6801/1095035473", 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "active_change": "2024-09-10T10:35:59.338398+0000", 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-09-10T10:54:03.687 INFO:teuthology.orchestra.run.smithi143.stdout: "standbys": [ 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 24109, 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "z", 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "available_modules": [ 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "alerts", 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:03.688 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.689 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.690 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.691 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.692 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.693 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.694 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_destination": { 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_destination", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.695 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_from_name": { 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_from_name", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "Ceph", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email From: name", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_host": { 2024-09-10T10:54:03.696 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_host", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP server", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.697 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_password": { 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_password", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Password to authenticate with", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.698 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_port": { 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_port", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "465", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP port", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_sender": { 2024-09-10T10:54:03.699 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_sender", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.700 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_ssl": { 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_ssl", 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.701 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.702 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_user": { 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_user", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "User to authenticate as", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.703 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "balancer", 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "active": { 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "active", 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.704 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_time": { 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_time", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0000", 2024-09-10T10:54:03.705 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_weekday": { 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_weekday", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.706 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_max_iterations": { 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.707 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "25", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "250", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_metrics": { 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_metrics", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.708 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_step": { 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_step", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:03.709 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.5", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0.001", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "0.999", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "end_time": { 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_time", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.710 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2359", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "end_weekday": { 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_weekday", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:03.711 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.713 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.714 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.715 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.716 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.717 INFO:teuthology.orchestra.run.smithi143.stdout: "min_score": { 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_score", 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.718 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "upmap", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.719 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "crush-compat", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "none", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "read", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap-read" 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Balancer mode", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.720 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_ids": { 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_ids", 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.721 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.722 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_deviation": { 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_deviation", 2024-09-10T10:54:03.723 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.724 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_optimizations": { 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm", 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_down_multiplier": { 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_down_multiplier", 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.726 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_refresh_rate": { 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_refresh_rate", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.727 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "20", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_starting_port": { 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_starting_port", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.728 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4721", 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_ptrace": { 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_ptrace", 2024-09-10T10:54:03.729 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_interval": { 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_interval", 2024-09-10T10:54:03.730 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:54:03.731 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.7", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.732 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm_log_destination": { 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm_log_destination", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "file", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "file,syslog", 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: "syslog" 2024-09-10T10:54:03.733 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "cgroups_split": { 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cgroups_split", 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.734 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "config_checks_enabled": { 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_checks_enabled", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.735 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "config_dashboard": { 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_dashboard", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.736 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_alertmanager": { 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_alertmanager", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.737 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_base": { 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_base", 2024-09-10T10:54:03.738 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.739 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_elasticsearch": { 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "elasticsearch container image", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_grafana": { 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_grafana", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_haproxy": { 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_haproxy", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.742 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "HAproxy container image", 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_agent": { 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.743 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_collector": { 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.744 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_query": { 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:54:03.745 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger query container image", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.746 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_keepalived": { 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_keepalived", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Keepalived container image", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.747 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_loki": { 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_loki", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Loki container image", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.748 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_node_exporter": { 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_node_exporter", 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.749 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_nvmeof": { 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_nvmeof", 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.750 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Nvme-of container image", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_prometheus": { 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_prometheus", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.751 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_promtail": { 2024-09-10T10:54:03.752 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_promtail", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Promtail container image", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.753 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_samba": { 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_samba", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.754 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_snmp_gateway": { 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.755 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "container_init": { 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_init", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:54:03.756 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "daemon_cache_timeout": { 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.757 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "900", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.758 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "default_registry": { 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_registry", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io", 2024-09-10T10:54:03.759 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "device_cache_timeout": { 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_cache_timeout", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.760 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1800", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "device_enhanced_scan": { 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_enhanced_scan", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.761 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "facts_cache_timeout": { 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "facts_cache_timeout", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.762 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "grafana_dashboards_path": { 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:54:03.763 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "host_check_interval": { 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "host_check_interval", 2024-09-10T10:54:03.764 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "hw_monitoring": { 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hw_monitoring", 2024-09-10T10:54:03.765 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.766 INFO:teuthology.orchestra.run.smithi143.stdout: "inventory_list_all": { 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "inventory_list_all", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.767 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.768 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "log_refresh_metadata": { 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_refresh_metadata", 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.769 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.770 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.772 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.773 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.774 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.775 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.776 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.777 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "*", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "max_count_per_host": { 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_count_per_host", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.778 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "max_osd_draining_count": { 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_osd_draining_count", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.779 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "migration_current": { 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "migration_current", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.780 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "internal - do not modify", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.781 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "root", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm-package", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "root" 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.782 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "oob_default_addr": { 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "oob_default_addr", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "169.254.1.1", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.783 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "prometheus_alerts_path": { 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.784 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_insecure": { 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_insecure", 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.785 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_password": { 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_password", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.786 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_url": { 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_url", 2024-09-10T10:54:03.787 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.788 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_username": { 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_username", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.789 INFO:teuthology.orchestra.run.smithi143.stdout: "secure_monitoring_stack": { 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.790 INFO:teuthology.orchestra.run.smithi143.stdout: "service_discovery_port": { 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "service_discovery_port", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8765", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.791 INFO:teuthology.orchestra.run.smithi143.stdout: "ssh_config_file": { 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssh_config_file", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.792 INFO:teuthology.orchestra.run.smithi143.stdout: "use_agent": { 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_agent", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.793 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "use_repo_digest": { 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_repo_digest", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.794 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_failed_host_check": { 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.795 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_daemons": { 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.796 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_hosts": { 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.797 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crash", 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.798 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.799 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.800 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.801 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.802 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.803 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "retain_interval": { 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retain_interval", 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.804 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "31536000", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_recent_interval": { 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_recent_interval", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.805 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1209600", 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.806 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dashboard", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.807 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.808 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.809 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.810 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.811 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.812 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:54:03.813 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:54:03.814 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:54:03.815 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:54:03.816 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.817 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.818 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.819 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.820 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.821 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.822 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.823 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_URL": { 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.824 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.825 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.826 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.827 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:54:03.828 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:54:03.829 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.830 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.831 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.832 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.833 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.834 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.835 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.836 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.837 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.838 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.839 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.840 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.841 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.842 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8", 2024-09-10T10:54:03.843 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.844 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "45", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.845 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:54:03.846 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.847 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.848 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.849 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.850 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.851 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.852 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "cross_origin_url": { 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cross_origin_url", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.853 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "crt_file": { 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crt_file", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.854 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "debug": { 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "debug", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.855 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "jwt_token_ttl": { 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "jwt_token_ttl", 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.856 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "28800", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.857 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.858 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.859 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.860 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.861 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.862 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.863 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "motd": { 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "motd", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.864 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "The message of the day", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.865 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.866 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8080", 2024-09-10T10:54:03.867 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.868 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl_server_port": { 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl_server_port", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.869 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8443", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.870 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "redirect", 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect" 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.871 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.872 INFO:teuthology.orchestra.run.smithi143.stdout: "url_prefix": { 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url_prefix", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.873 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_monitoring": { 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_monitoring", 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.874 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "monitor device health metrics", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.875 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.876 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.877 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.878 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.879 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.880 INFO:teuthology.orchestra.run.smithi143.stdout: "mark_out_threshold": { 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mark_out_threshold", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2419200", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.881 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_name": { 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_name", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "device_health_metrics", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.882 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "retention_period": { 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retention_period", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15552000", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.883 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_frequency": { 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_frequency", 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.884 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "self_heal": { 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "self_heal", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.885 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:54:03.886 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_threshold": { 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_threshold", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7257600", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:54:03.887 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "diskprediction_local", 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.888 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.889 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.890 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.891 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.892 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "predict_interval": { 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predict_interval", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.893 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "predictor_model": { 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predictor_model", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.894 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "prophetstor", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.895 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.896 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "influx", 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": false, 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "batch_size": { 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "batch_size", 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5000", 2024-09-10T10:54:03.897 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "database": { 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "database", 2024-09-10T10:54:03.898 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "ceph", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": { 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hostname", 2024-09-10T10:54:03.899 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:03.900 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "30", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "5", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.901 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.902 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.903 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.904 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.905 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.906 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "password": { 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "password", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.907 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "port": { 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "port", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.908 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8086", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server port", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.909 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "false", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "threads": { 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threads", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.910 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "32", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "username": { 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "username", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.911 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "verify_ssl": { 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "verify_ssl", 2024-09-10T10:54:03.912 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "true", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.913 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "insights", 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.914 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.916 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.917 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.918 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.919 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "iostat", 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.920 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.921 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.922 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.923 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.924 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.925 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "k8sevents", 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.926 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_event_retention_days": { 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:54:03.927 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "config_check_secs": { 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_check_secs", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "10", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.928 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.929 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.930 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.931 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.932 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.933 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "localpool", 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.934 INFO:teuthology.orchestra.run.smithi143.stdout: "failure_domain": { 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "failure_domain", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "host", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.935 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.936 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.937 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.938 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.939 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.940 INFO:teuthology.orchestra.run.smithi143.stdout: "min_size": { 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_size", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "num_rep": { 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "num_rep", 2024-09-10T10:54:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_num": { 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_num", 2024-09-10T10:54:03.942 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "128", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "prefix": { 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prefix", 2024-09-10T10:54:03.943 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "subtree": { 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subtree", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.944 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "rack", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mds_autoscaler", 2024-09-10T10:54:03.945 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.946 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.947 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.948 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.949 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.950 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirroring", 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.951 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.952 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.953 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.954 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.955 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "nfs", 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.956 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.957 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.958 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.959 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.960 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.961 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "fail_fs": { 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "fail_fs", 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.962 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.963 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.964 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.965 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.966 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.967 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.968 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator": { 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "rook", 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: "test_orchestrator" 2024-09-10T10:54:03.969 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Orchestrator backend", 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_perf_query", 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.970 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.971 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.972 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.973 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.974 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.975 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_support", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.977 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.978 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.979 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.980 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_autoscaler", 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.981 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.982 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.983 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.984 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.985 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.986 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "threshold": { 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threshold", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.987 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1.0", 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "scaling threshold", 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "progress", 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.988 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_pg_recovery_event": { 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.989 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.990 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.991 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.992 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.993 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:03.994 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.995 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "max_completed_events": { 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_completed_events", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "50", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.996 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.997 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus", 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:03.998 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "cache": { 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cache", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:03.999 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "exclude_perf_counters": { 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "exclude_perf_counters", 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.000 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.001 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.002 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.003 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.004 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.005 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools": { 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools", 2024-09-10T10:54:04.006 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.007 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "300", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.008 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_interval": { 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_interval", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15.0", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.009 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:54:04.010 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "9283", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.011 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "stale_cache_strategy": { 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stale_cache_strategy", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "log", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.012 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "default", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.013 INFO:teuthology.orchestra.run.smithi143.stdout: "default", 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "error" 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.014 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2024-09-10T10:54:04.015 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.016 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.017 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.018 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.019 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.020 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_snap_create": { 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.021 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.022 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "trash_purge_schedule": { 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "trash_purge_schedule", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.023 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "restful", 2024-09-10T10:54:04.024 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_auth": { 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_auth", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.025 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.026 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.027 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.028 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.029 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.030 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.031 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.034 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rgw", 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.035 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.036 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.037 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.038 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.039 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.040 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rook", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.042 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.043 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.044 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.045 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "storage_class": { 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "storage_class", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "local", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "selftest", 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.049 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.050 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.051 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.052 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.053 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.054 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "roption1": { 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption1", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.055 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "roption2": { 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption2", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "xyz", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.056 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption1": { 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption1", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.057 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption2": { 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption2", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.058 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption3": { 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption3", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.059 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption4": { 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption4", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.060 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption5": { 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption5", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.061 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption6": { 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption6", 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.062 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption7": { 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption7", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.063 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "42", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "testkey": { 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testkey", 2024-09-10T10:54:04.064 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "testlkey": { 2024-09-10T10:54:04.065 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testlkey", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "testnewline": { 2024-09-10T10:54:04.066 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testnewline", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.067 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snap_schedule", 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_m_granularity": { 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_m_granularity", 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.068 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "dump_on_update": { 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dump_on_update", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.069 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.070 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.071 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.072 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.073 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.074 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.075 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stats", 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.076 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.077 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.078 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.079 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.080 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.081 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "status", 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.082 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.083 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.084 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.085 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.086 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.087 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telegraf", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "address": { 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "address", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.088 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15", 2024-09-10T10:54:04.089 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.090 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.091 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.092 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.093 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.094 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.095 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telemetry", 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_basic": { 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_basic", 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.096 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_crash": { 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_crash", 2024-09-10T10:54:04.097 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_device": { 2024-09-10T10:54:04.098 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_device", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.099 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_ident": { 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_ident", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.100 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_perf": { 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_perf", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.101 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "contact": { 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "contact", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.102 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "description": { 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "description", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.103 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "device_url": { 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_url", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.104 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.105 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.106 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "24", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "8", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "last_opt_revision": { 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "last_opt_revision", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.107 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1", 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard": { 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard", 2024-09-10T10:54:04.108 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard_description": { 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard_description", 2024-09-10T10:54:04.109 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.110 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.111 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.112 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.113 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.114 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.115 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "organization": { 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "organization", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.116 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "proxy": { 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "proxy", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.117 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "url": { 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.118 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "test_orchestrator", 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.119 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.120 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.121 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.122 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.123 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.124 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.125 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.126 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.127 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.128 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.129 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_clones": { 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_clones", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.130 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "periodic_async_work": { 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "periodic_async_work", 2024-09-10T10:54:04.131 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Periodically check for async work", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_delay": { 2024-09-10T10:54:04.132 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.133 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.134 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix", 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "discovery_interval": { 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "discovery_interval", 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "100", 2024-09-10T10:54:04.135 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "identifier": { 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "identifier", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.136 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.137 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.138 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.139 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.140 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.141 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.142 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_host": { 2024-09-10T10:54:04.143 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_host", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.144 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_port": { 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_port", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10051", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.145 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_sender": { 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_sender", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.146 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 24113, 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "y", 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: "available_modules": [ 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "alerts", 2024-09-10T10:54:04.147 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:54:04.148 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.149 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.150 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.151 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.152 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.153 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_destination": { 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_destination", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:54:04.154 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_from_name": { 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_from_name", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "Ceph", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email From: name", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.155 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_host": { 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_host", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP server", 2024-09-10T10:54:04.156 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_password": { 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_password", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Password to authenticate with", 2024-09-10T10:54:04.157 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_port": { 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_port", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "465", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.158 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP port", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_sender": { 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_sender", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.159 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_ssl": { 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_ssl", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.160 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_user": { 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_user", 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.161 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "User to authenticate as", 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "balancer", 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.162 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "active": { 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "active", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.163 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_time": { 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_time", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0000", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:54:04.164 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_weekday": { 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_weekday", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.165 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_max_iterations": { 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "25", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "250", 2024-09-10T10:54:04.166 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_metrics": { 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_metrics", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.167 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_step": { 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_step", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.5", 2024-09-10T10:54:04.168 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0.001", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "0.999", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "end_time": { 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_time", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.169 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2359", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "end_weekday": { 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_weekday", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.171 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.172 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.173 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.174 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.175 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.176 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "min_score": { 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_score", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.177 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "upmap", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "crush-compat", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "none", 2024-09-10T10:54:04.178 INFO:teuthology.orchestra.run.smithi143.stdout: "read", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap-read" 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Balancer mode", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_ids": { 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_ids", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.179 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.180 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_deviation": { 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_deviation", 2024-09-10T10:54:04.181 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_optimizations": { 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.182 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm", 2024-09-10T10:54:04.183 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_down_multiplier": { 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_down_multiplier", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:54:04.184 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_refresh_rate": { 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_refresh_rate", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "20", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.185 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_starting_port": { 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_starting_port", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4721", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.186 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_ptrace": { 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_ptrace", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.187 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_interval": { 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_interval", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.188 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.7", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.189 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm_log_destination": { 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm_log_destination", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.190 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "file", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "file,syslog", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "syslog" 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "cgroups_split": { 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cgroups_split", 2024-09-10T10:54:04.191 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.192 INFO:teuthology.orchestra.run.smithi143.stdout: "config_checks_enabled": { 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_checks_enabled", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.193 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "config_dashboard": { 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_dashboard", 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.194 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_alertmanager": { 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_alertmanager", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.195 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_base": { 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_base", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.196 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_elasticsearch": { 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.197 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "elasticsearch container image", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_grafana": { 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_grafana", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.198 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_haproxy": { 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_haproxy", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.199 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "HAproxy container image", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_agent": { 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:54:04.200 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_collector": { 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.201 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_query": { 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.202 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger query container image", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_keepalived": { 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_keepalived", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.203 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Keepalived container image", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_loki": { 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_loki", 2024-09-10T10:54:04.204 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Loki container image", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_node_exporter": { 2024-09-10T10:54:04.205 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_node_exporter", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_nvmeof": { 2024-09-10T10:54:04.206 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_nvmeof", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Nvme-of container image", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.207 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_prometheus": { 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_prometheus", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.208 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_promtail": { 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_promtail", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Promtail container image", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.209 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_samba": { 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_samba", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.210 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_snmp_gateway": { 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.211 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "container_init": { 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_init", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:54:04.212 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "daemon_cache_timeout": { 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.213 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "900", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.214 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "default_registry": { 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_registry", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.215 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "device_cache_timeout": { 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_cache_timeout", 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.216 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1800", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "device_enhanced_scan": { 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_enhanced_scan", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.217 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "facts_cache_timeout": { 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "facts_cache_timeout", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.218 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "grafana_dashboards_path": { 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.219 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "host_check_interval": { 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "host_check_interval", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.220 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "hw_monitoring": { 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hw_monitoring", 2024-09-10T10:54:04.221 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "inventory_list_all": { 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "inventory_list_all", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.222 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.223 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.224 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "log_refresh_metadata": { 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_refresh_metadata", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:54:04.225 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.226 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.227 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.228 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:54:04.229 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:54:04.230 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "*", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "max_count_per_host": { 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_count_per_host", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.231 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "max_osd_draining_count": { 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_osd_draining_count", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.232 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "migration_current": { 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "migration_current", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "internal - do not modify", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.234 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "root", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm-package", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "root" 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "oob_default_addr": { 2024-09-10T10:54:04.235 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "oob_default_addr", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "169.254.1.1", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.236 INFO:teuthology.orchestra.run.smithi143.stdout: "prometheus_alerts_path": { 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.237 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_insecure": { 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_insecure", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.238 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_password": { 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_password", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.239 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_url": { 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_url", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.240 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_username": { 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_username", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.241 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "secure_monitoring_stack": { 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.242 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "service_discovery_port": { 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "service_discovery_port", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8765", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.243 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "ssh_config_file": { 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssh_config_file", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:54:04.244 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "use_agent": { 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_agent", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:54:04.245 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "use_repo_digest": { 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_repo_digest", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.246 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_failed_host_check": { 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.247 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_daemons": { 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.248 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_hosts": { 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.249 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crash", 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.250 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.251 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.252 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.253 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.254 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.255 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "retain_interval": { 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retain_interval", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "31536000", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.256 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_recent_interval": { 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_recent_interval", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1209600", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:54:04.257 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dashboard", 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:54:04.258 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:54:04.259 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:54:04.260 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:54:04.261 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:54:04.262 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.263 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.264 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:54:04.265 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:54:04.266 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:54:04.267 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:54:04.268 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:54:04.269 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:54:04.270 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.271 INFO:teuthology.orchestra.run.smithi143.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.272 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.273 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.274 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_URL": { 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.275 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.276 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.277 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.278 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.279 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.280 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.281 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.282 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.283 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.284 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.285 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.286 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:54:04.287 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:54:04.288 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.289 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.290 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.291 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.292 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.293 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8", 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "45", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.295 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.296 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:04.297 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.299 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.300 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.301 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.302 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "cross_origin_url": { 2024-09-10T10:54:04.303 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cross_origin_url", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.304 INFO:teuthology.orchestra.run.smithi143.stdout: "crt_file": { 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crt_file", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.305 INFO:teuthology.orchestra.run.smithi143.stdout: "debug": { 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "debug", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.306 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "jwt_token_ttl": { 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "jwt_token_ttl", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "28800", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.307 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.308 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.309 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.310 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.311 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.312 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.313 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "motd": { 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "motd", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.314 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "The message of the day", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.315 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.316 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.317 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8080", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl_server_port": { 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl_server_port", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8443", 2024-09-10T10:54:04.319 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "redirect", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.320 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect" 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2024-09-10T10:54:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.322 INFO:teuthology.orchestra.run.smithi143.stdout: "url_prefix": { 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url_prefix", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_monitoring": { 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_monitoring", 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.324 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "monitor device health metrics", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.325 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.326 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.327 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.328 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.329 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "mark_out_threshold": { 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mark_out_threshold", 2024-09-10T10:54:04.330 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2419200", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_name": { 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_name", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.331 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "device_health_metrics", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "retention_period": { 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retention_period", 2024-09-10T10:54:04.332 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15552000", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_frequency": { 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_frequency", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.333 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "self_heal": { 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "self_heal", 2024-09-10T10:54:04.334 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.335 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.336 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_threshold": { 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_threshold", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7257600", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.337 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "diskprediction_local", 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.338 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.339 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.340 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.341 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.342 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.343 INFO:teuthology.orchestra.run.smithi143.stdout: "predict_interval": { 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predict_interval", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.344 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "predictor_model": { 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predictor_model", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "prophetstor", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.345 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.346 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "influx", 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": false, 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "batch_size": { 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "batch_size", 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.347 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5000", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "database": { 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "database", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.348 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "ceph", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": { 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hostname", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.349 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.350 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "30", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "5", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.351 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.352 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.353 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.355 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "password": { 2024-09-10T10:54:04.356 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "password", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "port": { 2024-09-10T10:54:04.357 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "port", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8086", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server port", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.358 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "false", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.359 INFO:teuthology.orchestra.run.smithi143.stdout: "threads": { 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threads", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "32", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.360 INFO:teuthology.orchestra.run.smithi143.stdout: "username": { 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "username", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.361 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "verify_ssl": { 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "verify_ssl", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "true", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.362 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "insights", 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.363 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.364 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.365 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.366 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.367 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.368 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "iostat", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.369 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.370 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.371 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.372 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.373 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.374 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "k8sevents", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_event_retention_days": { 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:54:04.375 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "config_check_secs": { 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_check_secs", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "10", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.376 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.377 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.378 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.379 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.380 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.381 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "localpool", 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.382 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "failure_domain": { 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "failure_domain", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "host", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:54:04.383 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.384 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.385 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.386 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.388 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.389 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "min_size": { 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_size", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.390 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "num_rep": { 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "num_rep", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3", 2024-09-10T10:54:04.391 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_num": { 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_num", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.392 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "128", 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "prefix": { 2024-09-10T10:54:04.393 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prefix", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.394 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "subtree": { 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subtree", 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "rack", 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.395 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mds_autoscaler", 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.396 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.397 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.398 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.399 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.400 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.401 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirroring", 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.402 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.403 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.404 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.405 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.406 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.407 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "nfs", 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.408 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.409 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.410 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.411 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.412 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.413 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "fail_fs": { 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "fail_fs", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.415 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.416 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.417 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.418 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.419 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator": { 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.420 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm", 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "rook", 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "test_orchestrator" 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Orchestrator backend", 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.421 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_perf_query", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.422 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.423 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.424 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.425 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.426 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_support", 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.427 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.429 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.430 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.431 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.432 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_autoscaler", 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.433 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.435 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.436 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.437 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.438 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.439 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "threshold": { 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threshold", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1.0", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "scaling threshold", 2024-09-10T10:54:04.440 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "progress", 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_pg_recovery_event": { 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:54:04.441 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2024-09-10T10:54:04.442 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.443 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.444 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.445 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.446 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.447 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.448 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "max_completed_events": { 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_completed_events", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "50", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:54:04.449 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.450 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus", 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "cache": { 2024-09-10T10:54:04.451 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cache", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.452 INFO:teuthology.orchestra.run.smithi143.stdout: "exclude_perf_counters": { 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "exclude_perf_counters", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools": { 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "300", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_interval": { 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_interval", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15.0", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.462 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "9283", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.463 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "stale_cache_strategy": { 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stale_cache_strategy", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "log", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.464 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "default", 2024-09-10T10:54:04.465 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "default", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "error" 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.466 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.467 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.468 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.469 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.470 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.471 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.472 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_snap_create": { 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.473 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:54:04.474 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "trash_purge_schedule": { 2024-09-10T10:54:04.475 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "trash_purge_schedule", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.476 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "restful", 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_auth": { 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_auth", 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.477 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.478 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.479 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.480 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.481 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.482 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.483 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.484 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.485 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.486 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rgw", 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.487 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.488 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.489 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.490 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.491 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.492 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rook", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.493 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.494 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.495 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.496 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.497 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.498 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "storage_class": { 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "storage_class", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "local", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:54:04.499 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "selftest", 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.500 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.501 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.502 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.503 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.504 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.505 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "roption1": { 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption1", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.506 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "roption2": { 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption2", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "xyz", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.507 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption1": { 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption1", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.508 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption2": { 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption2", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.509 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption3": { 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption3", 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.510 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.511 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.511 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.511 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.511 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.511 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption4": { 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption4", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.512 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption5": { 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption5", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.513 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption6": { 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption6", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.514 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption7": { 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption7", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "42", 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.515 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "testkey": { 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testkey", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.516 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "testlkey": { 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testlkey", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.517 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "testnewline": { 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testnewline", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.518 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snap_schedule", 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_m_granularity": { 2024-09-10T10:54:04.519 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_m_granularity", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.520 INFO:teuthology.orchestra.run.smithi143.stdout: "dump_on_update": { 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dump_on_update", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.521 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.522 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.523 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.524 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.525 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.526 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stats", 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.527 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.528 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.529 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.530 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.531 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.532 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "status", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.533 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.534 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.535 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.536 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.537 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telegraf", 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.538 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "address": { 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "address", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.539 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.540 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.541 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.542 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.543 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.544 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.545 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telemetry", 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.546 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_basic": { 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_basic", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.547 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_crash": { 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_crash", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.548 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_device": { 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_device", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.549 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_ident": { 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_ident", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.550 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_perf": { 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_perf", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:54:04.551 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "contact": { 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "contact", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.552 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "description": { 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "description", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.553 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "device_url": { 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_url", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.554 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.555 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "24", 2024-09-10T10:54:04.556 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "8", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "last_opt_revision": { 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "last_opt_revision", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.557 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard": { 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.558 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard_description": { 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard_description", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.559 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.560 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.561 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.562 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.563 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.564 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "organization": { 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "organization", 2024-09-10T10:54:04.565 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "proxy": { 2024-09-10T10:54:04.566 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "proxy", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.567 INFO:teuthology.orchestra.run.smithi143.stdout: "url": { 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "test_orchestrator", 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.569 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.570 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.572 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.573 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.574 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.575 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.576 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.577 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.578 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.579 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_clones": { 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_clones", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.580 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "periodic_async_work": { 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "periodic_async_work", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.581 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Periodically check for async work", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_delay": { 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.582 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.583 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix", 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.584 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "discovery_interval": { 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "discovery_interval", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "100", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.585 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "identifier": { 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "identifier", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.586 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.588 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.589 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.590 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.591 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.592 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_host": { 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_host", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.593 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_port": { 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_port", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.594 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10051", 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.595 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_sender": { 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_sender", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.596 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm", 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "dashboard", 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.597 INFO:teuthology.orchestra.run.smithi143.stdout: "available_modules": [ 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "alerts", 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.598 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.599 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.600 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.601 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.602 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.603 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.604 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_destination": { 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_destination", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.605 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_from_name": { 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_from_name", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "Ceph", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Email From: name", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.606 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_host": { 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_host", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP server", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_password": { 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_password", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Password to authenticate with", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.608 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_port": { 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_port", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "465", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP port", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.609 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_sender": { 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_sender", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.610 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_ssl": { 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_ssl", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.611 INFO:teuthology.orchestra.run.smithi143.stdout: "smtp_user": { 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "smtp_user", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "User to authenticate as", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.612 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "balancer", 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "active": { 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "active", 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.613 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_time": { 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_time", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0000", 2024-09-10T10:54:04.614 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "begin_weekday": { 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "begin_weekday", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.615 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_max_iterations": { 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.616 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "25", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "250", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_metrics": { 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_metrics", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.617 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "crush_compat_step": { 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crush_compat_step", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.618 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.5", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0.001", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "0.999", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "end_time": { 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_time", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.619 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2359", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "end_weekday": { 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "end_weekday", 2024-09-10T10:54:04.620 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "0", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "6", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.621 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.622 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.623 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.624 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.625 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.626 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "min_score": { 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_score", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:54:04.627 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "upmap", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.628 INFO:teuthology.orchestra.run.smithi143.stdout: "crush-compat", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "none", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "read", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap-read" 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Balancer mode", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_ids": { 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_ids", 2024-09-10T10:54:04.629 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.630 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_deviation": { 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_deviation", 2024-09-10T10:54:04.631 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "upmap_max_optimizations": { 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:54:04.632 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.633 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_down_multiplier": { 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_down_multiplier", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.634 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_refresh_rate": { 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_refresh_rate", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "20", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.635 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "agent_starting_port": { 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "agent_starting_port", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4721", 2024-09-10T10:54:04.636 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_ptrace": { 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_ptrace", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.637 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_interval": { 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_interval", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.638 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.639 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0.7", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm_log_destination": { 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephadm_log_destination", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.640 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "file", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "file,syslog", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "syslog" 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.641 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "cgroups_split": { 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cgroups_split", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.642 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "config_checks_enabled": { 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_checks_enabled", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.643 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "config_dashboard": { 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_dashboard", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.644 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_alertmanager": { 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_alertmanager", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.645 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_base": { 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_base", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.646 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_elasticsearch": { 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "elasticsearch container image", 2024-09-10T10:54:04.647 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_grafana": { 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_grafana", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.648 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_haproxy": { 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_haproxy", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "HAproxy container image", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_agent": { 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.650 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_collector": { 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.651 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_jaeger_query": { 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:54:04.652 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Jaeger query container image", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_keepalived": { 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_keepalived", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:54:04.653 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Keepalived container image", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_loki": { 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_loki", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:54:04.654 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Loki container image", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_node_exporter": { 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_node_exporter", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:54:04.655 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_nvmeof": { 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_nvmeof", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.656 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Nvme-of container image", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_prometheus": { 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_prometheus", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.657 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Prometheus container image", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_promtail": { 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_promtail", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.658 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Promtail container image", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_samba": { 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_samba", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.659 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "container_image_snmp_gateway": { 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.660 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "container_init": { 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "container_init", 2024-09-10T10:54:04.661 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: "daemon_cache_timeout": { 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.663 INFO:teuthology.orchestra.run.smithi143.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "900", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.664 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "default_registry": { 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "default_registry", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "docker.io", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.665 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "device_cache_timeout": { 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_cache_timeout", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1800", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.666 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "device_enhanced_scan": { 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_enhanced_scan", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.667 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "facts_cache_timeout": { 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "facts_cache_timeout", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:54:04.668 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "grafana_dashboards_path": { 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:54:04.669 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "host_check_interval": { 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "host_check_interval", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:54:04.670 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "hw_monitoring": { 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hw_monitoring", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:54:04.671 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "inventory_list_all": { 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "inventory_list_all", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.672 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.673 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.674 INFO:teuthology.orchestra.run.smithi143.stdout: "log_refresh_metadata": { 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_refresh_metadata", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.675 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.677 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.678 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.679 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.680 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "*", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "max_count_per_host": { 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_count_per_host", 2024-09-10T10:54:04.681 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "max_osd_draining_count": { 2024-09-10T10:54:04.682 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_osd_draining_count", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.683 INFO:teuthology.orchestra.run.smithi143.stdout: "migration_current": { 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "migration_current", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "internal - do not modify", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.684 INFO:teuthology.orchestra.run.smithi143.stdout: "mode": { 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mode", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "root", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm-package", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "root" 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:54:04.685 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "oob_default_addr": { 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "oob_default_addr", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "169.254.1.1", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:54:04.686 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "prometheus_alerts_path": { 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:54:04.687 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_insecure": { 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_insecure", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:54:04.688 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_password": { 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_password", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.689 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_url": { 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_url", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.690 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "registry_username": { 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "registry_username", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:54:04.691 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "secure_monitoring_stack": { 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.692 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "service_discovery_port": { 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "service_discovery_port", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8765", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.693 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "ssh_config_file": { 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssh_config_file", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.694 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "use_agent": { 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_agent", 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.695 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "use_repo_digest": { 2024-09-10T10:54:04.696 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "use_repo_digest", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.697 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_failed_host_check": { 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.698 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_daemons": { 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.699 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_on_stray_hosts": { 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.700 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crash", 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.701 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.702 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.703 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.704 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.705 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "retain_interval": { 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retain_interval", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.706 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "31536000", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_recent_interval": { 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_recent_interval", 2024-09-10T10:54:04.707 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1209600", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.708 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dashboard", 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.709 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.710 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.711 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.712 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.713 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:54:04.714 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:54:04.715 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.716 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.717 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.718 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.719 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.720 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.721 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.722 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.723 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.724 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.725 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_URL": { 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.726 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.727 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.728 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.729 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.730 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.731 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.732 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.733 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.734 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:54:04.735 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:54:04.736 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:54:04.737 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.738 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.739 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.740 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.741 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.742 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.743 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.744 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.745 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "45", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.746 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.747 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.748 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "admin", 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:54:04.749 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:54:04.750 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:54:04.751 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.752 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:54:04.753 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "cross_origin_url": { 2024-09-10T10:54:04.754 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cross_origin_url", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "crt_file": { 2024-09-10T10:54:04.755 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "crt_file", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.756 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "debug": { 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "debug", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.757 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "jwt_token_ttl": { 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "jwt_token_ttl", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "28800", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.758 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.759 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.760 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.761 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.762 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.763 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.764 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "motd": { 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "motd", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.765 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "The message of the day", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.766 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.767 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8080", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.769 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl_server_port": { 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl_server_port", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8443", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.770 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "redirect", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.771 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "redirect" 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.772 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "url_prefix": { 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url_prefix", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.773 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.774 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_monitoring": { 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_monitoring", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.775 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "monitor device health metrics", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.776 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.777 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.778 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.779 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.780 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "mark_out_threshold": { 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mark_out_threshold", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "2419200", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "pool_name": { 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pool_name", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.782 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "device_health_metrics", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "retention_period": { 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "retention_period", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.783 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15552000", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_frequency": { 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_frequency", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.784 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "self_heal": { 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "self_heal", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.785 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.786 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "warn_threshold": { 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "warn_threshold", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.787 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7257600", 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.788 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "diskprediction_local", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.789 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.790 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.791 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.792 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.793 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.796 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "predict_interval": { 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predict_interval", 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "86400", 2024-09-10T10:54:04.797 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "predictor_model": { 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "predictor_model", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.798 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "prophetstor", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.799 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "600", 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.800 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "influx", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": false, 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "batch_size": { 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "batch_size", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5000", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.801 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "database": { 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "database", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "ceph", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.802 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": { 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "hostname", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.803 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.804 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "30", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "5", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.805 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.806 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.807 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.808 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.809 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "password": { 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "password", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "port": { 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "port", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.811 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "8086", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "InfluxDB server port", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "ssl": { 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ssl", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.812 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "false", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "threads": { 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threads", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.813 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "32", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "username": { 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "username", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.814 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "verify_ssl": { 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "verify_ssl", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.815 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "true", 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "insights", 2024-09-10T10:54:04.816 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.817 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.818 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.819 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.820 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.821 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "iostat", 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.822 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.823 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.824 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.825 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.826 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.827 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "k8sevents", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_event_retention_days": { 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "7", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.828 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "config_check_secs": { 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "config_check_secs", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "10", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.829 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.830 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.831 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.832 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.833 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.834 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "localpool", 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.835 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "failure_domain": { 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "failure_domain", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "host", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.836 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.837 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.838 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.839 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.840 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.841 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "min_size": { 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "min_size", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:54:04.842 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "num_rep": { 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "num_rep", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.843 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_num": { 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_num", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "128", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:54:04.844 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "prefix": { 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prefix", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:54:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "subtree": { 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subtree", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "rack", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:54:04.846 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mds_autoscaler", 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.847 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.850 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.851 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.852 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirroring", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.853 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.854 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.855 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.856 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.857 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "nfs", 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.858 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.859 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.860 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.861 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.862 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.863 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "fail_fs": { 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "fail_fs", 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.864 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.865 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.866 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.867 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.868 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator": { 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "orchestrator", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "cephadm", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "rook", 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "test_orchestrator" 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.871 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Orchestrator backend", 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_perf_query", 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.872 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.874 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.875 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.876 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.877 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "osd_support", 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.878 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.881 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.882 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.883 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "pg_autoscaler", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.884 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.885 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.886 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.887 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.888 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:04.889 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "threshold": { 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "threshold", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "3.0", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1.0", 2024-09-10T10:54:04.890 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "scaling threshold", 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "progress", 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_pg_recovery_event": { 2024-09-10T10:54:04.891 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2024-09-10T10:54:04.892 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.893 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.894 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.896 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.898 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "max_completed_events": { 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_completed_events", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "50", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.899 INFO:teuthology.orchestra.run.smithi143.stdout: "sleep_interval": { 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sleep_interval", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "5", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.900 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "prometheus", 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "cache": { 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cache", 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.901 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "exclude_perf_counters": { 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "exclude_perf_counters", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.902 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.903 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.904 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.905 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.906 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.907 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.908 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools": { 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.909 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "300", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.910 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "scrape_interval": { 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "scrape_interval", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15.0", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.911 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "::", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.912 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "9283", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.913 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "stale_cache_strategy": { 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stale_cache_strategy", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "log", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.914 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_behaviour": { 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_behaviour", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "default", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "default", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "error" 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.915 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "standby_error_status_code": { 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "standby_error_status_code", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "500", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "400", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "599", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.916 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.917 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.920 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.921 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_snap_create": { 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.922 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.923 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "trash_purge_schedule": { 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "trash_purge_schedule", 2024-09-10T10:54:04.924 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.925 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "restful", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "enable_auth": { 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enable_auth", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.926 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "key_file": { 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "key_file", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.927 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.928 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.929 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.930 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.931 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.932 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "server_addr": { 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_addr", 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.933 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "server_port": { 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "server_port", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.934 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rgw", 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.935 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.936 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.937 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.938 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.939 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.940 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rook", 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.941 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.942 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.943 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.944 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.945 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.946 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "storage_class": { 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "storage_class", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "local", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.947 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "selftest", 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.948 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.949 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.950 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.951 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.952 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "roption1": { 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption1", 2024-09-10T10:54:04.953 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.954 INFO:teuthology.orchestra.run.smithi143.stdout: "roption2": { 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "roption2", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "xyz", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.955 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption1": { 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption1", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.956 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption2": { 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption2", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.957 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption3": { 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption3", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "float", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.958 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption4": { 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption4", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.959 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption5": { 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption5", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.960 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption6": { 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption6", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.961 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "rwoption7": { 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rwoption7", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "1", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "42", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.962 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "testkey": { 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testkey", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.963 INFO:teuthology.orchestra.run.smithi143.stdout: "testlkey": { 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testlkey", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.964 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "testnewline": { 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "testnewline", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.965 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snap_schedule", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "allow_m_granularity": { 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "allow_m_granularity", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.966 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "dump_on_update": { 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "dump_on_update", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.967 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.968 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.969 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.970 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.971 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.972 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "stats", 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.973 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.974 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.975 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.976 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.978 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "status", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.979 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.980 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.981 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.982 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.983 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telegraf", 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "address": { 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "address", 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.984 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:04.985 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "15", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.986 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.987 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.988 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:04.989 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.990 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "telemetry", 2024-09-10T10:54:04.991 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_basic": { 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_basic", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.992 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_crash": { 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_crash", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.993 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_device": { 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_device", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.994 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_ident": { 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_ident", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.995 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "channel_perf": { 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "channel_perf", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:54:04.996 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "contact": { 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "contact", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:04.997 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "description": { 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "description", 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.000 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "device_url": { 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "device_url", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.001 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "enabled": { 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "enabled", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.002 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:05.003 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "24", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "8", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "last_opt_revision": { 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "last_opt_revision", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:05.004 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "1", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard": { 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.005 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "leaderboard_description": { 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "leaderboard_description", 2024-09-10T10:54:05.006 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:05.007 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.008 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.009 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.010 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.011 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.012 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "organization": { 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "organization", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.013 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "proxy": { 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "proxy", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.014 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "url": { 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "url", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.015 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "test_orchestrator", 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.016 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.017 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.018 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.019 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.020 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.021 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.022 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.023 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:05.024 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.025 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.026 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "max_concurrent_clones": { 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "max_concurrent_clones", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "4", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.027 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "periodic_async_work": { 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "periodic_async_work", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.028 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Periodically check for async work", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_delay": { 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "0", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.029 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "True", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.030 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix", 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "can_run": true, 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "error_string": "", 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "module_options": { 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "discovery_interval": { 2024-09-10T10:54:05.031 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "discovery_interval", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "uint", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "100", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "identifier": { 2024-09-10T10:54:05.032 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "identifier", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.033 INFO:teuthology.orchestra.run.smithi143.stdout: "interval": { 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "interval", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "secs", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "60", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.034 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "log_level": { 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_level", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.035 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster": { 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.036 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_cluster_level": { 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_cluster_level", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.037 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "info", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [ 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "critical", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "debug", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "error", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "info", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "warning" 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.038 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "log_to_file": { 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "log_to_file", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "bool", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 1, 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "False", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.039 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_host": { 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_host", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_port": { 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_port", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "int", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "10051", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.041 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "zabbix_sender": { 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "zabbix_sender", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "str", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "level": "advanced", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "flags": 0, 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "min": "", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "max": "", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "enum_allowed": [], 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "desc": "", 2024-09-10T10:54:05.042 INFO:teuthology.orchestra.run.smithi143.stdout: "long_desc": "", 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "tags": [], 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "see_also": [] 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "dashboard": "https://172.21.15.143:8443/" 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "always_on_modules": { 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "octopus": [ 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2024-09-10T10:54:05.043 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "pacific": [ 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2024-09-10T10:54:05.044 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "quincy": [ 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2024-09-10T10:54:05.045 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "reef": [ 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2024-09-10T10:54:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "squid": [ 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "balancer", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "crash", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "devicehealth", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "orchestrator", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "pg_autoscaler", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "progress", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "rbd_support", 2024-09-10T10:54:05.047 INFO:teuthology.orchestra.run.smithi143.stdout: "status", 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "telemetry", 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "volumes" 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "last_failure_osd_epoch": 3, 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "active_clients": [ 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "devicehealth", 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2024-09-10T10:54:05.048 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2758892857 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "libcephsqlite", 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1417438014 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:05.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "rbd_support", 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2940924984 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "volumes", 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: "addrvec": [ 2024-09-10T10:54:05.050 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v2", 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 507656575 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-09-10T10:54:05.051 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:54:05.052 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1273502531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.052 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3652500895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/457937715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:03 smithi144 ceph-mon[34230]: pgmap v621: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:03 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3739738096' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1273502531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3652500895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/457937715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:03 smithi178 ceph-mon[34813]: pgmap v621: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T10:54:05.053 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:03 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3739738096' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f252c0a2b90 msgr2=0x7f252c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f252c0a2b90 0x7f252c0a5050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2540102b80 tx=0x7f253c005e70 comp rx=0 tx=0).stop 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 msgr2=0x7f254019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 0x7f254019f380 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f253800b970 tx=0x7f253800be30 comp rx=0 tx=0).stop 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 shutdown_connections 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f252c0a2b90 0x7f252c0a5050 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25401036e0 0x7f254019f380 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:05.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2540102d20 0x7f2540198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:05.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 --2- 172.21.15.143:0/3739738096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101b30 0x7f2540197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:05.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 >> 172.21.15.143:0/3739738096 conn(0x7f25400fd2b0 msgr2=0x7f2540106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:05.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 shutdown_connections 2024-09-10T10:54:05.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:03.679+0000 7f25478f8640 1 -- 172.21.15.143:0/3739738096 wait complete. 2024-09-10T10:54:05.061 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T10:54:05.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T10:54:05.062 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1273502531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.062 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3652500895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.062 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/457937715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:54:05.062 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:03 smithi143 ceph-mon[31057]: pgmap v621: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T10:54:05.062 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:03 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3739738096' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:54:05.322 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1422, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1422, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 189172485, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 451373, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 8, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.238621106, 2024-09-10T10:54:05.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.036922534, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.153576602, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.036829697, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1421, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.046832290, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000032957 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 792, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 932, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 205734546, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 569478, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 6, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.200483487, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.024491174, 2024-09-10T10:54:05.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.149259268, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.019872429, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 930, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.026941027, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000028968 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 317872 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 640, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 621, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 149338204, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 251094, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.136921752, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.021440961, 2024-09-10T10:54:05.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.109471347, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.009781103, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 620, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.019988215, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000032239 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 857104, 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 114672 2024-09-10T10:54:05.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4120, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.355822699, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000086364 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.847 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.848 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 17562, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 2409, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 15153, 2024-09-10T10:54:05.849 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 14120, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:05.850 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:05.851 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:05.852 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:05.853 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:05.854 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 416091, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-09-10T10:54:05.855 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 140, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 35, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 389226589, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 546825710, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 546824572, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.856 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 416091 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 13, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 17844, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 17831, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 11, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 706, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 695, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 17562, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 45767, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 28205, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-09-10T10:54:05.857 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 91016, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215328 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.858 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-09-10T10:54:05.859 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.860 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.861 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.862 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.863 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-09-10T10:54:05.863 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.863 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.863 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.863 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.864 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.865 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.866 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.867 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.868 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.869 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.870 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 1, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 2, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 1, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 1, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 1571, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-09-10T10:54:05.871 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:05.872 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:05.873 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 239431, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 78, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:05.874 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 162, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 12407240, 2024-09-10T10:54:05.875 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 70376, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 2638, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 2638, 2024-09-10T10:54:05.877 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2638, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2.313938751, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000877156 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2638, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2639 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 2638, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 2638, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-09-10T10:54:05.878 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 1567, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 1069, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:05.879 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-09-10T10:54:05.880 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-09-10T10:54:05.881 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 89, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 2, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 11, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 11, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-09-10T10:54:05.882 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-09-10T10:54:05.883 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 2850, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 543974921, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 2850, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:05.884 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2850, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 543974921, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:05.885 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2638, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 542659067, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2638, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 542659067, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:05.886 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2638, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 2638, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2638, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2638, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.887 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:05.888 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-09-10T10:54:05.890 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-09-10T10:54:05.890 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:05.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.890 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.890 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x55bb385dab40": { 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:05.891 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:05.892 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:54:06.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:05 smithi143 ceph-mon[31057]: pgmap v622: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:06.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:05 smithi178 ceph-mon[34813]: pgmap v622: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:06.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:05 smithi144 ceph-mon[34230]: pgmap v622: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:06.766 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T10:54:06.766 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T10:54:06.766 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-10T10:54:06.766 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-10T10:54:06.766 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-10T10:54:06.766 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T10:54:06.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T10:54:07.023 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:07.563 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:54:07.563 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:07.563 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 195, 2024-09-10T10:54:07.563 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 190, 2024-09-10T10:54:07.563 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 746512, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 116688, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 0, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.029209536, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.012456723, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.021811522, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 189, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.007795843, 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000041247 2024-09-10T10:54:07.564 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 746512, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 116688 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 145, 2024-09-10T10:54:07.565 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 160, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 104064, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 3, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 2, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.010486313, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.008182051, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.001073943, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 143, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.006823814, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000047718 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.566 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 104064 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 4, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 4, 2024-09-10T10:54:07.567 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 3040, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 448, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 2, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.002261543, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.000035750, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.002217176, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.003990094, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001330031 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 0, 2024-09-10T10:54:07.569 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:07.570 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:07.571 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:07.572 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 148984, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 30, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 0, 2024-09-10T10:54:07.573 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 928, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 2, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:07.574 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 196, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 718981, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 196, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 196, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 718981, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:07.575 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:07.576 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:07.576 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:07.576 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:07.576 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:54:07.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:07 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:54:07.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:07 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:54:07.935 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:07 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:54:08.351 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T10:54:08.351 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T10:54:08.351 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-10T10:54:08.351 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-10T10:54:08.351 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-10T10:54:08.351 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T10:54:08.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T10:54:08.639 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:08.762 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:08 smithi143 ceph-mon[31057]: pgmap v623: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T10:54:08.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:08 smithi178 ceph-mon[34813]: pgmap v623: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T10:54:08.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:08 smithi144 ceph-mon[34230]: pgmap v623: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T10:54:09.191 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:54:09.191 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:09.191 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 641, 2024-09-10T10:54:09.191 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 641, 2024-09-10T10:54:09.191 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 217017, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 203015, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 6, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 2, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.067576541, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.026038161, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.017192401, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.018313070, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 640, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.032762447, 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000051191 2024-09-10T10:54:09.192 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1386, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1367, 2024-09-10T10:54:09.193 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1158037, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 488701, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.095132478, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.030421578, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.040185433, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.025957011, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1366, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.044843209, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000032828 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.194 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 857376, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 115312 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 4, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 145, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1508, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 320629, 2024-09-10T10:54:09.195 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.010880561, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.008101101, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.001645377, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000051403, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 143, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.006740354, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000047135 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:09.196 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 319664 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-09-10T10:54:09.197 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3061, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.071162972, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000023248 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.198 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:09.199 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-09-10T10:54:09.200 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:09.201 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:09.202 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-09-10T10:54:09.203 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:09.204 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:09.205 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 5, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 2, 2024-09-10T10:54:09.206 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 4195921, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 4195780, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 5 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 3, 2024-09-10T10:54:09.207 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 3, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 1, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 1, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 48644, 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215328 2024-09-10T10:54:09.208 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.209 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.210 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.211 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.212 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.213 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.214 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.215 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.216 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.217 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:09.218 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.219 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:09.220 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 1, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 2, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 1, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 2, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-09-10T10:54:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 2619, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:09.222 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 197441, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 54, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 162, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 8512, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 13, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 1814, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 1814, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1814, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.947974856, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000522588 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1814, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 1815 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 1814, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 1814, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 1169, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 643, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 89, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 2, 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 6, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 6, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 2027, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 1180266, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 2027, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2027, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1180266, 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 1814, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 1622, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 1814, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1622, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:09.236 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 1814, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 1814, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 1814, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1814, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.237 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x55df6ba2ab40": { 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:09.239 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:09.240 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:54:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:54:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:54:09.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:09 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:54:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:54:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:54:09.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:09 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:54:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:54:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:54:09.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:09 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' 2024-09-10T10:54:10.037 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-10T10:54:10.037 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-10T10:54:10.037 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-10T10:54:10.037 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-10T10:54:10.037 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-10T10:54:10.037 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T10:54:10.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T10:54:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:10.731 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:10 smithi143 ceph-mon[31057]: pgmap v624: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:10.839 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 197, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 192, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 747056, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 117968, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 0, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.028856343, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.011632168, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.022470753, 2024-09-10T10:54:10.840 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 191, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.008262019, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000043256 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 747056, 2024-09-10T10:54:10.841 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 117968 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 5, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 148, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 3200, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 104688, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 3, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 2, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.012325084, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.008044003, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.003047241, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:54:10.842 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 147, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.010025431, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000068200 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 104688 2024-09-10T10:54:10.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 0, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 2, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 0, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 2, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.000345247, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.000271164, 2024-09-10T10:54:10.844 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.845 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:10.846 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:10.847 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:10.848 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 148967, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 29, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:10.849 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 928, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 2, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 198, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 719241, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 198, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 198, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 719241, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-09-10T10:54:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-09-10T10:54:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-09-10T10:54:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-09-10T10:54:10.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:10 smithi178 ceph-mon[34813]: pgmap v624: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:10.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:10 smithi144 ceph-mon[34230]: pgmap v624: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:11.687 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T10:54:11.688 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T10:54:11.688 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-10T10:54:11.688 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-10T10:54:11.688 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-10T10:54:11.688 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T10:54:11.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T10:54:11.949 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:54:12.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:11 smithi143 ceph-mon[31057]: pgmap v625: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T10:54:12.070 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:11 smithi144 ceph-mon[34230]: pgmap v625: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T10:54:12.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:11 smithi178 ceph-mon[34813]: pgmap v625: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T10:54:12.482 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 32585, 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 33379, 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 10701605, 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 5705359, 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 28, 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-09-10T10:54:12.483 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 2.186438052, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.780145175, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 1.938045183, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.009072464, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 33377, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.533519304, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000015984 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.484 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 64295, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.009935667, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000154 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 888512, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 121760 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1614, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1297, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 531195, 2024-09-10T10:54:12.485 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 769740, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 9, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.103369496, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.040775275, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.067010976, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.005797654, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1297, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.055483533, 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000042778 2024-09-10T10:54:12.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2483, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000406086, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000163 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 321296 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 12358, 2024-09-10T10:54:12.487 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 7906, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 6139863, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 1901447, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 20, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 9, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.626387087, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.233078483, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.529344787, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.005963769, 2024-09-10T10:54:12.488 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 7904, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.162226361, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020524 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 19656, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002804773, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000142 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:54:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 712, 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.263726379, 2024-09-10T10:54:12.490 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000370402 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 246, 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.015318872, 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000062271 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T10:54:12.491 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 31224, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 1427, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1427, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 66.695000200, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.046737911 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 29797, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:12.492 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 20, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 1, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 27, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 8, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 9, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 10, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 13, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 248, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 2, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 2, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 4, 2024-09-10T10:54:12.493 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 34241, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 2846, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 29797, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 132, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 8, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 2, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 1, 2024-09-10T10:54:12.494 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 4, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 122, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 1084, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 21251863, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 122, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:12.495 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 122, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 579, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 431, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 581, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 118, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 91, 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 711 2024-09-10T10:54:12.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 24, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 1153, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 632, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 680, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 24, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 120, 2024-09-10T10:54:12.497 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 120, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 6, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 459, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:12.498 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 8, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 1, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 873, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 873, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-09-10T10:54:12.499 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 2, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 5144412, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 259, 2024-09-10T10:54:12.500 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.607468157, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002345436 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 32, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 272, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 240, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 31, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 689, 2024-09-10T10:54:12.501 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 658, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 27, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 633, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 606, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 2, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 208, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 206, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 59144, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223520 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 32928, 2024-09-10T10:54:12.502 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 20, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 31224, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 8, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 713, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 118, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.779682773, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.006607481 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001335248, 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000667624 2024-09-10T10:54:12.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1054, 2024-09-10T10:54:12.504 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 41.275367994, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.039160690 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.505 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.507 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.508 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 15, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.009372522, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000624834 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 118, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 24.462336858, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.207307939 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.509 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:12.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.511 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.512 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 120, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.166904805, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001390873 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 661, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 661, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 533, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:12.515 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:12.516 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:12.517 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 287349, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 105, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-09-10T10:54:12.518 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 162, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 131341, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 484, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-09-10T10:54:12.519 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 737, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 1112671, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 29, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 737, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 737, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.912967313, 2024-09-10T10:54:12.520 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002595613 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 737, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 834 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 737, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 125, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 612, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-09-10T10:54:12.521 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 3, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 262, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 12, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 1, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 252, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 122, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:12.522 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-09-10T10:54:12.523 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-09-10T10:54:12.524 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 89, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 12, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 12, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 121, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 31 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.525 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 463 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 5, 2024-09-10T10:54:12.526 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 240, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 120, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 46553, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 13167299, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:12.527 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 46553, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 46553, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 13167299, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.528 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 737, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1125751, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 737, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1125751, 2024-09-10T10:54:12.529 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:12.530 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 737, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 737, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 737, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 737, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:12.531 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 120, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 13080, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 120, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 13080, 2024-09-10T10:54:12.532 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x564c2cabe960": { 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 873, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 950108, 2024-09-10T10:54:12.533 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 873, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 259, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 950108, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:12.534 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T10:54:13.291 INFO:tasks.check_counter:Daemon mds.b mds.exported=8 2024-09-10T10:54:13.291 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-10T10:54:13.291 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=24 2024-09-10T10:54:13.291 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=1153 2024-09-10T10:54:13.291 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-09-10T10:54:13.291 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T10:54:13.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T10:54:13.547 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:54:13.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:13 smithi144 ceph-mon[34230]: pgmap v626: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-09-10T10:54:14.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:13 smithi143 ceph-mon[31057]: pgmap v626: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-09-10T10:54:14.076 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T10:54:14.076 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:14.076 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1125, 2024-09-10T10:54:14.076 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1103, 2024-09-10T10:54:14.076 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 5340520, 2024-09-10T10:54:14.076 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 405960, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.112979793, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.044679682, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.045148072, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.021689855, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1103, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.049449137, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000044831 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.077 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 861936, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 119120 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 255, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 399, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 63316, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 401030, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-09-10T10:54:14.078 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.027420988, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.010466141, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.006170365, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.007985011, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 397, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.016650445, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000041940 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.079 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 320192 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 924, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 926, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 7772237, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 293839, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-09-10T10:54:14.080 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.065382699, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.016477095, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.022284462, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.021396697, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 924, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.024299486, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000026298 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.081 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3450, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.073487256, 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000021300 2024-09-10T10:54:14.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.083 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 941, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 941, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:14.084 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 941, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 2, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-09-10T10:54:14.085 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 4, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 234, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:14.086 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.087 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:14.088 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-09-10T10:54:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 8306, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 11, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 15926154, 2024-09-10T10:54:14.090 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 15921853, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 8306 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 22, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 1121, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 1099, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 17, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 49, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 32, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 941, 2024-09-10T10:54:14.091 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 2308, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 1367, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 52500, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215328 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-09-10T10:54:14.092 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.093 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.094 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.095 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.096 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.097 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.098 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.100 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.101 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.102 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.103 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 4, 2024-09-10T10:54:14.104 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 3, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 2629, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:14.105 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:14.106 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:14.107 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 198264, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 57, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:14.108 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 162, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 733814, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 4346, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.109 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 2080, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 26, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 4, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 2080, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2080, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.754969980, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000843735 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2080, 2024-09-10T10:54:14.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2081 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 2080, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 2080, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 1280, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 797, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-09-10T10:54:14.111 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 2, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-09-10T10:54:14.112 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-09-10T10:54:14.113 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 89, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 6, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 6, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-09-10T10:54:14.114 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-09-10T10:54:14.115 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 2300, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 12954901, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 2300, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2300, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 12954901, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2080, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 11732453, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2080, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 11732453, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2080, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 2080, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2080, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2080, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:14.120 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55a655582b40": { 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:14.121 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:14.122 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T10:54:14.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:13 smithi178 ceph-mon[34813]: pgmap v626: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-09-10T10:54:14.875 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T10:54:14.875 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T10:54:14.875 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-10T10:54:14.875 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-10T10:54:14.875 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-10T10:54:14.875 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T10:54:14.876 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T10:54:15.121 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:54:15.626 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1539, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1059, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 518890, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 566625, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 25, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.095603083, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.033339323, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.066888887, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003518201, 2024-09-10T10:54:15.627 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1057, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.035352800, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033446 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2501, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000378167, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000151 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.628 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 3704, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 3663, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1918698, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 994281, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 13, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.275617334, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.100545394, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.235322485, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000813381, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3662, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.080605444, 2024-09-10T10:54:15.629 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000022011 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 6847, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001162240, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000169 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 871200, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 123312 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 10509, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 6044, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 5616162, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 1590186, 2024-09-10T10:54:15.630 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 16, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 8, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.530039041, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.199567717, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.454154294, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.001620722, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 6043, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.126389005, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020914 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 16192, 2024-09-10T10:54:15.631 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002364072, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000146 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 6576, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 326608 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 96, 2024-09-10T10:54:15.632 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002060988, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000021468 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000030300, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000015150 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 2886, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 362, 2024-09-10T10:54:15.633 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 362, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 34.094063067, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.094182494 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 2524, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 29, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 9, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 20, 2024-09-10T10:54:15.634 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 13, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 220, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 4, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 3930, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 1285, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 2523, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 122, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-09-10T10:54:15.635 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 1084, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 21251863, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:15.636 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 429, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 75, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 431, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:54:15.637 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 86, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 1224, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 668, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 739, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:15.638 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 6, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 413, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-09-10T10:54:15.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 5, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 5, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 2, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-09-10T10:54:15.640 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4195921, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.006763041, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001690760 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 34, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 144, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 110, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 33, 2024-09-10T10:54:15.641 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 712, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 679, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 29, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 249, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 220, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 81, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 81, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 54028, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223520 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 3524, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 7, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 2886, 2024-09-10T10:54:15.642 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 12, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 263, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000192686, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000192686 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 361, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 34.093870381, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.094442854 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.644 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:15.645 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.646 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.648 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.651 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.652 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-09-10T10:54:15.653 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-09-10T10:54:15.654 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 64, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 64, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 536, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:15.655 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:15.656 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-09-10T10:54:15.657 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 246919, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 91, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:15.658 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 162, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 138797, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 486, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-09-10T10:54:15.659 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 86, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 18083, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 33, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 86, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 86, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.292994979, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.003406918 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 86, 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 161 2024-09-10T10:54:15.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 86, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 86, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 4, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 7, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-09-10T10:54:15.661 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 6, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:15.662 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-09-10T10:54:15.663 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 89, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 9, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 9, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 90, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 24 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.664 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 416 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-09-10T10:54:15.665 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 15748, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 6620327, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 15748, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 15748, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 6620327, 2024-09-10T10:54:15.666 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:15.667 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 86, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 18083, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 86, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 18083, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:15.668 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 86, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 86, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 86, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 86, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.669 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-09-10T10:54:15.670 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5591a9cfc960": { 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 5, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1617, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:15.671 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 5, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 4, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1617, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:15.672 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T10:54:15.936 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:15 smithi144 ceph-mon[34230]: pgmap v627: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:16.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:15 smithi143 ceph-mon[31057]: pgmap v627: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:16.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:15 smithi178 ceph-mon[34813]: pgmap v627: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:16.434 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-10T10:54:16.434 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-10T10:54:16.434 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-10T10:54:16.434 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=1224 2024-09-10T10:54:16.434 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-10T10:54:16.434 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T10:54:16.435 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T10:54:16.682 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:54:17.206 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 670, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 670, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 852771, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 212300, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.054628376, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.021024234, 2024-09-10T10:54:17.207 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.014954407, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.013152114, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 669, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.025423920, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000038002 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:54:17.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 780, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 761, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 997984, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 298535, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.075195639, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.025918239, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.036804324, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.016140010, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 760, 2024-09-10T10:54:17.209 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.028427144, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000037404 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 860448, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 120768 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 645, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 792, 2024-09-10T10:54:17.210 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 216544, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 527128, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.074480513, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.026525651, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.018553821, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.021202042, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 790, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.043088999, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000054543 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.211 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 322336 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-09-10T10:54:17.212 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3101, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.078016830, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000025158 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-09-10T10:54:17.214 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 90, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 90, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 90, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-09-10T10:54:17.215 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-09-10T10:54:17.216 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-09-10T10:54:17.218 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:54:17.219 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 611, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-09-10T10:54:17.220 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4869208, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4866985, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.221 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 611 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 4, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 105, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 101, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 2, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 15, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 13, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 90, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 278, 2024-09-10T10:54:17.222 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 188, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 48788, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215328 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-09-10T10:54:17.223 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.224 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-09-10T10:54:17.227 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.228 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.228 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.228 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-09-10T10:54:17.228 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.228 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.229 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:17.231 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.232 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.233 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-09-10T10:54:17.234 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-09-10T10:54:17.235 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 1582, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:17.236 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:17.237 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:17.238 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 189423, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 51, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:17.239 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 162, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 75416, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 389, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-09-10T10:54:17.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 1869, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 1869, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1869, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.855985294, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000457991 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-09-10T10:54:17.241 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1869, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1870 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 1869, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 1869, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 1201, 2024-09-10T10:54:17.242 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 666, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-09-10T10:54:17.243 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-09-10T10:54:17.244 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 89, 2024-09-10T10:54:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 5, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 5, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-09-10T10:54:17.246 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 2091, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1864788, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 2091, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2091, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1864788, 2024-09-10T10:54:17.248 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:17.249 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 1869, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 674909, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1869, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 674909, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:17.250 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 1869, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1869, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1869, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1869, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:17.251 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.252 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5636507f8b40": { 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-09-10T10:54:17.253 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T10:54:17.254 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T10:54:18.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:17 smithi143 ceph-mon[31057]: pgmap v628: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T10:54:18.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:17 smithi144 ceph-mon[34230]: pgmap v628: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T10:54:18.137 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-10T10:54:18.137 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-10T10:54:18.137 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-10T10:54:18.137 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-10T10:54:18.137 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-10T10:54:18.137 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T10:54:18.138 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T10:54:18.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:17 smithi178 ceph-mon[34813]: pgmap v628: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-10T10:54:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:54:19.120 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T10:54:19.120 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:19.120 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 12789, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 13376, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3895519, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 2948463, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 14, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.666666155, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.234015455, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.536835163, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.007073859, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 13374, 2024-09-10T10:54:19.121 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.221125182, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000016533 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 25456, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.003844023, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000151 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 736 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 11194, 2024-09-10T10:54:19.122 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 7045, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 6626897, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1681069, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 20, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 7, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.623156262, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.270465455, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.465665297, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.004825884, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 7045, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.155853896, 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000022122 2024-09-10T10:54:19.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17373, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002536200, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000145 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 885520, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 123936 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1584, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1240, 2024-09-10T10:54:19.124 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 544365, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 919269, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 24, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 10, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.119299982, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.053433199, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.072982540, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.003940199, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1238, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.047650438, 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000038489 2024-09-10T10:54:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2516, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000441598, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000175 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 321808 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.126 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-10T10:54:19.127 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:19.127 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:19.127 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 470, 2024-09-10T10:54:19.127 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.181604127, 2024-09-10T10:54:19.127 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000386391 2024-09-10T10:54:19.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 183, 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.012015404, 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000065657 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 11821, 2024-09-10T10:54:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 1098, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1098, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 54.200945882, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.049363338 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 10723, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-10T10:54:19.129 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 29, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 9, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 20, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 13, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 226, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 1, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 1, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 4, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 14260, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 2375, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 10723, 2024-09-10T10:54:19.130 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 123, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 1, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 1, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 12, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 1084, 2024-09-10T10:54:19.131 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 21251863, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 545, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 337, 2024-09-10T10:54:19.132 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 553, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 84, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 88, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 512 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 4, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 1205, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 658, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 723, 2024-09-10T10:54:19.133 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 4, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 89, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 89, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-10T10:54:19.134 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 6, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 452, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 1, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 611, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-10T10:54:19.135 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 611, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4869208, 2024-09-10T10:54:19.136 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 184, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.282458642, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001535101 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 34, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 235, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 201, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 33, 2024-09-10T10:54:19.137 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 705, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 672, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 29, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 521, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 492, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 1, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 177, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 176, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 57652, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223520 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 13167, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 9, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 11821, 2024-09-10T10:54:19.138 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 12, 2024-09-10T10:54:19.139 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 556, 2024-09-10T10:54:19.139 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 83, 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.630485015, 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.007596205 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000276325, 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000276325 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:19.140 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 841, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 29.351121145, 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.034900262 2024-09-10T10:54:19.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.142 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-10T10:54:19.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.144 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.144 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-10T10:54:19.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001432702, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001432702 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 84, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 24.113215145, 2024-09-10T10:54:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.287062085 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.148 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 88, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.104415550, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001186540 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.151 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-10T10:54:19.152 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 415, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 415, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 543, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:19.153 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:19.155 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 303477, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 112, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 162, 2024-09-10T10:54:19.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 139173, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 489, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-10T10:54:19.157 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 478, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 703164, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 29, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 478, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 478, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.242977434, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002600371 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 478, 2024-09-10T10:54:19.158 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 571 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 478, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 4, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 474, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 4, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 188, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 13, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-10T10:54:19.159 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 190, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-10T10:54:19.160 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-10T10:54:19.161 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 2, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-09-10T10:54:19.162 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 112, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 34 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 456 2024-09-10T10:54:19.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 5, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 178, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 89, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 25563, 2024-09-10T10:54:19.164 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 8750293, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 25563, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 25563, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 8750293, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:19.165 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 478, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 712865, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 478, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 712865, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.166 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 478, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 478, 2024-09-10T10:54:19.167 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 478, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 478, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 89, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 9701, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:19.168 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 89, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 4, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 9701, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x5623b2eee960": { 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:19.169 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 611, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 674904, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 611, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 184, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 674904, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:19.170 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.171 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:19.171 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T10:54:20.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:19 smithi143 ceph-mon[31057]: pgmap v629: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:20.160 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:19 smithi178 ceph-mon[34813]: pgmap v629: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:20.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:19 smithi144 ceph-mon[34230]: pgmap v629: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:20.189 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-09-10T10:54:20.189 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-09-10T10:54:20.189 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=4 2024-09-10T10:54:20.189 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=1205 2024-09-10T10:54:20.190 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-10T10:54:20.190 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T10:54:20.190 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T10:54:20.482 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 56491, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 66328, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 19985926, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 217093093, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 10, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 4.607300610, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 2.107637620, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 1.346793107, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 1.012681039, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 66328, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 15.031651423, 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000226626 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 42991, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.007623007, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000177 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 69260, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 73859, 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 26624142, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 256071946, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 19, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 7, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 5.007605672, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 2.103166328, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 1.194646712, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 1.390685259, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 73857, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 30.406349002, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000411692 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 19458, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.003793432, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000194 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 324144 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1144705, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 457448, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 397678648, 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 371479608, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 29, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 11, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 27.615288492, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 11.039571539, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 22.496438930, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 1.032476916, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 457446, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 28.213298984, 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000061675 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1466239, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.159689469, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000108 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 886160, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 134960 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 39108, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 102.979463738, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002633207 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 32556, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 7.213110867, 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000221560 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 237760, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 236339, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 236339, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 4431.811993314, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.018751928 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 1421, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 1435, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 72, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 14, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 1, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 1, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 12, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 12, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 836, 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 424040, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 261985, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 1376, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 16, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 8, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 19, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 539, 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 26, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 522, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 71041, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 1084, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 21251863, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 71041, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 505, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 505, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 70536, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 62, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 63620, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 36400, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 858413, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 17462, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 294, 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 101385 2024-09-10T10:54:21.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 1949, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 42, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 28, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 31, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 2566, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 17552, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 17552, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:21.321 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 6, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 49557, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 21, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 74, 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 416091, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 306125, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 294961, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 121130, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 11164, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 476, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 348, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 336, 2024-09-10T10:54:21.323 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 140, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 10, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 12, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 389226589, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 546825710, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5665, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 102.439587124, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.018082892 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-10T10:54:21.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 19, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 17859, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 17840, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 17, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 723, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 706, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 14, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 47553, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 47539, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 17826, 2024-09-10T10:54:21.325 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 17826, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 146404, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 223520 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 241954, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 2486, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 237760, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 30, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 5336, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17461, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 516.371112437, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.029572825 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.326 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 301, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.437382006, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001453096 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 166895, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2792.397403554, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.016731462 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.328 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 96, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.164923853, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.022551290 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-10T10:54:21.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 19259, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 443.672963834, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.023037175 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-10T10:54:21.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 986, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.855937535, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000868090 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17463, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 647.321266405, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.037068159 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.332 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 99, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 9.360984739, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.094555401 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.334 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.335 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 13779, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 19.230018951, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001395603 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:21.336 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 47669, 2024-09-10T10:54:21.337 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 47669, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 541, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:21.338 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:21.339 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:21.340 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 306765, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 120, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 162, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:21.341 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 80696, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 516, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 142571, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 577289039, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-09-10T10:54:21.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 142571, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 142571, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2090.772842616, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.014664783 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 142571, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 185971 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.343 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 142571, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 71670, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 70901, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 752, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 26573, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 629, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 6294, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 353, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-10T10:54:21.344 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 72, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 35821, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 71041, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 39917, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:21.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-10T10:54:21.346 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 2, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 3475, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 1044 2024-09-10T10:54:21.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 49595 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-10T10:54:21.348 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 17, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 5, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 35589, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 17552, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1270452, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 329931107, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:21.349 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1270452, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1270452, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 329931107, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:21.350 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 142571, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 579201999, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 142571, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 579201999, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 142571, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 142571, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 142571, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 142571, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.352 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 17552, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1912960, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 17552, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:21.353 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 629, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1912960, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55db21b96960": { 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 416091, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 542631406, 2024-09-10T10:54:21.354 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 416091, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5665, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 542631406, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:21.355 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T10:54:22.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:21 smithi143 ceph-mon[31057]: pgmap v630: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T10:54:22.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:21 smithi178 ceph-mon[34813]: pgmap v630: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T10:54:22.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:21 smithi144 ceph-mon[34230]: pgmap v630: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-10T10:54:22.376 INFO:tasks.check_counter:Daemon mds.f mds.exported=19 2024-09-10T10:54:22.377 INFO:tasks.check_counter:Daemon mds.f mds.imported=26 2024-09-10T10:54:22.377 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=1949 2024-09-10T10:54:22.377 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=42 2024-09-10T10:54:22.377 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=72 2024-09-10T10:54:22.377 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T10:54:22.377 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T10:54:22.642 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:54:23.170 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T10:54:23.170 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 172254, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 148595, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 56095852, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 34609194, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 33, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 7.276511484, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 2.690526418, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 5.663289890, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.169823606, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 148591, 2024-09-10T10:54:23.171 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.207498841, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000014856 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 312282, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.042613148, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000136 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 58880, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 30304 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 3047, 2024-09-10T10:54:23.172 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 3668, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 964269, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 4963144, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 11, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.235082278, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.100230205, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.088548532, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.045115072, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3669, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.721674451, 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000196695 2024-09-10T10:54:23.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2468, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000564617, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000228 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 339408 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 4400, 2024-09-10T10:54:23.174 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 5237, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 2152924, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 3119549, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 112, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 7, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.323340674, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.104432711, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.180238878, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.049548723, 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:23.175 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5236, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.305558591, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000058357 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5121, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000948483, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000185 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 936672, 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 143664 2024-09-10T10:54:23.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3792, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 3.175327461, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000837375 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.177 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1664, 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.117931799, 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000070872 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 126713, 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 9788, 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-10T10:54:23.178 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 9788, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 623.767451223, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.063727773 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 116925, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 12, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 40, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 9, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 13, 2024-09-10T10:54:23.179 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 18, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 26, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 879, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 2, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 2, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 136189, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 14561, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 116923, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 151, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-10T10:54:23.180 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 17, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 520, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 17, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 523, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 2615, 2024-09-10T10:54:23.181 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 2615, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 141, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 12, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 129, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 2474, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:23.182 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 2964, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 2012, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 5, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 26249, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 997, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 143, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 3803 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 149, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 1070, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 582, 2024-09-10T10:54:23.183 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 648, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 227, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 3, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 833, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 830, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-10T10:54:23.184 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 6, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 2393, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 12, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 17, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-10T10:54:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 8306, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 8306, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 11, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 11, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-10T10:54:23.186 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 15926154, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 827, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.523898428, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.003051872 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 37, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 1292, 2024-09-10T10:54:23.187 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 1255, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 32, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 666, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 634, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 40, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 2703, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 2663, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 2, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 1277, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 1275, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 68312, 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 231712 2024-09-10T10:54:23.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 129021, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 306, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 126713, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 23, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 1527, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 822, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 3.657177640, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004449121 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 613, 2024-09-10T10:54:23.189 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.138710334, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000226281 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6834, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 562.408962412, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.082295721 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.191 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 21, 2024-09-10T10:54:23.193 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 24.636973610, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 1.173189219 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-10T10:54:23.194 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 152, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.088105742, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000579643 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 94, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.373370155, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.003972022 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 441, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 31.258555182, 2024-09-10T10:54:23.195 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.070881077 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-10T10:54:23.196 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 12, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.108109781, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.009009148 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001467088, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000733544 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:23.198 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 9, 2024-09-10T10:54:23.199 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.022741364, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002526818 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002615606, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000871868 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 785, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.070662309, 2024-09-10T10:54:23.200 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001363901 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-10T10:54:23.201 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 4, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 3, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 1654, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 1654, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 544, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:23.202 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:23.203 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:23.204 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 310695, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 107, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:23.205 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 162, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 159383, 2024-09-10T10:54:23.206 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 1247, 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-10T10:54:23.207 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-10T10:54:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-10T10:54:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 6025, 2024-09-10T10:54:23.208 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 11949935, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 32, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 6025, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6025, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 27.949486499, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004638918 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-10T10:54:23.209 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 6025, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 6306 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 6025, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 2689, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 3336, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 11, 2024-09-10T10:54:23.210 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 62, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 877, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 71, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 2162, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2627, 2024-09-10T10:54:23.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 22, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:23.212 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 5, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-10T10:54:23.213 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 8, 2024-09-10T10:54:23.214 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 349, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 120 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:23.215 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 2398 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 6, 2024-09-10T10:54:23.216 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 1648, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 830, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 179697, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 43013272, 2024-09-10T10:54:23.217 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 179697, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 179697, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 43013272, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.218 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:23.219 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 6025, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 12040309, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6025, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 12040309, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.220 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 6025, 2024-09-10T10:54:23.221 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 6025, 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6025, 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 6025, 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:23.222 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 830, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 90374, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 830, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 50, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 90374, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.223 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55d3b4c72960": { 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 8306, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 11731850, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:23.224 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 8306, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 827, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 11731850, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:23.225 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T10:54:23.925 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:23 smithi178 ceph-mon[34813]: pgmap v631: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-09-10T10:54:24.031 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2024-09-10T10:54:24.031 INFO:tasks.check_counter:Daemon mds.i mds.imported=17 2024-09-10T10:54:24.031 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=149 2024-09-10T10:54:24.031 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1070 2024-09-10T10:54:24.031 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-10T10:54:24.031 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T10:54:24.032 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T10:54:24.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:23 smithi143 ceph-mon[31057]: pgmap v631: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-09-10T10:54:24.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:23 smithi144 ceph-mon[34230]: pgmap v631: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-09-10T10:54:24.280 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1874, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1876, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1480847, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 595939, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.135149174, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.043164136, 2024-09-10T10:54:24.791 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.034680682, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.046891271, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1874, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.057778791, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000030831 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:54:24.792 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 226, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 207, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 861820, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 123386, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.030111747, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.012129180, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.025732551, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000080604, 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:24.793 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 206, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.011949794, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000058008 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 861264, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 122688 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 6, 2024-09-10T10:54:24.794 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 153, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3468, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 325070, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.010617518, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.005894200, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.003431221, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000025986, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 152, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.011757290, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000077350 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:54:24.795 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 324752 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:24.796 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3180, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.074413747, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000023400 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-10T10:54:24.797 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-10T10:54:24.798 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 121, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 121, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 121, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-10T10:54:24.799 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-09-10T10:54:24.800 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:54:24.801 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:54:24.802 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-10T10:54:24.803 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 873, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-10T10:54:24.804 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 5144412, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 5142761, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 873 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 4, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 136, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 132, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 2, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 23, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 21, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 121, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 373, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 252, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-10T10:54:24.806 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 52728, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 215328 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.807 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.808 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.809 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.811 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.812 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.813 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.814 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.817 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.818 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 1, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1570, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-10T10:54:24.819 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:54:24.820 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-10T10:54:24.821 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 181360, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 48, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:54:24.822 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8664, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 162, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 97232, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 512, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-10T10:54:24.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 1877, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 1877, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1877, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.972982244, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000518370 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.824 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1877, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1878 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 1877, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 1877, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 1201, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 674, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-10T10:54:24.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-10T10:54:24.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-10T10:54:24.827 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 89, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 2, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 4, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 4, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-10T10:54:24.828 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "osd-slow-ops": { 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-10T10:54:24.829 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 2102, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 2142478, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 2102, 2024-09-10T10:54:24.830 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2102, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2142478, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:24.831 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1877, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 950113, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1877, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 950113, 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:24.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:24.833 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1877, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1877, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1877, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1877, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:24.834 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55983277eb40": { 2024-09-10T10:54:24.835 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-10T10:54:24.836 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-10T10:54:24.837 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.837 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T10:54:24.837 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T10:54:25.608 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T10:54:25.644 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T10:54:25.644 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-10T10:54:25.644 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-10T10:54:25.644 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-10T10:54:25.645 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T10:54:25.660 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T10:54:25.660 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:54:25.660 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:25.687 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:54:25.687 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:25.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:54:26.045 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:26.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:25 smithi143 ceph-mon[31057]: pgmap v632: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:26.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:25 smithi178 ceph-mon[34813]: pgmap v632: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:26.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:25 smithi144 ceph-mon[34230]: pgmap v632: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T10:54:26.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 -- 172.21.15.143:0/1609863042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 msgr2=0x7f6ba410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 --2- 172.21.15.143:0/1609863042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba410da40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009ca0 tx=0x7f6b9c02a950 comp rx=0 tx=0).stop 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 -- 172.21.15.143:0/1609863042 shutdown_connections 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 --2- 172.21.15.143:0/1609863042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba410da40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 --2- 172.21.15.143:0/1609863042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4102570 0x7f6ba41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 --2- 172.21.15.143:0/1609863042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4100e20 0x7f6ba4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 -- 172.21.15.143:0/1609863042 >> 172.21.15.143:0/1609863042 conn(0x7f6ba40fc7d0 msgr2=0x7f6ba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 -- 172.21.15.143:0/1609863042 shutdown_connections 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 -- 172.21.15.143:0/1609863042 wait complete. 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 Processor -- start 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.530+0000 7f6bab505640 1 -- start start 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4100e20 0x7f6ba4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4102570 0x7f6ba4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:26.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4110ed0 con 0x7f6ba4102ef0 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4110d50 con 0x7f6ba4102570 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4111050 con 0x7f6ba4100e20 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54884/0 (socket says 172.21.15.143:54884) 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 -- 172.21.15.143:0/539123746 learned_addr learned my addr 172.21.15.143:0/539123746 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba8aa0640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4102570 0x7f6ba4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba92a1640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4100e20 0x7f6ba4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 -- 172.21.15.143:0/539123746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4100e20 msgr2=0x7f6ba4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4100e20 0x7f6ba4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 -- 172.21.15.143:0/539123746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4102570 msgr2=0x7f6ba4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4102570 0x7f6ba4197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 -- 172.21.15.143:0/539123746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba419c3e0 con 0x7f6ba4102ef0 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba92a1640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4100e20 0x7f6ba4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba8aa0640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4102570 0x7f6ba4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:26.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6ba9aa2640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba419bd00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006010 tx=0x7f6b9c02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c034040 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b9c003e00 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba419c670 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c045480 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.531+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41a3fb0 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.532+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba4101280 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.534+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6b9c037020 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.534+0000 7f6b967fc640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b900a2c00 0x7f6b900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.534+0000 7f6ba92a1640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b900a2c00 0x7f6b900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.534+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f6b9c0f05d0 con 0x7f6ba4102ef0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.535+0000 7f6ba92a1640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b900a2c00 0x7f6b900a50c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006fd0 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:26.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.538+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b9c0383b0 con 0x7f6ba4102ef0 2024-09-10T10:54:26.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.716+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6ba4109090 con 0x7f6ba4102ef0 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.716+0000 7f6b967fc640 1 -- 172.21.15.143:0/539123746 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+601 (secure 0 0 0) 0x7f6b9c0bd1d0 con 0x7f6ba4102ef0 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3629809110 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/212886119 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3153560299 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:26.718 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:26.719 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/1093984961 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:26.719 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/2707450669 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:26.719 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3503275671 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:26.719 INFO:teuthology.orchestra.run.smithi143.stderr:listed 10 entries 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.718+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b900a2c00 msgr2=0x7f6b900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.718+0000 7f6bab505640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b900a2c00 0x7f6b900a50c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006fd0 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.718+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 msgr2=0x7f6ba419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.718+0000 7f6bab505640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba419bd00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006010 tx=0x7f6b9c02f710 comp rx=0 tx=0).stop 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 shutdown_connections 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f6b900a2c00 0x7f6b900a50c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ba4102ef0 0x7f6ba419bd00 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba4102570 0x7f6ba4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 --2- 172.21.15.143:0/539123746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4100e20 0x7f6ba4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:26.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 >> 172.21.15.143:0/539123746 conn(0x7f6ba40fc7d0 msgr2=0x7f6ba40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 shutdown_connections 2024-09-10T10:54:26.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:26.719+0000 7f6bab505640 1 -- 172.21.15.143:0/539123746 wait complete. 2024-09-10T10:54:27.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:26 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/539123746' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:54:27.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:26 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/539123746' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:54:27.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:26 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/539123746' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:54:27.552 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-09-10T10:54:27.552 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:27.571 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:54:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:28.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:27 smithi143 ceph-mon[31057]: pgmap v633: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:54:28.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:27 smithi178 ceph-mon[34813]: pgmap v633: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:54:28.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:27 smithi144 ceph-mon[34230]: pgmap v633: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 -- 172.21.15.143:0/1300015262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec102d90 msgr2=0x7ff2ec103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 --2- 172.21.15.143:0/1300015262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec102d90 0x7ff2ec103170 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009930 tx=0x7ff2dc02a420 comp rx=0 tx=0).stop 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 -- 172.21.15.143:0/1300015262 shutdown_connections 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 --2- 172.21.15.143:0/1300015262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec108290 0x7ff2ec10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 --2- 172.21.15.143:0/1300015262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec103740 0x7ff2ec107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 --2- 172.21.15.143:0/1300015262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec102d90 0x7ff2ec103170 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 -- 172.21.15.143:0/1300015262 >> 172.21.15.143:0/1300015262 conn(0x7ff2ec0fc7d0 msgr2=0x7ff2ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 -- 172.21.15.143:0/1300015262 shutdown_connections 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.359+0000 7ff2f0977640 1 -- 172.21.15.143:0/1300015262 wait complete. 2024-09-10T10:54:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 Processor -- start 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 -- start start 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec102d90 0x7ff2ec1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec103740 0x7ff2ec1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 0x7ff2ec19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec111f40 con 0x7ff2ec108290 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec111dc0 con 0x7ff2ec102d90 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.360+0000 7ff2f0977640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1120c0 con 0x7ff2ec103740 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 0x7ff2ec19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 0x7ff2ec19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54908/0 (socket says 172.21.15.143:54908) 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 -- 172.21.15.143:0/791001140 learned_addr learned my addr 172.21.15.143:0/791001140 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2e1d9b640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec103740 0x7ff2ec1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ea59c640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec102d90 0x7ff2ec1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 -- 172.21.15.143:0/791001140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec103740 msgr2=0x7ff2ec1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec103740 0x7ff2ec1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 -- 172.21.15.143:0/791001140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec102d90 msgr2=0x7ff2ec1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec102d90 0x7ff2ec1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 -- 172.21.15.143:0/791001140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec19d450 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ea59c640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec102d90 0x7ff2ec1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2ead9d640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 0x7ff2ec19cd70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400eaa0 tx=0x7ff2e400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4004d20 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e4005020 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec19d740 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40116b0 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.361+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1a5020 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.362+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec06a7f0 con 0x7ff2ec108290 2024-09-10T10:54:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.363+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff2e40051d0 con 0x7ff2ec108290 2024-09-10T10:54:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.364+0000 7ff2e3fff640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2d80a2b80 0x7ff2d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.364+0000 7ff2ea59c640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2d80a2b80 0x7ff2d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.364+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff2e40cf510 con 0x7ff2ec108290 2024-09-10T10:54:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.365+0000 7ff2ea59c640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2d80a2b80 0x7ff2d80a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc000c00 tx=0x7ff2dc009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.368+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2e4016030 con 0x7ff2ec108290 2024-09-10T10:54:28.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.545+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2ec10a2a0 con 0x7ff2ec108290 2024-09-10T10:54:28.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.546+0000 7ff2e3fff640 1 -- 172.21.15.143:0/791001140 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+601 (secure 0 0 0) 0x7ff2e409c110 con 0x7ff2ec108290 2024-09-10T10:54:28.547 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3629809110 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:28.547 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/212886119 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3153560299 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/1050213403 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6800/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6801/3171173333 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/1093984961 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/2707450669 2024-09-11T10:35:59.338289+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/3503275671 2024-09-11T10:35:32.440973+0000 2024-09-10T10:54:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:listed 10 entries 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2d80a2b80 msgr2=0x7ff2d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2d80a2b80 0x7ff2d80a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc000c00 tx=0x7ff2dc009610 comp rx=0 tx=0).stop 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 msgr2=0x7ff2ec19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 0x7ff2ec19cd70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400eaa0 tx=0x7ff2e400eea0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 shutdown_connections 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7ff2d80a2b80 0x7ff2d80a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2ec108290 0x7ff2ec19cd70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2ec103740 0x7ff2ec1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 --2- 172.21.15.143:0/791001140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ec102d90 0x7ff2ec1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 >> 172.21.15.143:0/791001140 conn(0x7ff2ec0fc7d0 msgr2=0x7ff2ec106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 shutdown_connections 2024-09-10T10:54:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:28.548+0000 7ff2f0977640 1 -- 172.21.15.143:0/791001140 wait complete. 2024-09-10T10:54:28.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:28 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/791001140' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:54:29.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:28 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/791001140' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:54:29.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:28 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/791001140' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:54:29.344 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-09-10T10:54:29.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:29.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:29.372 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:54:29.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[85758]: fuse finished with error 0 and tester_r 0 2024-09-10T10:54:30.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:29 smithi143 ceph-mon[31057]: pgmap v634: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:30.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:29 smithi178 ceph-mon[34813]: pgmap v634: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:30.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:29 smithi144 ceph-mon[34230]: pgmap v634: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T10:54:32.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:31 smithi143 ceph-mon[31057]: pgmap v635: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T10:54:32.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:31 smithi178 ceph-mon[34813]: pgmap v635: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T10:54:32.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:31 smithi144 ceph-mon[34230]: pgmap v635: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T10:54:34.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:33 smithi143 ceph-mon[31057]: pgmap v636: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-10T10:54:34.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:33 smithi178 ceph-mon[34813]: pgmap v636: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-10T10:54:34.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:33 smithi144 ceph-mon[34230]: pgmap v636: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-10T10:54:35.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T10:54:35.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:35.475 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:54:35.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:35.496 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-09-10T10:54:35.555 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:54:35.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:35.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:54:35.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:35.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-09-10T10:54:35.735 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T10:54:35.746 INFO:tasks.cephadm:Teardown begin 2024-09-10T10:54:35.746 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:54:35.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:54:36.000 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:36.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:35 smithi143 ceph-mon[31057]: pgmap v637: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:36.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:35 smithi178 ceph-mon[34813]: pgmap v637: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:36.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:35 smithi144 ceph-mon[34230]: pgmap v637: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:36.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 -- 172.21.15.143:0/3667146298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c103660 msgr2=0x7fc28c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 --2- 172.21.15.143:0/3667146298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c103660 0x7fc28c10bb50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc280009900 tx=0x7fc28002a480 comp rx=0 tx=0).stop 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 -- 172.21.15.143:0/3667146298 shutdown_connections 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 --2- 172.21.15.143:0/3667146298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc28c10c170 0x7fc28c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 --2- 172.21.15.143:0/3667146298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c103660 0x7fc28c10bb50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 --2- 172.21.15.143:0/3667146298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c102d40 0x7fc28c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 -- 172.21.15.143:0/3667146298 >> 172.21.15.143:0/3667146298 conn(0x7fc28c0fc7a0 msgr2=0x7fc28c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 -- 172.21.15.143:0/3667146298 shutdown_connections 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 -- 172.21.15.143:0/3667146298 wait complete. 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 Processor -- start 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.644+0000 7fc29280b640 1 -- start start 2024-09-10T10:54:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc28c102d40 0x7fc28c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c103660 0x7fc28c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 0x7fc28c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111e20 con 0x7fc28c10c170 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111ca0 con 0x7fc28c103660 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111fa0 con 0x7fc28c102d40 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 0x7fc28c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 0x7fc28c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37286/0 (socket says 172.21.15.143:37286) 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 -- 172.21.15.143:0/3752461559 learned_addr learned my addr 172.21.15.143:0/3752461559 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc28b7fe640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c103660 0x7fc28c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc28bfff640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc28c102d40 0x7fc28c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 -- 172.21.15.143:0/3752461559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc28c102d40 msgr2=0x7fc28c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc28c102d40 0x7fc28c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 -- 172.21.15.143:0/3752461559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c103660 msgr2=0x7fc28c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c103660 0x7fc28c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 -- 172.21.15.143:0/3752461559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c19d2f0 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc290da8640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 0x7fc28c19cc10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc28400cce0 tx=0x7fc28400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400f450 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c19d5e0 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc28400faa0 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284018b70 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.645+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a4ec0 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.646+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c103120 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.648+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc284018d10 con 0x7fc28c10c170 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.648+0000 7fc2897fa640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2740a2b80 0x7fc2740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.648+0000 7fc28bfff640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2740a2b80 0x7fc2740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.649+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc2840cf8d0 con 0x7fc28c10c170 2024-09-10T10:54:36.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.649+0000 7fc28bfff640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2740a2b80 0x7fc2740a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005fd0 tx=0x7fc27c009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.652+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc284014030 con 0x7fc28c10c170 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.854+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc28c1996a0 con 0x7fc28c10c170 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.855+0000 7fc2897fa640 1 -- 172.21.15.143:0/3752461559 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12014 (secure 0 0 0) 0x7fc28c1996a0 con 0x7fc28c10c170 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":42,"btime":"2024-09-10T10:53:46:702117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:53:46.416985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 42 2024-09-10T10:54:36.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2740a2b80 msgr2=0x7fc2740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2740a2b80 0x7fc2740a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005fd0 tx=0x7fc27c009470 comp rx=0 tx=0).stop 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 msgr2=0x7fc28c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 0x7fc28c19cc10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc28400cce0 tx=0x7fc28400a620 comp rx=0 tx=0).stop 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 shutdown_connections 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7fc2740a2b80 0x7fc2740a5040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc28c10c170 0x7fc28c19cc10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c103660 0x7fc28c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 --2- 172.21.15.143:0/3752461559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc28c102d40 0x7fc28c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 >> 172.21.15.143:0/3752461559 conn(0x7fc28c0fc7a0 msgr2=0x7fc28c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 shutdown_connections 2024-09-10T10:54:36.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:36.857+0000 7fc29280b640 1 -- 172.21.15.143:0/3752461559 wait complete. 2024-09-10T10:54:37.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:36 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3752461559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:54:37.177 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:36 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3752461559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:54:37.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:36 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3752461559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:54:37.532 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:54:37.789 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:38.062 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:37 smithi143 ceph-mon[31057]: pgmap v638: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T10:54:38.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:37 smithi178 ceph-mon[34813]: pgmap v638: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T10:54:38.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:37 smithi144 ceph-mon[34230]: pgmap v638: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T10:54:38.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 -- 172.21.15.143:0/131804049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538101df0 msgr2=0x7f15381021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 --2- 172.21.15.143:0/131804049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538101df0 0x7f15381021d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f152c009900 tx=0x7f152c02a410 comp rx=0 tx=0).stop 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 -- 172.21.15.143:0/131804049 shutdown_connections 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 --2- 172.21.15.143:0/131804049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1538106f20 0x7f153810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 --2- 172.21.15.143:0/131804049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15381027a0 0x7f15381067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 --2- 172.21.15.143:0/131804049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538101df0 0x7f15381021d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 -- 172.21.15.143:0/131804049 >> 172.21.15.143:0/131804049 conn(0x7f15380fd8f0 msgr2=0x7f15380ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 -- 172.21.15.143:0/131804049 shutdown_connections 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.256+0000 7f153ef86640 1 -- 172.21.15.143:0/131804049 wait complete. 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 Processor -- start 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 -- start start 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1538101df0 0x7f15380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15381027a0 0x7f1538074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 0x7f1538074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1538111e30 con 0x7f1538106f20 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1538111cb0 con 0x7f15381027a0 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153ef86640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1538111fb0 con 0x7f1538101df0 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 0x7f1538074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 0x7f1538074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37302/0 (socket says 172.21.15.143:37302) 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 -- 172.21.15.143:0/2910273557 learned_addr learned my addr 172.21.15.143:0/2910273557 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153cd22640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1538101df0 0x7f15380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 -- 172.21.15.143:0/2910273557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1538101df0 msgr2=0x7f15380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f1537fff640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15381027a0 0x7f1538074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1538101df0 0x7f15380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 -- 172.21.15.143:0/2910273557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15381027a0 msgr2=0x7f1538074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15381027a0 0x7f1538074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153d523640 1 -- 172.21.15.143:0/2910273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15381a6950 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f153cd22640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1538101df0 0x7f15380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.257+0000 7f1537fff640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15381027a0 0x7f1538074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.258+0000 7f153d523640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 0x7f1538074f70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f153000cad0 tx=0x7f153000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.258+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000ae20 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.258+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530005020 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.258+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15381a6c40 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.258+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000f6b0 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.258+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381a7130 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.259+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15381021d0 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.260+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f15300051d0 con 0x7f1538106f20 2024-09-10T10:54:38.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.261+0000 7f1535ffb640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15280a2b80 0x7f15280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.261+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f15300cf440 con 0x7f1538106f20 2024-09-10T10:54:38.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.261+0000 7f153cd22640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15280a2b80 0x7f15280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.261+0000 7f153cd22640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15280a2b80 0x7f15280a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f152c031440 tx=0x7f152c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.265+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1530014030 con 0x7f1538106f20 2024-09-10T10:54:38.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.474+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f153810a100 con 0x7f1538106f20 2024-09-10T10:54:38.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.475+0000 7f1535ffb640 1 -- 172.21.15.143:0/2910273557 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12014 (secure 0 0 0) 0x7f153009c040 con 0x7f1538106f20 2024-09-10T10:54:38.476 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:54:38.476 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":42,"btime":"2024-09-10T10:53:46:702117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3176590385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":3176590385},{"type":"v1","addr":"172.21.15.143:6835","nonce":3176590385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2245538131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2245538131},{"type":"v1","addr":"172.21.15.143:6837","nonce":2245538131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:42:53.956292+0000","modified":"2024-09-10T10:53:46.416985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24485,"mds_2":24481,"mds_3":24475,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6839/2518540467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2518540467},{"type":"v1","addr":"172.21.15.143:6839","nonce":2518540467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6841/1083221625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6840","nonce":1083221625},{"type":"v1","addr":"172.21.15.143:6841","nonce":1083221625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3121359191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3121359191},{"type":"v1","addr":"172.21.15.144:6833","nonce":3121359191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1016684638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016684638},{"type":"v1","addr":"172.21.15.178:6833","nonce":1016684638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3190468681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3190468681},{"type":"v1","addr":"172.21.15.144:6835","nonce":3190468681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.144:6837/3174266316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3174266316},{"type":"v1","addr":"172.21.15.144:6837","nonce":3174266316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6835/261851614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":261851614},{"type":"v1","addr":"172.21.15.178:6835","nonce":261851614}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.178:6837/2330516637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2330516637},{"type":"v1","addr":"172.21.15.178:6837","nonce":2330516637}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.144:6839/1994626034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1994626034},{"type":"v1","addr":"172.21.15.144:6839","nonce":1994626034}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.178:6839/774166191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":774166191},{"type":"v1","addr":"172.21.15.178:6839","nonce":774166191}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24485,24481,24475,24458]},"id":1}]} 2024-09-10T10:54:38.477 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 42 2024-09-10T10:54:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15280a2b80 msgr2=0x7f15280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15280a2b80 0x7f15280a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f152c031440 tx=0x7f152c0095f0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 msgr2=0x7f1538074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 0x7f1538074f70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f153000cad0 tx=0x7f153000cf90 comp rx=0 tx=0).stop 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 shutdown_connections 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f15280a2b80 0x7f15280a5040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1538106f20 0x7f1538074f70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15381027a0 0x7f1538074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 --2- 172.21.15.143:0/2910273557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1538101df0 0x7f15380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 >> 172.21.15.143:0/2910273557 conn(0x7f15380fd8f0 msgr2=0x7f15380fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 shutdown_connections 2024-09-10T10:54:38.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:38.477+0000 7f153ef86640 1 -- 172.21.15.143:0/2910273557 wait complete. 2024-09-10T10:54:39.129 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:38 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2910273557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:54:39.157 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:54:39.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:38 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2910273557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:54:39.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:38 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2910273557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:54:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 -- 172.21.15.143:0/695011143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f081036b0 msgr2=0x7f4f0810bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 --2- 172.21.15.143:0/695011143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f081036b0 0x7f4f0810bba0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009900 tx=0x7f4efc02a480 comp rx=0 tx=0).stop 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 -- 172.21.15.143:0/695011143 shutdown_connections 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 --2- 172.21.15.143:0/695011143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f0810c170 0x7f4f0810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 --2- 172.21.15.143:0/695011143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f081036b0 0x7f4f0810bba0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 --2- 172.21.15.143:0/695011143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f08102d90 0x7f4f08103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 -- 172.21.15.143:0/695011143 >> 172.21.15.143:0/695011143 conn(0x7f4f080fc790 msgr2=0x7f4f080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 -- 172.21.15.143:0/695011143 shutdown_connections 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.922+0000 7f4f0d721640 1 -- 172.21.15.143:0/695011143 wait complete. 2024-09-10T10:54:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.923+0000 7f4f0d721640 1 Processor -- start 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 -- start start 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 0x7f4f08198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f081036b0 0x7f4f081988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f0810c170 0x7f4f0819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08111e50 con 0x7f4f08102d90 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08111cd0 con 0x7f4f0810c170 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f0d721640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08111fd0 con 0x7f4f081036b0 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 0x7f4f08198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 0x7f4f08198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37330/0 (socket says 172.21.15.143:37330) 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 -- 172.21.15.143:0/2366572901 learned_addr learned my addr 172.21.15.143:0/2366572901 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f067fc640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f081036b0 0x7f4f081988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f077fe640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f0810c170 0x7f4f0819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 -- 172.21.15.143:0/2366572901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f081036b0 msgr2=0x7f4f081988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f081036b0 0x7f4f081988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 -- 172.21.15.143:0/2366572901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f0810c170 msgr2=0x7f4f0819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f0810c170 0x7f4f0819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 -- 172.21.15.143:0/2366572901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0819d320 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.924+0000 7f4f06ffd640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 0x7f4f08198370 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.925+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8017070 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.925+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef8009c30 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.925+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0819d610 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.925+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800f620 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.925+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a4ef0 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.926+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08103170 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.927+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4ef80109a0 con 0x7f4f08102d90 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.927+0000 7f4eeffff640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4ef00a2b80 0x7f4ef00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.928+0000 7f4f067fc640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4ef00a2b80 0x7f4ef00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.928+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f4ef80cefd0 con 0x7f4f08102d90 2024-09-10T10:54:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.928+0000 7f4f067fc640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4ef00a2b80 0x7f4ef00a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005ab0 tx=0x7f4efc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:39.932+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ef8009de0 con 0x7f4f08102d90 2024-09-10T10:54:40.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.097+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f081071b0 con 0x7f4f08102d90 2024-09-10T10:54:40.099 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:39 smithi143 ceph-mon[31057]: pgmap v639: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:40.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.099+0000 7f4eeffff640 1 -- 172.21.15.143:0/2366572901 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24835 (secure 0 0 0) 0x7f4ef809bbd0 con 0x7f4f08102d90 2024-09-10T10:54:40.101 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-09-10T10:54:40.101 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":89,"fsid":"34212b78-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:35:07.959463+0000","modified":"2024-09-10T10:44:00.496675+0000","last_up_change":"2024-09-10T10:42:01.508463+0000","last_in_change":"2024-09-10T10:41:37.897265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:38:11.637482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:42:49.489312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:42:52.085668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:42:58.507118+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b681e90c-4078-4a6f-8d49-607d1b7505b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6803","nonce":3953963737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6805","nonce":3953963737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6809","nonce":3953963737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":3953963737},{"type":"v1","addr":"172.21.15.143:6807","nonce":3953963737}]},"public_addr":"172.21.15.143:6803/3953963737","cluster_addr":"172.21.15.143:6805/3953963737","heartbeat_back_addr":"172.21.15.143:6809/3953963737","heartbeat_front_addr":"172.21.15.143:6807/3953963737","state":["exists","up"]},{"osd":1,"uuid":"d1c81ecf-0a81-4ef2-8abd-cc38f88f5482","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6801","nonce":3635948930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6803","nonce":3635948930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6807","nonce":3635948930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3635948930},{"type":"v1","addr":"172.21.15.144:6805","nonce":3635948930}]},"public_addr":"172.21.15.144:6801/3635948930","cluster_addr":"172.21.15.144:6803/3635948930","heartbeat_back_addr":"172.21.15.144:6807/3635948930","heartbeat_front_addr":"172.21.15.144:6805/3635948930","state":["exists","up"]},{"osd":2,"uuid":"b5d7e27a-4fb6-4951-86f4-b0f40050cdd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6801","nonce":3522332994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6803","nonce":3522332994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6807","nonce":3522332994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":3522332994},{"type":"v1","addr":"172.21.15.178:6805","nonce":3522332994}]},"public_addr":"172.21.15.178:6801/3522332994","cluster_addr":"172.21.15.178:6803/3522332994","heartbeat_back_addr":"172.21.15.178:6807/3522332994","heartbeat_front_addr":"172.21.15.178:6805/3522332994","state":["exists","up"]},{"osd":3,"uuid":"4a60f699-6beb-43e3-a94b-e5544e39fec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6811","nonce":3687852925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6813","nonce":3687852925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6817","nonce":3687852925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":3687852925},{"type":"v1","addr":"172.21.15.143:6815","nonce":3687852925}]},"public_addr":"172.21.15.143:6811/3687852925","cluster_addr":"172.21.15.143:6813/3687852925","heartbeat_back_addr":"172.21.15.143:6817/3687852925","heartbeat_front_addr":"172.21.15.143:6815/3687852925","state":["exists","up"]},{"osd":4,"uuid":"d383b3ca-df71-4034-ac1f-bcb431e38265","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6809","nonce":2906236694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6811","nonce":2906236694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6815","nonce":2906236694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2906236694},{"type":"v1","addr":"172.21.15.144:6813","nonce":2906236694}]},"public_addr":"172.21.15.144:6809/2906236694","cluster_addr":"172.21.15.144:6811/2906236694","heartbeat_back_addr":"172.21.15.144:6815/2906236694","heartbeat_front_addr":"172.21.15.144:6813/2906236694","state":["exists","up"]},{"osd":5,"uuid":"bb5db187-d7f1-4f74-a4ec-ff8e4e686270","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6809","nonce":1007276569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6811","nonce":1007276569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6815","nonce":1007276569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1007276569},{"type":"v1","addr":"172.21.15.178:6813","nonce":1007276569}]},"public_addr":"172.21.15.178:6809/1007276569","cluster_addr":"172.21.15.178:6811/1007276569","heartbeat_back_addr":"172.21.15.178:6815/1007276569","heartbeat_front_addr":"172.21.15.178:6813/1007276569","state":["exists","up"]},{"osd":6,"uuid":"3b57aa64-3a10-4155-a2c6-984a15789ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6819","nonce":148519296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6821","nonce":148519296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6825","nonce":148519296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":148519296},{"type":"v1","addr":"172.21.15.143:6823","nonce":148519296}]},"public_addr":"172.21.15.143:6819/148519296","cluster_addr":"172.21.15.143:6821/148519296","heartbeat_back_addr":"172.21.15.143:6825/148519296","heartbeat_front_addr":"172.21.15.143:6823/148519296","state":["exists","up"]},{"osd":7,"uuid":"680d5255-bade-46c7-a4be-00bfb073eafb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6817","nonce":3746149853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6819","nonce":3746149853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6823","nonce":3746149853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3746149853},{"type":"v1","addr":"172.21.15.144:6821","nonce":3746149853}]},"public_addr":"172.21.15.144:6817/3746149853","cluster_addr":"172.21.15.144:6819/3746149853","heartbeat_back_addr":"172.21.15.144:6823/3746149853","heartbeat_front_addr":"172.21.15.144:6821/3746149853","state":["exists","up"]},{"osd":8,"uuid":"f707e74d-dfb8-4966-802f-5df28f3ac0e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6817","nonce":3485367257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6819","nonce":3485367257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6823","nonce":3485367257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3485367257},{"type":"v1","addr":"172.21.15.178:6821","nonce":3485367257}]},"public_addr":"172.21.15.178:6817/3485367257","cluster_addr":"172.21.15.178:6819/3485367257","heartbeat_back_addr":"172.21.15.178:6823/3485367257","heartbeat_front_addr":"172.21.15.178:6821/3485367257","state":["exists","up"]},{"osd":9,"uuid":"3005ede3-bfda-4cfd-a562-8c1540c829af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6827","nonce":4014720297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6829","nonce":4014720297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6833","nonce":4014720297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":4014720297},{"type":"v1","addr":"172.21.15.143:6831","nonce":4014720297}]},"public_addr":"172.21.15.143:6827/4014720297","cluster_addr":"172.21.15.143:6829/4014720297","heartbeat_back_addr":"172.21.15.143:6833/4014720297","heartbeat_front_addr":"172.21.15.143:6831/4014720297","state":["exists","up"]},{"osd":10,"uuid":"1f83408d-f6e1-4760-8b33-81493fb716f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6825","nonce":2444988938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6827","nonce":2444988938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6831","nonce":2444988938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2444988938},{"type":"v1","addr":"172.21.15.144:6829","nonce":2444988938}]},"public_addr":"172.21.15.144:6825/2444988938","cluster_addr":"172.21.15.144:6827/2444988938","heartbeat_back_addr":"172.21.15.144:6831/2444988938","heartbeat_front_addr":"172.21.15.144:6829/2444988938","state":["exists","up"]},{"osd":11,"uuid":"38c6f977-35b1-4373-95af-491a20581f8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6825","nonce":2309120918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6827","nonce":2309120918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6831","nonce":2309120918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2309120918},{"type":"v1","addr":"172.21.15.178:6829","nonce":2309120918}]},"public_addr":"172.21.15.178:6825/2309120918","cluster_addr":"172.21.15.178:6827/2309120918","heartbeat_back_addr":"172.21.15.178:6831/2309120918","heartbeat_front_addr":"172.21.15.178:6829/2309120918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:19.624368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:37:43.933745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:08.955575+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:35.511636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:01.036534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:25.012655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:51.927095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:17.447099+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:43.024223+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:08.575253+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:34.041973+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:59.854681+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:0/3629809110":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/212886119":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6801/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3153560299":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6800/1050213403":"2024-09-11T10:35:59.338289+0000","172.21.15.143:6800/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:6801/3171173333":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/1093984961":"2024-09-11T10:35:32.440973+0000","172.21.15.143:0/2707450669":"2024-09-11T10:35:59.338289+0000","172.21.15.143:0/3503275671":"2024-09-11T10:35:32.440973+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4ef00a2b80 msgr2=0x7f4ef00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4ef00a2b80 0x7f4ef00a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005ab0 tx=0x7f4efc005a20 comp rx=0 tx=0).stop 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 msgr2=0x7f4f08198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 0x7f4f08198370 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800d950 tx=0x7f4ef800de10 comp rx=0 tx=0).stop 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 shutdown_connections 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f4ef00a2b80 0x7f4ef00a5040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f0810c170 0x7f4f0819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f081036b0 0x7f4f081988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 --2- 172.21.15.143:0/2366572901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f08102d90 0x7f4f08198370 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:40.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.101+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 >> 172.21.15.143:0/2366572901 conn(0x7f4f080fc790 msgr2=0x7f4f080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:40.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.102+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 shutdown_connections 2024-09-10T10:54:40.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:40.102+0000 7f4f0d721640 1 -- 172.21.15.143:0/2366572901 wait complete. 2024-09-10T10:54:40.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:39 smithi178 ceph-mon[34813]: pgmap v639: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:40.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:39 smithi144 ceph-mon[34230]: pgmap v639: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:41.021 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T10:54:41.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:40 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2366572901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:54:41.185 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:40 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2366572901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:54:41.267 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:41.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:40 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2366572901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:54:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 -- 172.21.15.143:0/1120574974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 msgr2=0x7f86841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 --2- 172.21.15.143:0/1120574974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841029b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8678009900 tx=0x7f867802a480 comp rx=0 tx=0).stop 2024-09-10T10:54:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 -- 172.21.15.143:0/1120574974 shutdown_connections 2024-09-10T10:54:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 --2- 172.21.15.143:0/1120574974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 0x7f868410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 --2- 172.21.15.143:0/1120574974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841029b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 --2- 172.21.15.143:0/1120574974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 0x7f8684101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 -- 172.21.15.143:0/1120574974 >> 172.21.15.143:0/1120574974 conn(0x7f86840fc7d0 msgr2=0x7f86840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 -- 172.21.15.143:0/1120574974 shutdown_connections 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.758+0000 7f868c26a640 1 -- 172.21.15.143:0/1120574974 wait complete. 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 Processor -- start 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 -- start start 2024-09-10T10:54:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 0x7f86841993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 0x7f86841a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8684110ed0 con 0x7f8684102570 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8684110d50 con 0x7f8684102ef0 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868c26a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8684111050 con 0x7f8684100e20 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37352/0 (socket says 172.21.15.143:37352) 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 -- 172.21.15.143:0/3638881988 learned_addr learned my addr 172.21.15.143:0/3638881988 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868a807640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 0x7f86841a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868a006640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 0x7f86841993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 -- 172.21.15.143:0/3638881988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 msgr2=0x7f86841993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 0x7f86841993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 -- 172.21.15.143:0/3638881988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 msgr2=0x7f86841a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:41.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 0x7f86841a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 -- 172.21.15.143:0/3638881988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86841a4960 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868a807640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 0x7f86841a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f868a006640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 0x7f86841993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f8689805640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841998e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f86780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.759+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867803f070 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.760+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8678003aa0 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.760+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86841a4b30 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.760+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678035030 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.760+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86841a4fe0 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.760+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8684101200 con 0x7f8684102570 2024-09-10T10:54:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.762+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8678042070 con 0x7f8684102570 2024-09-10T10:54:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.762+0000 7f86777fe640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86700a2c00 0x7f86700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.762+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f86780f0380 con 0x7f8684102570 2024-09-10T10:54:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.762+0000 7f868a006640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86700a2c00 0x7f86700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.763+0000 7f868a006640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86700a2c00 0x7f86700a50c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8680006fd0 tx=0x7f8680008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.765+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86780bcfc0 con 0x7f8684102570 2024-09-10T10:54:41.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.968+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8684109090 con 0x7f8684102570 2024-09-10T10:54:41.970 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:41 smithi143 ceph-mon[31057]: pgmap v640: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:41.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.971+0000 7f86777fe640 1 -- 172.21.15.143:0/3638881988 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f86780bc8c0 con 0x7f8684102570 2024-09-10T10:54:41.972 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86700a2c00 msgr2=0x7f86700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86700a2c00 0x7f86700a50c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8680006fd0 tx=0x7f8680008040 comp rx=0 tx=0).stop 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 msgr2=0x7f86841998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841998e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f86780042f0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 shutdown_connections 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f86700a2c00 0x7f86700a50c0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8680006fd0 tx=0x7f8680008040 comp rx=0 tx=0).stop 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8684102ef0 0x7f86841a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8684102570 0x7f86841998e0 secure :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f86780042f0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 --2- 172.21.15.143:0/3638881988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8684100e20 0x7f86841993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:41.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.974+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 >> 172.21.15.143:0/3638881988 conn(0x7f86840fc7d0 msgr2=0x7f86840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:41.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.975+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 shutdown_connections 2024-09-10T10:54:41.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:41.975+0000 7f868c26a640 1 -- 172.21.15.143:0/3638881988 wait complete. 2024-09-10T10:54:42.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:41 smithi144 ceph-mon[34230]: pgmap v640: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:42.186 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k[63222]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.186 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k[63222]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.186 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mds[63226]: -- 172.21.15.144:0/1090590913 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4dea3b860 con 0x55e4dea18800 2024-09-10T10:54:42.186 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k[63222]: starting mds.k at 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b[64142]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b[64142]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mds[64146]: -- 172.21.15.144:0/708388768 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55940ab83860 con 0x55940ab60800 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b[64142]: starting mds.b at 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e[63526]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e[63526]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mds[63530]: -- 172.21.15.144:0/2466323787 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e098b67860 con 0x55e098b44800 2024-09-10T10:54:42.187 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e[63526]: starting mds.e at 2024-09-10T10:54:42.188 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h[63830]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.188 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h[63830]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.188 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mds[63834]: -- 172.21.15.144:0/885957275 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c17f77860 con 0x563c17f54800 2024-09-10T10:54:42.188 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h[63830]: starting mds.h at 2024-09-10T10:54:42.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:41 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a[31053]: 2024-09-10T10:54:41.969+0000 7f21b2f48640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:42.301 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a[72470]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a[72470]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mds[72484]: -- 172.21.15.143:0/2858721361 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e07fc97860 con 0x55e07fc75400 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a[72470]: starting mds.a at 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g[72138]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g[72138]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mds[72144]: -- 172.21.15.143:0/1686238471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55641f855860 con 0x55641f833400 2024-09-10T10:54:42.302 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g[72138]: starting mds.g at 2024-09-10T10:54:42.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:41 smithi178 ceph-mon[34813]: pgmap v640: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T10:54:42.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mds[65286]: -- 172.21.15.178:0/4087040014 <== mon.0 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d14be51860 con 0x55d14be2e800 2024-09-10T10:54:42.426 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: starting mds.i at 2024-09-10T10:54:42.426 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l[64380]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.426 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l[64380]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.426 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l[64380]: starting mds.l at 2024-09-10T10:54:42.426 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c[64979]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.426 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c[64979]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.427 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mds[64983]: -- 172.21.15.178:0/2170392468 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590d043f860 con 0x5590d041d400 2024-09-10T10:54:42.427 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c[64979]: starting mds.c at 2024-09-10T10:54:42.427 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:42.427 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:42.427 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mds[64684]: -- 172.21.15.178:0/2195792808 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cceb397860 con 0x55cceb375400 2024-09-10T10:54:42.427 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: starting mds.f at 2024-09-10T10:54:42.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:54:43.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3638881988' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:54:43.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mon[31057]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:54:43.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mon[31057]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:43.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/3638881988' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:54:43.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mon[31057]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:54:43.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:42 smithi143 ceph-mon[31057]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:54:43.059 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3638881988' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:54:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mon[34230]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:54:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mon[34230]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/3638881988' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:54:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mon[34230]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:54:43.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:42 smithi144 ceph-mon[34230]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:54:43.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3638881988' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:54:43.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mon[34813]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:54:43.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mon[34813]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:43.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/3638881988' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:54:43.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mon[34813]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:54:43.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:42 smithi178 ceph-mon[34813]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:54:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 -- 172.21.15.143:0/3967943558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c102d40 msgr2=0x7f8a3c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 --2- 172.21.15.143:0/3967943558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c102d40 0x7f8a3c103120 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009900 tx=0x7f8a3802a480 comp rx=0 tx=0).stop 2024-09-10T10:54:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 -- 172.21.15.143:0/3967943558 shutdown_connections 2024-09-10T10:54:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 --2- 172.21.15.143:0/3967943558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a3c10c170 0x7f8a3c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 --2- 172.21.15.143:0/3967943558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 0x7f8a3c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 --2- 172.21.15.143:0/3967943558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c102d40 0x7f8a3c103120 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 -- 172.21.15.143:0/3967943558 >> 172.21.15.143:0/3967943558 conn(0x7f8a3c0fc7a0 msgr2=0x7f8a3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 -- 172.21.15.143:0/3967943558 shutdown_connections 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.569+0000 7f8a444c8640 1 -- 172.21.15.143:0/3967943558 wait complete. 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 Processor -- start 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 -- start start 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a3c102d40 0x7f8a3c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 0x7f8a3c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:43.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 0x7f8a3c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111ee0 con 0x7f8a3c10c170 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111d60 con 0x7f8a3c103660 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a444c8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c112060 con 0x7f8a3c102d40 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 0x7f8a3c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 0x7f8a3c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37424/0 (socket says 172.21.15.143:37424) 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 -- 172.21.15.143:0/1594794033 learned_addr learned my addr 172.21.15.143:0/1594794033 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:43.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42264640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a3c102d40 0x7f8a3c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a41a63640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 0x7f8a3c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 -- 172.21.15.143:0/1594794033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a3c102d40 msgr2=0x7f8a3c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a3c102d40 0x7f8a3c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 -- 172.21.15.143:0/1594794033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 msgr2=0x7f8a3c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 0x7f8a3c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 -- 172.21.15.143:0/1594794033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1a69c0 con 0x7f8a3c10c170 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a41a63640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 0x7f8a3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.570+0000 7f8a42a65640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 0x7f8a3c101420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b970 tx=0x7f8a3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.571+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34019070 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.571+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a34014440 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.571+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a6cb0 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.571+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a340075d0 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.571+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a71a0 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.572+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c1031a0 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.573+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8a3401c070 con 0x7f8a3c10c170 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.574+0000 7f8a337fe640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a280a2b80 0x7f8a280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.574+0000 7f8a42264640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a280a2b80 0x7f8a280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.574+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9522+0+0 (secure 0 0 0) 0x7f8a340ceb20 con 0x7f8a3c10c170 2024-09-10T10:54:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.574+0000 7f8a42264640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a280a2b80 0x7f8a280a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a38031440 tx=0x7f8a38005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.578+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a3409af20 con 0x7f8a3c10c170 2024-09-10T10:54:43.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.781+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8a3c107210 con 0x7f8a3c10c170 2024-09-10T10:54:43.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.978+0000 7f8a337fe640 1 -- 172.21.15.143:0/1594794033 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f8a3409f220 con 0x7f8a3c10c170 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a280a2b80 msgr2=0x7f8a280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a280a2b80 0x7f8a280a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a38031440 tx=0x7f8a38005e80 comp rx=0 tx=0).stop 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 msgr2=0x7f8a3c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 0x7f8a3c101420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b970 tx=0x7f8a3400be30 comp rx=0 tx=0).stop 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 shutdown_connections 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f8a280a2b80 0x7f8a280a5040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3c10c170 0x7f8a3c101420 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c103660 0x7f8a3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 --2- 172.21.15.143:0/1594794033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a3c102d40 0x7f8a3c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 >> 172.21.15.143:0/1594794033 conn(0x7f8a3c0fc7a0 msgr2=0x7f8a3c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 shutdown_connections 2024-09-10T10:54:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:43.980+0000 7f8a444c8640 1 -- 172.21.15.143:0/1594794033 wait complete. 2024-09-10T10:54:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.143:6838/1431092883,v1:172.21.15.143:6839/1431092883] up:boot 2024-09-10T10:54:44.051 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.143:6840/3672517148,v1:172.21.15.143:6841/3672517148] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6832/1563216534,v1:172.21.15.144:6833/1563216534] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6834/1479894998,v1:172.21.15.144:6835/1479894998] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6836/295478371,v1:172.21.15.144:6837/295478371] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6833/1109733917,v1:172.21.15.178:6837/1109733917] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6832/2607386382,v1:172.21.15.178:6835/2607386382] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6834/576348704,v1:172.21.15.178:6836/576348704] up:boot 2024-09-10T10:54:44.052 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.144:6838/2654798538,v1:172.21.15.144:6839/2654798538] up:boot 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: mds.? [v2:172.21.15.178:6838/1778754051,v1:172.21.15.178:6839/1778754051] up:boot 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:54:44.053 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:54:44.054 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:54:44.054 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:54:44.054 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:54:44.054 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:54:44.054 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: pgmap v642: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-10T10:54:44.054 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:43 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1594794033' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:54:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.143:6838/1431092883,v1:172.21.15.143:6839/1431092883] up:boot 2024-09-10T10:54:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.143:6840/3672517148,v1:172.21.15.143:6841/3672517148] up:boot 2024-09-10T10:54:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6832/1563216534,v1:172.21.15.144:6833/1563216534] up:boot 2024-09-10T10:54:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6834/1479894998,v1:172.21.15.144:6835/1479894998] up:boot 2024-09-10T10:54:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6836/295478371,v1:172.21.15.144:6837/295478371] up:boot 2024-09-10T10:54:44.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6833/1109733917,v1:172.21.15.178:6837/1109733917] up:boot 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6832/2607386382,v1:172.21.15.178:6835/2607386382] up:boot 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6834/576348704,v1:172.21.15.178:6836/576348704] up:boot 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.144:6838/2654798538,v1:172.21.15.144:6839/2654798538] up:boot 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: mds.? [v2:172.21.15.178:6838/1778754051,v1:172.21.15.178:6839/1778754051] up:boot 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:54:44.426 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:54:44.427 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:54:44.427 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: pgmap v642: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-10T10:54:44.427 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:43 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1594794033' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:54:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.143:6838/1431092883,v1:172.21.15.143:6839/1431092883] up:boot 2024-09-10T10:54:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.143:6840/3672517148,v1:172.21.15.143:6841/3672517148] up:boot 2024-09-10T10:54:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6832/1563216534,v1:172.21.15.144:6833/1563216534] up:boot 2024-09-10T10:54:44.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6834/1479894998,v1:172.21.15.144:6835/1479894998] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6836/295478371,v1:172.21.15.144:6837/295478371] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6833/1109733917,v1:172.21.15.178:6837/1109733917] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6832/2607386382,v1:172.21.15.178:6835/2607386382] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6834/576348704,v1:172.21.15.178:6836/576348704] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.144:6838/2654798538,v1:172.21.15.144:6839/2654798538] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: mds.? [v2:172.21.15.178:6838/1778754051,v1:172.21.15.178:6839/1778754051] up:boot 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:54:44.437 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:54:44.438 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:54:44.438 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:54:44.438 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:54:44.438 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='mgr.14150 172.21.15.143:0/2054631927' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:54:44.438 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: pgmap v642: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-09-10T10:54:44.438 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:43 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1594794033' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:54:44.742 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:54:44.987 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:45.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:44 smithi143 ceph-mon[31057]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:54:45.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:44 smithi143 ceph-mon[31057]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:54:45.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:44 smithi143 ceph-mon[31057]: Cluster is now healthy 2024-09-10T10:54:45.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:44 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1594794033' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:54:45.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:44 smithi143 ceph-mon[31057]: fsmap 12 up:standby 2024-09-10T10:54:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:44 smithi178 ceph-mon[34813]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:54:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:44 smithi178 ceph-mon[34813]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:54:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:44 smithi178 ceph-mon[34813]: Cluster is now healthy 2024-09-10T10:54:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:44 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1594794033' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:54:45.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:44 smithi178 ceph-mon[34813]: fsmap 12 up:standby 2024-09-10T10:54:45.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:44 smithi144 ceph-mon[34230]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:54:45.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:44 smithi144 ceph-mon[34230]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:54:45.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:44 smithi144 ceph-mon[34230]: Cluster is now healthy 2024-09-10T10:54:45.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:44 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1594794033' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:54:45.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:44 smithi144 ceph-mon[34230]: fsmap 12 up:standby 2024-09-10T10:54:45.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 -- 172.21.15.143:0/3671607105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec106f10 msgr2=0x7f83ec10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 --2- 172.21.15.143:0/3671607105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec106f10 0x7f83ec10ea90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009ca0 tx=0x7f83e402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 -- 172.21.15.143:0/3671607105 shutdown_connections 2024-09-10T10:54:45.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 --2- 172.21.15.143:0/3671607105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec106f10 0x7f83ec10ea90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 --2- 172.21.15.143:0/3671607105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83ec102790 0x7f83ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 --2- 172.21.15.143:0/3671607105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83ec101de0 0x7f83ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 -- 172.21.15.143:0/3671607105 >> 172.21.15.143:0/3671607105 conn(0x7f83ec0fd8c0 msgr2=0x7f83ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 -- 172.21.15.143:0/3671607105 shutdown_connections 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 -- 172.21.15.143:0/3671607105 wait complete. 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 Processor -- start 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.474+0000 7f83f2f16640 1 -- start start 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f2f16640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 0x7f83ec198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f2f16640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83ec102790 0x7f83ec198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f2f16640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83ec106f10 0x7f83ec19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f2f16640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec111e40 con 0x7f83ec101de0 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f2f16640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec111cc0 con 0x7f83ec102790 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f2f16640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec111fc0 con 0x7f83ec106f10 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 0x7f83ec198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 0x7f83ec198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37434/0 (socket says 172.21.15.143:37434) 2024-09-10T10:54:45.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 -- 172.21.15.143:0/715762137 learned_addr learned my addr 172.21.15.143:0/715762137 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83ebfff640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83ec102790 0x7f83ec198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f14b3640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83ec106f10 0x7f83ec19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 -- 172.21.15.143:0/715762137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83ec106f10 msgr2=0x7f83ec19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83ec106f10 0x7f83ec19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 -- 172.21.15.143:0/715762137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83ec102790 msgr2=0x7f83ec198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83ec102790 0x7f83ec198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 -- 172.21.15.143:0/715762137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec19d3b0 con 0x7f83ec101de0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83f0cb2640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 0x7f83ec198400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f83e000d950 tx=0x7f83e000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.475+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0017070 con 0x7f83ec101de0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.476+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec19d6a0 con 0x7f83ec101de0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.476+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83e000de80 con 0x7f83ec101de0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.476+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e000f620 con 0x7f83ec101de0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.476+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec1a4f80 con 0x7f83ec101de0 2024-09-10T10:54:45.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.477+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec1021c0 con 0x7f83ec101de0 2024-09-10T10:54:45.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.478+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f83e0009c30 con 0x7f83ec101de0 2024-09-10T10:54:45.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.478+0000 7f83e9ffb640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f83dc0a2b80 0x7f83dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.478+0000 7f83ebfff640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f83dc0a2b80 0x7f83dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.478+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9522+0+0 (secure 0 0 0) 0x7f83e00cf120 con 0x7f83ec101de0 2024-09-10T10:54:45.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.479+0000 7f83ebfff640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f83dc0a2b80 0x7f83dc0a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0ffe50 tx=0x7f83d8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.482+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83e009bb70 con 0x7f83ec101de0 2024-09-10T10:54:45.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.653+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f83ec10a0f0 con 0x7f83ec101de0 2024-09-10T10:54:45.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.984+0000 7f83e9ffb640 1 -- 172.21.15.143:0/715762137 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f83e009b470 con 0x7f83ec101de0 2024-09-10T10:54:45.986 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f83dc0a2b80 msgr2=0x7f83dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f83dc0a2b80 0x7f83dc0a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0ffe50 tx=0x7f83d8009110 comp rx=0 tx=0).stop 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 msgr2=0x7f83ec198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 0x7f83ec198400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f83e000d950 tx=0x7f83e000dd50 comp rx=0 tx=0).stop 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 shutdown_connections 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f83dc0a2b80 0x7f83dc0a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83ec106f10 0x7f83ec19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83ec102790 0x7f83ec198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 --2- 172.21.15.143:0/715762137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83ec101de0 0x7f83ec198400 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.988+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 >> 172.21.15.143:0/715762137 conn(0x7f83ec0fd8c0 msgr2=0x7f83ec0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.989+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 shutdown_connections 2024-09-10T10:54:45.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:45.989+0000 7f83f2f16640 1 -- 172.21.15.143:0/715762137 wait complete. 2024-09-10T10:54:46.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:45 smithi143 ceph-mon[31057]: pgmap v643: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T10:54:46.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:45 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/715762137' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:45 smithi178 ceph-mon[34813]: pgmap v643: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T10:54:46.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:45 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/715762137' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:46.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:45 smithi144 ceph-mon[34230]: pgmap v643: 129 pgs: 129 active+clean; 169 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-10T10:54:46.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:45 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/715762137' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:46.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:54:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:47.241 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:46 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/715762137' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:47.241 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:46 smithi143 ceph-mon[31057]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:54:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:46 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/715762137' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:47.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:46 smithi178 ceph-mon[34813]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:54:47.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:46 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/715762137' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:47.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:46 smithi144 ceph-mon[34230]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:54:47.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 -- 172.21.15.143:0/138434510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928101420 msgr2=0x7f692810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 --2- 172.21.15.143:0/138434510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928101420 0x7f692810d950 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6920009cd0 tx=0x7f692002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 -- 172.21.15.143:0/138434510 shutdown_connections 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 --2- 172.21.15.143:0/138434510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928101420 0x7f692810d950 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 --2- 172.21.15.143:0/138434510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928100a80 0x7f6928100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 --2- 172.21.15.143:0/138434510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6928106a80 0x7f6928106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 -- 172.21.15.143:0/138434510 >> 172.21.15.143:0/138434510 conn(0x7f69280fc7a0 msgr2=0x7f69280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 -- 172.21.15.143:0/138434510 shutdown_connections 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.467+0000 7f69300f1640 1 -- 172.21.15.143:0/138434510 wait complete. 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 Processor -- start 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 -- start start 2024-09-10T10:54:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 0x7f6928199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928101420 0x7f69281996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6928106a80 0x7f69281a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6928110d30 con 0x7f6928100a80 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6928110bb0 con 0x7f6928106a80 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f69300f1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6928110eb0 con 0x7f6928101420 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f692de8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 0x7f6928199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f692de8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 0x7f6928199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57770/0 (socket says 172.21.15.143:57770) 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f692de8d640 1 -- 172.21.15.143:0/1521690522 learned_addr learned my addr 172.21.15.143:0/1521690522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.468+0000 7f692e68e640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6928106a80 0x7f69281a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692d68c640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928101420 0x7f69281996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692de8d640 1 -- 172.21.15.143:0/1521690522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928101420 msgr2=0x7f69281996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692de8d640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928101420 0x7f69281996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692de8d640 1 -- 172.21.15.143:0/1521690522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6928106a80 msgr2=0x7f69281a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692de8d640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6928106a80 0x7f69281a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692de8d640 1 -- 172.21.15.143:0/1521690522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69281a4740 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692d68c640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928101420 0x7f69281996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f692de8d640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 0x7f6928199180 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f692400d950 tx=0x7f692400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924017030 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6924004020 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69281a4910 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240047a0 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.469+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69281a4e20 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.470+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6928100ee0 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.471+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f69240041d0 con 0x7f6928100a80 2024-09-10T10:54:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.472+0000 7f691effd640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f69140a2b80 0x7f69140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.472+0000 7f692d68c640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f69140a2b80 0x7f69140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.472+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8997+0+0 (secure 0 0 0) 0x7f69240ced70 con 0x7f6928100a80 2024-09-10T10:54:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.472+0000 7f692d68c640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f69140a2b80 0x7f69140a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f692806b2e0 tx=0x7f6918005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.476+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f692409b9d0 con 0x7f6928100a80 2024-09-10T10:54:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.643+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6928109040 con 0x7f6928100a80 2024-09-10T10:54:47.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.991+0000 7f691effd640 1 -- 172.21.15.143:0/1521690522 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f692409b2d0 con 0x7f6928100a80 2024-09-10T10:54:47.993 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2024-09-10T10:54:47.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f69140a2b80 msgr2=0x7f69140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f69140a2b80 0x7f69140a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f692806b2e0 tx=0x7f6918005e50 comp rx=0 tx=0).stop 2024-09-10T10:54:47.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 msgr2=0x7f6928199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 0x7f6928199180 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f692400d950 tx=0x7f692400de10 comp rx=0 tx=0).stop 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 shutdown_connections 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f69140a2b80 0x7f69140a5040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6928106a80 0x7f69281a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6928101420 0x7f69281996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 --2- 172.21.15.143:0/1521690522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6928100a80 0x7f6928199180 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 >> 172.21.15.143:0/1521690522 conn(0x7f69280fc7a0 msgr2=0x7f69280fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 shutdown_connections 2024-09-10T10:54:47.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:47.995+0000 7f69300f1640 1 -- 172.21.15.143:0/1521690522 wait complete. 2024-09-10T10:54:48.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:47 smithi143 ceph-mon[31057]: pgmap v645: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:54:48.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:47 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1521690522' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:48.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:47 smithi178 ceph-mon[34813]: pgmap v645: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:54:48.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:47 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1521690522' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:48.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:47 smithi144 ceph-mon[34230]: pgmap v645: 97 pgs: 97 active+clean; 577 KiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:54:48.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:47 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1521690522' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:48.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 34212b78-6f60-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T10:54:49.049 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:54:49.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:48 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/1521690522' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:49.050 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:48 smithi143 ceph-mon[31057]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:54:49.326 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:48 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/1521690522' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:49.326 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:48 smithi178 ceph-mon[34813]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:54:49.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:48 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/1521690522' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:49.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:48 smithi144 ceph-mon[34230]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 -- 172.21.15.143:0/3906876220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f658810c170 msgr2=0x7f658810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 --2- 172.21.15.143:0/3906876220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f658810c170 0x7f658810e610 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6580009900 tx=0x7f658002a480 comp rx=0 tx=0).stop 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 -- 172.21.15.143:0/3906876220 shutdown_connections 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 --2- 172.21.15.143:0/3906876220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f658810c170 0x7f658810e610 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 --2- 172.21.15.143:0/3906876220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6588103660 0x7f658810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 --2- 172.21.15.143:0/3906876220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6588102d40 0x7f6588103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 -- 172.21.15.143:0/3906876220 >> 172.21.15.143:0/3906876220 conn(0x7f65880fc7a0 msgr2=0x7f65880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 -- 172.21.15.143:0/3906876220 shutdown_connections 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.535+0000 7f658df5d640 1 -- 172.21.15.143:0/3906876220 wait complete. 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 Processor -- start 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 -- start start 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6588102d40 0x7f65881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 0x7f6588100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f658810c170 0x7f6588101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6588111e00 con 0x7f6588103660 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6588111c80 con 0x7f6588102d40 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f658df5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6588111f80 con 0x7f658810c170 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f6586ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 0x7f6588100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f6586ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 0x7f6588100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57776/0 (socket says 172.21.15.143:57776) 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f6586ffd640 1 -- 172.21.15.143:0/2028021157 learned_addr learned my addr 172.21.15.143:0/2028021157 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.536+0000 7f65877fe640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6588102d40 0x7f65881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6586ffd640 1 -- 172.21.15.143:0/2028021157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f658810c170 msgr2=0x7f6588101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6586ffd640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f658810c170 0x7f6588101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6586ffd640 1 -- 172.21.15.143:0/2028021157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6588102d40 msgr2=0x7f65881009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6586ffd640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6588102d40 0x7f65881009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6586ffd640 1 -- 172.21.15.143:0/2028021157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65881a6850 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6586ffd640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 0x7f6588100ee0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d750 tx=0x7f657c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c017070 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f657c005020 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65881a6b40 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c004c60 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.537+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65881a7080 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.538+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6588103120 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.539+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f657c009a30 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.540+0000 7f6584ff9640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65700a2c00 0x7f65700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.540+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8496+0+0 (secure 0 0 0) 0x7f657c0ceac0 con 0x7f6588103660 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.540+0000 7f65877fe640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65700a2c00 0x7f65700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.540+0000 7f65877fe640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65700a2c00 0x7f65700a50c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6578005fd0 tx=0x7f6578009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.544+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f657c09b920 con 0x7f6588103660 2024-09-10T10:54:49.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.710+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6588107240 con 0x7f6588103660 2024-09-10T10:54:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:49.997+0000 7f6584ff9640 1 -- 172.21.15.143:0/2028021157 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) ==== 178+0+0 (secure 0 0 0) 0x7f657c09b220 con 0x7f6588103660 2024-09-10T10:54:49.999 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data_ec' removed 2024-09-10T10:54:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.000+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65700a2c00 msgr2=0x7f65700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.000+0000 7f658df5d640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65700a2c00 0x7f65700a50c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6578005fd0 tx=0x7f6578009870 comp rx=0 tx=0).stop 2024-09-10T10:54:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.000+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 msgr2=0x7f6588100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.000+0000 7f658df5d640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 0x7f6588100ee0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d750 tx=0x7f657c00dc10 comp rx=0 tx=0).stop 2024-09-10T10:54:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.001+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 shutdown_connections 2024-09-10T10:54:50.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.001+0000 7f658df5d640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:6800/1095035473,v1:172.21.15.143:6801/1095035473] conn(0x7f65700a2c00 0x7f65700a50c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.001+0000 7f658df5d640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f658810c170 0x7f6588101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.001+0000 7f658df5d640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6588103660 0x7f6588100ee0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.001+0000 7f658df5d640 1 --2- 172.21.15.143:0/2028021157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6588102d40 0x7f65881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.001+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 >> 172.21.15.143:0/2028021157 conn(0x7f65880fc7a0 msgr2=0x7f65880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.002+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 shutdown_connections 2024-09-10T10:54:50.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-09-10T10:54:50.002+0000 7f658df5d640 1 -- 172.21.15.143:0/2028021157 wait complete. 2024-09-10T10:54:50.300 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:49 smithi143 ceph-mon[31057]: pgmap v647: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:54:50.301 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:49 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2028021157' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:50.337 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:50 smithi144 ceph-mon[34230]: pgmap v647: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:54:50.337 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:50 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2028021157' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:50.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:49 smithi178 ceph-mon[34813]: pgmap v647: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:54:50.338 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:49 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2028021157' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:54:51.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:54:51.091 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:54:51.119 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:54:51.169 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T10:54:51.170 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:54:51.185 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:54:51.200 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:54:51.224 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T10:54:51.224 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T10:54:51.224 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a 2024-09-10T10:54:51.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 ceph-mon[31057]: from='client.? 172.21.15.143:0/2028021157' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:51.253 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 ceph-mon[31057]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:54:51.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:51 smithi178 ceph-mon[34813]: from='client.? 172.21.15.143:0/2028021157' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:51.425 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:51 smithi178 ceph-mon[34813]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:54:51.435 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:51 smithi144 ceph-mon[34230]: from='client.? 172.21.15.143:0/2028021157' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:54:51.436 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:51 smithi144 ceph-mon[34230]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:54:51.539 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 systemd[1]: Stopping Ceph mon.a for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:51.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a[31053]: 2024-09-10T10:54:51.537+0000 7f21b8753640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:54:51.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a[31053]: 2024-09-10T10:54:51.537+0000 7f21b8753640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T10:54:51.801 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 podman[157675]: 2024-09-10 10:54:51.589461659 +0000 UTC m=+0.172507571 container died bd3317ce38ed9d1b3c7c110f0a5efffff7b36da64ba0b0809d790759dd7c683b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T10:54:52.266 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 podman[157675]: 2024-09-10 10:54:51.996419662 +0000 UTC m=+0.579465577 container cleanup bd3317ce38ed9d1b3c7c110f0a5efffff7b36da64ba0b0809d790759dd7c683b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T10:54:52.266 INFO:journalctl@ceph.mon.a.smithi143.stdout:Sep 10 10:54:51 smithi143 bash[157675]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-a 2024-09-10T10:54:52.413 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T10:54:52.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:54:52.503 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T10:54:52.504 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T10:54:52.504 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.b 2024-09-10T10:54:52.821 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:52 smithi144 systemd[1]: Stopping Ceph mon.b for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:52 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-b[34226]: 2024-09-10T10:54:52.818+0000 7f2e13098640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:54:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:52 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-b[34226]: 2024-09-10T10:54:52.818+0000 7f2e13098640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:54:53.186 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:52 smithi144 podman[68018]: 2024-09-10 10:54:52.901742229 +0000 UTC m=+0.205740330 container died 5d8c788f62f0f096757a84298542b012b3cdc0a77b0dd85dcfe9d69004a93357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:54:53.673 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:53 smithi144 podman[68018]: 2024-09-10 10:54:53.28706121 +0000 UTC m=+0.591059310 container cleanup 5d8c788f62f0f096757a84298542b012b3cdc0a77b0dd85dcfe9d69004a93357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:54:53.673 INFO:journalctl@ceph.mon.b.smithi144.stdout:Sep 10 10:54:53 smithi144 bash[68018]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-b 2024-09-10T10:54:53.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T10:54:53.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:54:53.909 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T10:54:53.909 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T10:54:53.909 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.c 2024-09-10T10:54:54.275 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:53 smithi178 systemd[1]: Stopping Ceph mon.c for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:54.550 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-c[34806]: 2024-09-10T10:54:54.273+0000 7f8505497640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:54:54.550 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-c[34806]: 2024-09-10T10:54:54.273+0000 7f8505497640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:54:54.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:54 smithi178 podman[69016]: 2024-09-10 10:54:54.549188559 +0000 UTC m=+0.399350891 container died 4cf4bb79c3bbbc12d4c5845abdfd4dca3338ee0338e0e994d40249f7c02c6329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:54:55.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:54 smithi178 podman[69016]: 2024-09-10 10:54:54.949277329 +0000 UTC m=+0.799439702 container cleanup 4cf4bb79c3bbbc12d4c5845abdfd4dca3338ee0338e0e994d40249f7c02c6329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:54:55.175 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 10:54:54 smithi178 bash[69016]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mon-c 2024-09-10T10:54:55.398 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T10:54:55.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:54:55.446 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T10:54:55.447 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T10:54:55.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x 2024-09-10T10:54:55.770 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:54:55 smithi143 systemd[1]: Stopping Ceph mgr.x for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:56.050 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:54:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x[31285]: 2024-09-10T10:54:55.767+0000 7f5466d31640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:54:56.051 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:54:55 smithi143 podman[157795]: 2024-09-10 10:54:55.799578571 +0000 UTC m=+0.160207223 container died 9b44e769c0df4075eb1d02bba7bbb7ed5eafcf92bf45be5946c29cb4ce31f8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T10:54:56.455 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:54:56 smithi143 podman[157795]: 2024-09-10 10:54:56.21120974 +0000 UTC m=+0.571838383 container cleanup 9b44e769c0df4075eb1d02bba7bbb7ed5eafcf92bf45be5946c29cb4ce31f8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T10:54:56.455 INFO:journalctl@ceph.mgr.x.smithi143.stdout:Sep 10 10:54:56 smithi143 bash[157795]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-x 2024-09-10T10:54:56.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T10:54:56.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:54:56.664 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T10:54:56.665 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T10:54:56.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.y 2024-09-10T10:54:57.014 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:54:56 smithi144 systemd[1]: Stopping Ceph mgr.y for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:57.310 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:54:57 smithi144 podman[68141]: 2024-09-10 10:54:57.013157885 +0000 UTC m=+0.150656263 container died 945d678f50e81f903711b277dc8b9fafb2653606612880c419e30e03c9b9c142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T10:54:57.661 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:54:57 smithi144 podman[68141]: 2024-09-10 10:54:57.401004721 +0000 UTC m=+0.538503099 container cleanup 945d678f50e81f903711b277dc8b9fafb2653606612880c419e30e03c9b9c142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T10:54:57.661 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Sep 10 10:54:57 smithi144 bash[68141]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-y 2024-09-10T10:54:57.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T10:54:57.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:54:57.866 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T10:54:57.866 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T10:54:57.866 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.z 2024-09-10T10:54:58.175 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:54:57 smithi178 systemd[1]: Stopping Ceph mgr.z for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:58.490 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:54:58 smithi178 podman[69138]: 2024-09-10 10:54:58.184469443 +0000 UTC m=+0.129491423 container died 0a2a6923d0d59979e50e28b27fdf0d48023e030dc50c3b56f2b3fa3e8ddd22e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T10:54:58.875 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:54:58 smithi178 podman[69138]: 2024-09-10 10:54:58.58909231 +0000 UTC m=+0.534114278 container cleanup 0a2a6923d0d59979e50e28b27fdf0d48023e030dc50c3b56f2b3fa3e8ddd22e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T10:54:58.875 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 10:54:58 smithi178 bash[69138]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mgr-z 2024-09-10T10:54:58.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T10:54:59.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:54:59.033 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T10:54:59.033 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T10:54:59.033 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.0 2024-09-10T10:54:59.358 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:54:59 smithi143 systemd[1]: Stopping Ceph osd.0 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:54:59.800 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:54:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0[45426]: 2024-09-10T10:54:59.356+0000 7f019f854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:54:59.801 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:54:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0[45426]: 2024-09-10T10:54:59.356+0000 7f019f854640 -1 osd.0 93 *** Got signal Terminated *** 2024-09-10T10:54:59.801 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:54:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0[45426]: 2024-09-10T10:54:59.356+0000 7f019f854640 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:05.235 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:04 smithi143 podman[157916]: 2024-09-10 10:55:04.80964845 +0000 UTC m=+5.576772833 container died 9870fca03ca123b9c126fbe132b99c761864268f2afad92be92f6a7de9f610f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:55:05.550 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:05 smithi143 podman[157916]: 2024-09-10 10:55:05.233783956 +0000 UTC m=+6.000908407 container cleanup 9870fca03ca123b9c126fbe132b99c761864268f2afad92be92f6a7de9f610f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:05.551 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:05 smithi143 bash[157916]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0 2024-09-10T10:55:05.551 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:05 smithi143 podman[157932]: 2024-09-10 10:55:05.452131922 +0000 UTC m=+0.643121760 container remove 9870fca03ca123b9c126fbe132b99c761864268f2afad92be92f6a7de9f610f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T10:55:06.056 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:05 smithi143 podman[158017]: 2024-09-10 10:55:05.801824858 +0000 UTC m=+0.016183773 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:06.056 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:05 smithi143 podman[158017]: 2024-09-10 10:55:05.92830899 +0000 UTC m=+0.142667880 container create 52ad8405c233eb79110a7e7c0042a90249e0791784168b1a09f2d671a5c5205b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:06.391 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:06 smithi143 podman[158017]: 2024-09-10 10:55:06.198130817 +0000 UTC m=+0.412489706 container init 52ad8405c233eb79110a7e7c0042a90249e0791784168b1a09f2d671a5c5205b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:55:06.391 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:06 smithi143 podman[158017]: 2024-09-10 10:55:06.201365428 +0000 UTC m=+0.415724329 container start 52ad8405c233eb79110a7e7c0042a90249e0791784168b1a09f2d671a5c5205b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T10:55:06.391 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:06 smithi143 podman[158017]: 2024-09-10 10:55:06.298208725 +0000 UTC m=+0.512567641 container attach 52ad8405c233eb79110a7e7c0042a90249e0791784168b1a09f2d671a5c5205b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:06.801 INFO:journalctl@ceph.osd.0.smithi143.stdout:Sep 10 10:55:06 smithi143 podman[158017]: 2024-09-10 10:55:06.390214205 +0000 UTC m=+0.604573110 container died 52ad8405c233eb79110a7e7c0042a90249e0791784168b1a09f2d671a5c5205b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:55:07.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T10:55:07.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:07.306 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T10:55:07.306 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T10:55:07.306 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.1 2024-09-10T10:55:07.623 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:07 smithi144 systemd[1]: Stopping Ceph osd.1 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:07.936 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1[38581]: 2024-09-10T10:55:07.621+0000 7f3956a0a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:07.936 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1[38581]: 2024-09-10T10:55:07.621+0000 7f3956a0a640 -1 osd.1 93 *** Got signal Terminated *** 2024-09-10T10:55:07.936 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1[38581]: 2024-09-10T10:55:07.621+0000 7f3956a0a640 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:13.358 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:13 smithi144 podman[68263]: 2024-09-10 10:55:13.049899592 +0000 UTC m=+5.543463107 container died 6d21faab35c565a9efed3df44c7793f56d4396868e433b3234fc1b3fdef03144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:55:13.686 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:13 smithi144 podman[68263]: 2024-09-10 10:55:13.463675924 +0000 UTC m=+5.957239437 container cleanup 6d21faab35c565a9efed3df44c7793f56d4396868e433b3234fc1b3fdef03144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T10:55:13.686 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:13 smithi144 bash[68263]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1 2024-09-10T10:55:14.028 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:13 smithi144 podman[68277]: 2024-09-10 10:55:13.698306936 +0000 UTC m=+0.644308917 container remove 6d21faab35c565a9efed3df44c7793f56d4396868e433b3234fc1b3fdef03144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:55:14.294 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:14 smithi144 podman[68370]: 2024-09-10 10:55:13.930632509 +0000 UTC m=+0.016054927 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:14.294 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:14 smithi144 podman[68370]: 2024-09-10 10:55:14.061087877 +0000 UTC m=+0.146510296 container create c497f4ad55788de4277732efa81272ec83da55c21896a8d5c041dc684b9ea7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:55:14.294 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:14 smithi144 podman[68370]: 2024-09-10 10:55:14.293817736 +0000 UTC m=+0.379240154 container init c497f4ad55788de4277732efa81272ec83da55c21896a8d5c041dc684b9ea7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:14.686 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:14 smithi144 podman[68370]: 2024-09-10 10:55:14.297587657 +0000 UTC m=+0.383010076 container start c497f4ad55788de4277732efa81272ec83da55c21896a8d5c041dc684b9ea7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:55:14.686 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:14 smithi144 podman[68370]: 2024-09-10 10:55:14.393960922 +0000 UTC m=+0.479383357 container attach c497f4ad55788de4277732efa81272ec83da55c21896a8d5c041dc684b9ea7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:55:14.686 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:14 smithi144 podman[68370]: 2024-09-10 10:55:14.484192496 +0000 UTC m=+0.569614915 container died c497f4ad55788de4277732efa81272ec83da55c21896a8d5c041dc684b9ea7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:55:15.109 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 10:55:15 smithi144 podman[68393]: 2024-09-10 10:55:15.106331985 +0000 UTC m=+0.616311392 container remove c497f4ad55788de4277732efa81272ec83da55c21896a8d5c041dc684b9ea7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T10:55:15.127 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T10:55:15.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:15.171 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T10:55:15.171 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T10:55:15.171 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.2 2024-09-10T10:55:15.486 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:15 smithi178 systemd[1]: Stopping Ceph osd.2 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:15.925 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2[39764]: 2024-09-10T10:55:15.484+0000 7fd650fe8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:15.925 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2[39764]: 2024-09-10T10:55:15.484+0000 7fd650fe8640 -1 osd.2 93 *** Got signal Terminated *** 2024-09-10T10:55:15.925 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2[39764]: 2024-09-10T10:55:15.484+0000 7fd650fe8640 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:21.303 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:20 smithi178 podman[69265]: 2024-09-10 10:55:20.955269289 +0000 UTC m=+5.588809379 container died b779649f7e77e004ece19766880d63de238787193eceeeef7a9a9d8d3049a462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T10:55:21.654 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:21 smithi178 podman[69265]: 2024-09-10 10:55:21.301933704 +0000 UTC m=+5.935473784 container cleanup b779649f7e77e004ece19766880d63de238787193eceeeef7a9a9d8d3049a462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:55:21.654 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:21 smithi178 bash[69265]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2 2024-09-10T10:55:21.925 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:21 smithi178 podman[69280]: 2024-09-10 10:55:21.653877182 +0000 UTC m=+0.700789631 container remove b779649f7e77e004ece19766880d63de238787193eceeeef7a9a9d8d3049a462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T10:55:22.267 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:21 smithi178 podman[69369]: 2024-09-10 10:55:21.868052392 +0000 UTC m=+0.016202041 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:22.267 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:21 smithi178 podman[69369]: 2024-09-10 10:55:21.988357242 +0000 UTC m=+0.136506873 container create 645bdbaf55a41d8b816dc27f9d56a796f3844d5d01dc35f1ac2a56b905b36db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:22.267 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:22 smithi178 podman[69369]: 2024-09-10 10:55:22.174976693 +0000 UTC m=+0.323126343 container init 645bdbaf55a41d8b816dc27f9d56a796f3844d5d01dc35f1ac2a56b905b36db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:22.267 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:22 smithi178 podman[69369]: 2024-09-10 10:55:22.17865011 +0000 UTC m=+0.326799745 container start 645bdbaf55a41d8b816dc27f9d56a796f3844d5d01dc35f1ac2a56b905b36db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T10:55:22.268 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:22 smithi178 podman[69369]: 2024-09-10 10:55:22.266285178 +0000 UTC m=+0.414434827 container attach 645bdbaf55a41d8b816dc27f9d56a796f3844d5d01dc35f1ac2a56b905b36db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:55:22.674 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 10:55:22 smithi178 podman[69369]: 2024-09-10 10:55:22.425125159 +0000 UTC m=+0.573274792 container died 645bdbaf55a41d8b816dc27f9d56a796f3844d5d01dc35f1ac2a56b905b36db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T10:55:23.011 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T10:55:23.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:23.050 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T10:55:23.050 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T10:55:23.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.3 2024-09-10T10:55:23.390 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:23 smithi143 systemd[1]: Stopping Ceph osd.3 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:23.800 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:23 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:55:23.388+0000 7f23c7725640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:23.801 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:23 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:55:23.388+0000 7f23c7725640 -1 osd.3 93 *** Got signal Terminated *** 2024-09-10T10:55:23.801 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:23 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:55:23.388+0000 7f23c7725640 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:26.551 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:26 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:55:26.110+0000 7f23c3d3b640 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:02.200293+0000 front 2024-09-10T10:55:02.200407+0000 (oldest deadline 2024-09-10T10:55:25.700496+0000) 2024-09-10T10:55:27.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:26.990+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:27.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:26.822+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:27.301 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:55:27.154+0000 7f23c3d3b640 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:02.200293+0000 front 2024-09-10T10:55:02.200407+0000 (oldest deadline 2024-09-10T10:55:25.700496+0000) 2024-09-10T10:55:27.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:26 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:26.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:27.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:26 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:26.938+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:28.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:28.010+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:28.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:27.827+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:28.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:27 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:27.910+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:28.300 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3[51875]: 2024-09-10T10:55:28.104+0000 7f23c3d3b640 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:02.200293+0000 front 2024-09-10T10:55:02.200407+0000 (oldest deadline 2024-09-10T10:55:25.700496+0000) 2024-09-10T10:55:28.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:27 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:27.964+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:29.051 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:28 smithi143 podman[158120]: 2024-09-10 10:55:28.778971933 +0000 UTC m=+5.529791274 container died 6d51633059cf4f6c23df8224e14f83c789e5dc61b9fb4ba431b818bd1405f05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T10:55:29.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:28 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:28.960+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:29.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:28.994+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:29.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:28.810+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:29.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:28 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:28.893+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:29.551 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:29 smithi143 podman[158120]: 2024-09-10 10:55:29.255992696 +0000 UTC m=+6.006812055 container cleanup 6d51633059cf4f6c23df8224e14f83c789e5dc61b9fb4ba431b818bd1405f05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD) 2024-09-10T10:55:29.551 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:29 smithi143 bash[158120]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3 2024-09-10T10:55:29.909 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:29 smithi143 podman[158139]: 2024-09-10 10:55:29.557619105 +0000 UTC m=+0.772445940 container remove 6d51633059cf4f6c23df8224e14f83c789e5dc61b9fb4ba431b818bd1405f05e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T10:55:30.170 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:29 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:29.976+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:30.170 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:29 smithi143 podman[158228]: 2024-09-10 10:55:29.908453787 +0000 UTC m=+0.107979965 container create a22b8e1e6638e1a8c12941100e06786e4d54409db938f7dfe554fd8e1db964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:55:30.170 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:29 smithi143 podman[158228]: 2024-09-10 10:55:29.816144901 +0000 UTC m=+0.015671078 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:30.170 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:30 smithi143 podman[158228]: 2024-09-10 10:55:30.169736548 +0000 UTC m=+0.369262725 container init a22b8e1e6638e1a8c12941100e06786e4d54409db938f7dfe554fd8e1db964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T10:55:30.174 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:30.011+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:30.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:29.803+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:30.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:29 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:29.891+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:30.551 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:30 smithi143 podman[158228]: 2024-09-10 10:55:30.173106712 +0000 UTC m=+0.372632889 container start a22b8e1e6638e1a8c12941100e06786e4d54409db938f7dfe554fd8e1db964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T10:55:30.551 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:30 smithi143 podman[158228]: 2024-09-10 10:55:30.286686714 +0000 UTC m=+0.486212892 container attach a22b8e1e6638e1a8c12941100e06786e4d54409db938f7dfe554fd8e1db964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T10:55:30.551 INFO:journalctl@ceph.osd.3.smithi143.stdout:Sep 10 10:55:30 smithi143 podman[158228]: 2024-09-10 10:55:30.355442269 +0000 UTC m=+0.554968447 container died a22b8e1e6638e1a8c12941100e06786e4d54409db938f7dfe554fd8e1db964e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T10:55:30.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T10:55:31.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:31.021 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T10:55:31.021 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T10:55:31.021 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.4 2024-09-10T10:55:31.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:31 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:31.012+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:31.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:30.773+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:31.186 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:31 smithi144 systemd[1]: Stopping Ceph osd.4 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:31.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:30 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:30.917+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:31.300 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:30 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:30.984+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:31.686 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:31 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4[45271]: 2024-09-10T10:55:31.358+0000 7fda60216640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:31.686 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:31 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4[45271]: 2024-09-10T10:55:31.358+0000 7fda60216640 -1 osd.4 93 *** Got signal Terminated *** 2024-09-10T10:55:31.686 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:31 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4[45271]: 2024-09-10T10:55:31.358+0000 7fda60216640 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:32.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:31 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:31.964+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:32.202 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:31 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:31.763+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:32.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:32 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:32.013+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:32.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:31 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:31.936+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:33.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:32 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:32.963+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:33.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:32 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:32.797+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:33.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:32 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:32.905+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:33.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:33 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:33.026+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:34.004 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:33 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:33.754+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:34.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:33 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:33.931+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:34.295 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:34 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:34.002+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:34.300 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:34.048+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:34.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:34 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:34.217+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:34.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:34 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:34.280+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:35.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:35 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:35.041+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:35.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:34 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:34.791+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:35.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:34 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:34.929+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:35.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:34 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:34.929+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:35.551 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:35 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:35.055+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:35.551 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:35 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:35.219+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:35.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:35 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:35.270+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:36.083 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:35 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:35.788+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:36.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:35 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:35.921+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:36.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:35 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:35.921+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:36.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:36 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:36.229+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:36.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:36 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:36.030+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:36.424 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:36 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:36.082+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:36.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:36 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:36.261+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:37.051 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:36 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:36.751+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:37.059 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:36 smithi144 podman[68485]: 2024-09-10 10:55:36.76297352 +0000 UTC m=+5.542392250 container died 34e13b5574c33e33422c346f888b94636d4a15a8605b6f34b62dad2a3841f60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:55:37.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:36 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:36.910+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:37.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:36 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:36.910+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:37.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:37 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:37.188+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:37.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:37 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:37.041+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:37.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:37 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:37.041+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:37.418 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:37 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:37.240+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:37.418 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:37 smithi144 podman[68485]: 2024-09-10 10:55:37.158196571 +0000 UTC m=+5.937615290 container cleanup 34e13b5574c33e33422c346f888b94636d4a15a8605b6f34b62dad2a3841f60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:55:37.419 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:37 smithi144 bash[68485]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4 2024-09-10T10:55:37.424 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:37 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:37.049+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:37.686 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:37 smithi144 podman[68503]: 2024-09-10 10:55:37.417788734 +0000 UTC m=+0.647582194 container remove 34e13b5574c33e33422c346f888b94636d4a15a8605b6f34b62dad2a3841f60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T10:55:37.924 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:37 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:37.497+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:37.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:37 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:37.735+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:38.025 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:37 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:37.896+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:38.025 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:37 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:37.896+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:38.025 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:37 smithi144 podman[68596]: 2024-09-10 10:55:37.674720164 +0000 UTC m=+0.015914589 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:38.025 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:37 smithi144 podman[68596]: 2024-09-10 10:55:37.802040657 +0000 UTC m=+0.143235077 container create 403d1ff4be9f06621a5cbd6c4f3df1b54d5481444070997c1a8b079eac7861a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T10:55:38.025 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:38 smithi144 podman[68596]: 2024-09-10 10:55:38.021087212 +0000 UTC m=+0.362281634 container init 403d1ff4be9f06621a5cbd6c4f3df1b54d5481444070997c1a8b079eac7861a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T10:55:38.025 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:38 smithi144 podman[68596]: 2024-09-10 10:55:38.024500266 +0000 UTC m=+0.365694686 container start 403d1ff4be9f06621a5cbd6c4f3df1b54d5481444070997c1a8b079eac7861a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T10:55:38.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:38 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:38.209+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:38.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:38 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:38.009+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:38.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:38 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:38.009+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:38.424 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:38 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:38.021+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:38.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:38 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:38.227+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:38.436 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:38 smithi144 podman[68596]: 2024-09-10 10:55:38.112963443 +0000 UTC m=+0.454157876 container attach 403d1ff4be9f06621a5cbd6c4f3df1b54d5481444070997c1a8b079eac7861a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:55:38.436 INFO:journalctl@ceph.osd.4.smithi144.stdout:Sep 10 10:55:38 smithi144 podman[68596]: 2024-09-10 10:55:38.210947744 +0000 UTC m=+0.552142164 container died 403d1ff4be9f06621a5cbd6c4f3df1b54d5481444070997c1a8b079eac7861a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T10:55:38.913 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T10:55:38.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:38 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:38.485+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:38.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:38 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:38.689+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:38.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:38.956 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T10:55:38.956 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T10:55:38.956 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.5 2024-09-10T10:55:39.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:38 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:38.937+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:39.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:38 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:38.937+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:39.293 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:38 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:38.994+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:39.293 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:38 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:38.994+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.548532+0000 front 2024-09-10T10:55:12.548411+0000 (oldest deadline 2024-09-10T10:55:38.448397+0000) 2024-09-10T10:55:39.293 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:39 smithi178 systemd[1]: Stopping Ceph osd.5 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:39.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:39 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:39.171+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:39.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:39 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:39.012+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:39.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:39 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:39.012+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:39.675 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:39 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:39.456+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:39.675 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:39 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:39.291+0000 7f5868b1e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:39.675 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:39 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:39.291+0000 7f5868b1e640 -1 osd.5 93 *** Got signal Terminated *** 2024-09-10T10:55:39.675 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:39 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:39.291+0000 7f5868b1e640 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:39.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:39 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:39.230+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:40.016 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:39 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:39.732+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:40.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:40.197+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:40.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:39 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:39.986+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:40.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:39 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:39.986+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:40.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:40 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:40.014+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:40.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:40 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:40.014+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.548532+0000 front 2024-09-10T10:55:12.548411+0000 (oldest deadline 2024-09-10T10:55:38.448397+0000) 2024-09-10T10:55:40.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:40 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:40.192+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:40.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:39 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:39.980+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:40.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:39 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:39.980+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:40.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:40 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:40.500+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:40.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:40 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:40.684+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:41.300 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:41 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:41.202+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:41.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:40.983+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:41.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:40 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:40.983+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:41.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:40 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:40.967+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:41.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:40 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:40.967+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.548532+0000 front 2024-09-10T10:55:12.548411+0000 (oldest deadline 2024-09-10T10:55:38.448397+0000) 2024-09-10T10:55:41.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:41 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:41.221+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:41.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:40 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:40.988+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:41.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:40 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:40.988+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:41.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:41 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:41.490+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:41.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:41 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:41.710+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:41.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:41 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:41.710+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:42.300 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:42.191+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:42.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:42.010+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:42.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:42.010+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:42.424 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:42.003+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:42.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:42.003+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.548532+0000 front 2024-09-10T10:55:12.548411+0000 (oldest deadline 2024-09-10T10:55:38.448397+0000) 2024-09-10T10:55:42.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:42.198+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:42.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:42.198+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:42.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:41 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:41.961+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:42.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:41 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:41.961+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:42.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:41 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:41.961+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:42.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:42.470+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:42.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:42.661+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:42.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:42 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:42.661+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:43.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:43.193+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:43.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:43.193+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:43.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:43.002+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:43.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:43.002+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:43.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:43.002+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:43.424 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:43.028+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:43.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:43.028+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.548532+0000 front 2024-09-10T10:55:12.548411+0000 (oldest deadline 2024-09-10T10:55:38.448397+0000) 2024-09-10T10:55:43.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:43.028+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:20.050062+0000 front 2024-09-10T10:55:20.050159+0000 (oldest deadline 2024-09-10T10:55:42.950113+0000) 2024-09-10T10:55:43.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:43 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:43.178+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:43.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:43 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:43.178+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:43.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:42.938+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:43.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:42.938+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:43.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:42 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:42.938+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:43.924 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:43.515+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:43.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:43.515+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:43.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:43.699+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:43.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:43 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:43.699+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:44.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:43 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:43.930+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:44.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:43 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:43.930+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:44.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:43 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:43.930+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:44.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:44 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:44.164+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:44.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:44 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:44.164+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:44.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:43.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:44.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:43.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:44.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:43 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:43.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:44.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:44.027+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.546815+0000 front 2024-09-10T10:55:01.546653+0000 (oldest deadline 2024-09-10T10:55:26.147114+0000) 2024-09-10T10:55:44.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:44.027+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.548532+0000 front 2024-09-10T10:55:12.548411+0000 (oldest deadline 2024-09-10T10:55:38.448397+0000) 2024-09-10T10:55:44.425 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5[46616]: 2024-09-10T10:55:44.027+0000 7f5865134640 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:20.050062+0000 front 2024-09-10T10:55:20.050159+0000 (oldest deadline 2024-09-10T10:55:42.950113+0000) 2024-09-10T10:55:44.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:44 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:44.186+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:44.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:44 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:44.186+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:44.754 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:44.650+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:44.755 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:44.650+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:44.755 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:44.489+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:44.755 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:44 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:44.489+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:45.042 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:44 smithi178 podman[69487]: 2024-09-10 10:55:44.75406775 +0000 UTC m=+5.604010866 container died 55412e95697e1e43a1b55c0716476d9e82c3c80f5cc1062827b987bf5e561ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T10:55:45.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:44 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:44.916+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:45.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:44 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:44.916+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:45.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:44 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:44.916+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:45.300 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:45 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:45.145+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:45.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:45 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:45.145+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:45.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:44 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:44.972+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:45.304 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:44 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:44.972+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:45.304 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:44 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:44.972+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:45.305 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69487]: 2024-09-10 10:55:45.040931757 +0000 UTC m=+5.890874872 container cleanup 55412e95697e1e43a1b55c0716476d9e82c3c80f5cc1062827b987bf5e561ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:55:45.305 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 bash[69487]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5 2024-09-10T10:55:45.644 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69502]: 2024-09-10 10:55:45.300924206 +0000 UTC m=+0.544301429 container remove 55412e95697e1e43a1b55c0716476d9e82c3c80f5cc1062827b987bf5e561ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:55:45.644 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:45 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:45.496+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:45.644 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:45 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:45.496+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:45.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:45.188+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:45.686 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:45.188+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:45.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:45 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:45.683+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:45.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:45 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:45.683+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:45.897 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69590]: 2024-09-10 10:55:45.643533987 +0000 UTC m=+0.104301357 container create fc915df20949a23f043a7eacc6feffc55dbbc3a869598d6cb40dcd7f37eb6506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:55:45.897 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69590]: 2024-09-10 10:55:45.555233485 +0000 UTC m=+0.016000857 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:45.897 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69590]: 2024-09-10 10:55:45.896339374 +0000 UTC m=+0.357106745 container init fc915df20949a23f043a7eacc6feffc55dbbc3a869598d6cb40dcd7f37eb6506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-10T10:55:46.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69590]: 2024-09-10 10:55:45.899716537 +0000 UTC m=+0.360483903 container start fc915df20949a23f043a7eacc6feffc55dbbc3a869598d6cb40dcd7f37eb6506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T10:55:46.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:45 smithi178 podman[69590]: 2024-09-10 10:55:45.988276514 +0000 UTC m=+0.449043882 container attach fc915df20949a23f043a7eacc6feffc55dbbc3a869598d6cb40dcd7f37eb6506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T10:55:46.175 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 10:55:46 smithi178 podman[69590]: 2024-09-10 10:55:46.08210574 +0000 UTC m=+0.542873115 container died fc915df20949a23f043a7eacc6feffc55dbbc3a869598d6cb40dcd7f37eb6506 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:55:46.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:46 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:46.118+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:46.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:46 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:46.118+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:46.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:46 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:46.017+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:46.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:46 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:46.017+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:46.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:46 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:46.017+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:46.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:46 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:46.156+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:46.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:46 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:46.156+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:46.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:45.967+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:46.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:45.967+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:46.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:45 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:45.967+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:46.693 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:46 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:46.680+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:46.693 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:46 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:46.680+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:46.694 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:46 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:46.499+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:46.694 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:46 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:46.499+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:46.713 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T10:55:46.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:46.752 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T10:55:46.752 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T10:55:46.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.6 2024-09-10T10:55:47.051 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:46 smithi143 systemd[1]: Stopping Ceph osd.6 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:47.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:47.014+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:47.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:47.014+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:47.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:47.014+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:47.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:47 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:47.115+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:47.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:47 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:47.115+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:47.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:47 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:47.013+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:47.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:47 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:47.013+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:47.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:47 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:47.013+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:47.552 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:47.075+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:47.574 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:47.075+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:47.575 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:47.095+0000 7fd10fb1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:47.575 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:47.095+0000 7fd10fb1a640 -1 osd.6 93 *** Got signal Terminated *** 2024-09-10T10:55:47.575 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:47.095+0000 7fd10fb1a640 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:47.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:47 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:47.489+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:47.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:47 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:47.489+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:47.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:47 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:47.690+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:47.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:47 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:47.690+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:48.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:48 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:48.072+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:48.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:48 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:48.072+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:48.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:47.992+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:48.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:47.992+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:48.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:47.992+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:48.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:48 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:48.073+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:48.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:48 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:48.073+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:48.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:48 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:48.029+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:48.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:48 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:48.029+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:48.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:48 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:48.029+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:48.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:48 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:48.497+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:48.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:48 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:48.497+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:48.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:48 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:48.723+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:48.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:48 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:48.723+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:49.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:49 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:49.063+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:49.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:49 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:49.063+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:49.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:48 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:48.979+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:49.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:48 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:48.979+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:49.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:48 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:48.979+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:49.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:49 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:49.041+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:49.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:49 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:49.041+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:49.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:49 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:49.049+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:49.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:49 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:49.049+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:49.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:49 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:49.049+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:49.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:49 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:49.539+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:49.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:49 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:49.539+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:49.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:49 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:49.695+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:49.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:49 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:49.695+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:50.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:50 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:50.080+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:50.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:50 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:50.080+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:50.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:50 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:50.007+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:50.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:50 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:50.007+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:50.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:50 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:50.007+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:50.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:50.002+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:50.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:50.002+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:50.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:50.046+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:50.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:50.046+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:50.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:50.046+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:50.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:50 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:50.520+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:50.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:50 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:50.520+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:50.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:50 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:50.682+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:50.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:50 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:50.682+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:51.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:51.049+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:51.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:51.049+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:51.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:51.049+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.040963+0000 front 2024-09-10T10:55:25.040994+0000 (oldest deadline 2024-09-10T10:55:50.241499+0000) 2024-09-10T10:55:51.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:51.049+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:51.302 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:51.049+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:51.302 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:51 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:51.049+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:51.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:50.963+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:51.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:50.963+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:51.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:50.963+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:51.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.011+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:51.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.011+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:51.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.011+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:51.826 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:51 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:51.721+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:51.826 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:51 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:51.721+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:51.826 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:51 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:51.550+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:51.827 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:51 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:51.550+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:51.827 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:51 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:51.550+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:52.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:52.003+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:07.638250+0000 front 2024-09-10T10:55:07.637990+0000 (oldest deadline 2024-09-10T10:55:33.538129+0000) 2024-09-10T10:55:52.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:52.003+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.039259+0000 front 2024-09-10T10:55:17.039170+0000 (oldest deadline 2024-09-10T10:55:42.939243+0000) 2024-09-10T10:55:52.301 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6[58496]: 2024-09-10T10:55:52.003+0000 7fd10c130640 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.040963+0000 front 2024-09-10T10:55:25.040994+0000 (oldest deadline 2024-09-10T10:55:50.241499+0000) 2024-09-10T10:55:52.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:52.017+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:52.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:52.017+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:52.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:52.017+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:52.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:51.965+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:52.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:51.965+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:52.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:51.965+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:52.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.987+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:52.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.987+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:52.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.987+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:52.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:51 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:51.987+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:52.761 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:52 smithi143 podman[158331]: 2024-09-10 10:55:52.472711538 +0000 UTC m=+5.522065886 container died 19e91a55d4437e03e47eb6bc3b2f84045112afe0112f84a4328897645d26db29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T10:55:52.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:52 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:52.733+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:52.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:52 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:52.733+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:52.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:52 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:52.597+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:52.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:52 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:52.597+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:52.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:52 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:52.597+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:53.050 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:52 smithi143 podman[158331]: 2024-09-10 10:55:52.859668812 +0000 UTC m=+5.909023159 container cleanup 19e91a55d4437e03e47eb6bc3b2f84045112afe0112f84a4328897645d26db29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T10:55:53.051 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:52 smithi143 bash[158331]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6 2024-09-10T10:55:53.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:52.980+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:53.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:52.980+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:53.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:52 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:52.980+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:53.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:53 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:53.007+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:53.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:53 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:53.007+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:53.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:53 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:53.007+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:53.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:52 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:52.988+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:53.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:52 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:52.988+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:53.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:52 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:52.988+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:53.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:52 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:52.988+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:53.472 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158347]: 2024-09-10 10:55:53.144782605 +0000 UTC m=+0.670168783 container remove 19e91a55d4437e03e47eb6bc3b2f84045112afe0112f84a4328897645d26db29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:55:53.740 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158437]: 2024-09-10 10:55:53.374150797 +0000 UTC m=+0.016172114 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:55:53.740 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158437]: 2024-09-10 10:55:53.495659953 +0000 UTC m=+0.137681298 container create 922a93492897faac9c782c68d96a300ba68eee348c7c72bded9139c2840d13a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T10:55:53.740 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158437]: 2024-09-10 10:55:53.739992429 +0000 UTC m=+0.382013731 container init 922a93492897faac9c782c68d96a300ba68eee348c7c72bded9139c2840d13a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T10:55:53.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:53.599+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:53.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:53.599+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:53.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:53.599+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:53.926 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:53.721+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:53.926 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:53 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:53.721+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:54.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:53 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:53.969+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:54.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:53 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:53.969+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:54.051 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:53 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:53.969+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:54.051 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158437]: 2024-09-10 10:55:53.743428334 +0000 UTC m=+0.385449644 container start 922a93492897faac9c782c68d96a300ba68eee348c7c72bded9139c2840d13a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:55:54.051 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158437]: 2024-09-10 10:55:53.806800273 +0000 UTC m=+0.448821590 container attach 922a93492897faac9c782c68d96a300ba68eee348c7c72bded9139c2840d13a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:55:54.051 INFO:journalctl@ceph.osd.6.smithi143.stdout:Sep 10 10:55:53 smithi143 podman[158437]: 2024-09-10 10:55:53.932853692 +0000 UTC m=+0.574875009 container died 922a93492897faac9c782c68d96a300ba68eee348c7c72bded9139c2840d13a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:55:54.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:53 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:53.993+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:54.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:53 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:53.993+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:54.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:53 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:53.993+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:54.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:54.028+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:54.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:54.028+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:54.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:54.028+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:54.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:54.028+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:54.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T10:55:54.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:55:54.687 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T10:55:54.688 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T10:55:54.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.7 2024-09-10T10:55:54.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:54.744+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:54.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:54.744+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:54.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:54.744+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:55:54.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:54.580+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:54.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:54.580+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:54.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:54 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:54.580+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:55.003 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:54 smithi144 systemd[1]: Stopping Ceph osd.7 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:55:55.003 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:54.952+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:55.003 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:54.952+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:55.004 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:54 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:54.952+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:55.004 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:55.001+0000 7f830ed77640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:55:55.004 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:55.001+0000 7f830ed77640 -1 osd.7 93 *** Got signal Terminated *** 2024-09-10T10:55:55.004 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:55.001+0000 7f830ed77640 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:55:55.300 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:54.967+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:55.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:54.967+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:55.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:54.967+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:55.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:54 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:54.967+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:55:55.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:55.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:55.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:55.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:55.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:55.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:55.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:55.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:55.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:55.743+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:55.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:55.743+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:55.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:55.743+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:55:55.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:55.621+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:55.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:55.621+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:55.926 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:55 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:55.621+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:56.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:55.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:56.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:55.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:56.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:55.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:56.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:55 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:55.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:55:56.337 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:55.959+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:56.337 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:55.959+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:56.337 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:55.959+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:56.337 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:56.012+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:56.338 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:56.012+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:56.338 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:56.012+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:56.338 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:56.012+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:56.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:56.761+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:56.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:56.761+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:56.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:56.761+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:55:56.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:56.616+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:56.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:56.616+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:56.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:56 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:56.616+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:57.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:56 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:56.952+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:57.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:56 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:56.952+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:57.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:56 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:56.952+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:57.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:56 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:56.952+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:55:57.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:56.992+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:56.992+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:56 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:56.992+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:57.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:57.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:57.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:57.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:57.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:57 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:57.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:55:57.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:57.766+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:57.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:57.766+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:57.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:57.766+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:55:57.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:57.575+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:57.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:57.575+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:57.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:57 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:57.575+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:58.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:57.994+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:58.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:57.994+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:58.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:57.994+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:58.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:57 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:57.994+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:55:58.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:58.032+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:58.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:58.032+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:58.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:58.032+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:58.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:58.039+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:58.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:58.039+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:58.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:58.039+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:58.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:58.039+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:58.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:58 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:58.039+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:55:58.808 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:58.554+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:58.808 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:58.554+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:58.808 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:58.554+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:58.808 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:58.554+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:55:59.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:58.805+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:55:59.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:58.805+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:55:59.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:58 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:58.805+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:55:59.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:59.028+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:55:59.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:59.028+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:55:59.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:59.028+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:55:59.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:55:59 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:55:59.028+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:55:59.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:59.036+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:12.316675+0000 front 2024-09-10T10:55:12.316696+0000 (oldest deadline 2024-09-10T10:55:33.416950+0000) 2024-09-10T10:55:59.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:59.036+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.618561+0000 front 2024-09-10T10:55:18.618576+0000 (oldest deadline 2024-09-10T10:55:42.118607+0000) 2024-09-10T10:55:59.436 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7[52031]: 2024-09-10T10:55:59.036+0000 7f830b38d640 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.320353+0000 front 2024-09-10T10:55:27.320267+0000 (oldest deadline 2024-09-10T10:55:50.220487+0000) 2024-09-10T10:55:59.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:59.053+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:55:59.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:59.053+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:55:59.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:59.053+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:55:59.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:59.053+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:55:59.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:55:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:55:59.053+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:55:59.817 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:59.540+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:55:59.818 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:59.540+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:55:59.818 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:59.540+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:55:59.818 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:55:59.540+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:00.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:59.816+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:00.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:59.816+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:00.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:59.816+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:00.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:55:59 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:55:59.816+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:00.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.009+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:00.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.009+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:00.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.009+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:00.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.009+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:00.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:00.020+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:00.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:00.020+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:00.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:00.020+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:00.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:00.020+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:00.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:00 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:00.020+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:00.835 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:00 smithi144 podman[68699]: 2024-09-10 10:56:00.489991602 +0000 UTC m=+5.606897242 container died 4d78244f427ca682ff1a0bd784bafd0f9c8c7172a05da7f34977fc9e797414cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T10:56:00.845 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:00.579+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:00.846 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:00.579+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:00.846 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:00.579+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:00.846 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:00.579+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:01.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:00.844+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:01.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:00.844+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:01.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:00.844+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:01.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:00 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:00.844+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:01.186 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:00 smithi144 podman[68699]: 2024-09-10 10:56:00.835029954 +0000 UTC m=+5.951935594 container cleanup 4d78244f427ca682ff1a0bd784bafd0f9c8c7172a05da7f34977fc9e797414cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:56:01.186 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:00 smithi144 bash[68699]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7 2024-09-10T10:56:01.186 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68728]: 2024-09-10 10:56:01.028364797 +0000 UTC m=+0.536615280 container remove 4d78244f427ca682ff1a0bd784bafd0f9c8c7172a05da7f34977fc9e797414cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:56:01.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:01.054+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:01.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:01.054+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:01.186 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:01.054+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:01.187 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:01.054+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:01.187 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:01 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:01.054+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:01.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.997+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:01.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.997+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:01.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.997+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:01.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.997+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:01.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:00 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:00.997+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:01.732 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68817]: 2024-09-10 10:56:01.354149639 +0000 UTC m=+0.016055485 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:56:01.732 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68817]: 2024-09-10 10:56:01.470737923 +0000 UTC m=+0.132643802 container create b83d0e4adab964ca65b821b700cf052b27b8122f2473270cb3549f652f18715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T10:56:01.732 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68817]: 2024-09-10 10:56:01.731406865 +0000 UTC m=+0.393312702 container init b83d0e4adab964ca65b821b700cf052b27b8122f2473270cb3549f652f18715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:56:01.824 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:01.538+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:01.824 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:01.538+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:01.825 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:01.538+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:01.825 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:01.538+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:02.090 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68817]: 2024-09-10 10:56:01.735314734 +0000 UTC m=+0.397220578 container start b83d0e4adab964ca65b821b700cf052b27b8122f2473270cb3549f652f18715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:56:02.090 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68817]: 2024-09-10 10:56:01.831456776 +0000 UTC m=+0.493362627 container attach b83d0e4adab964ca65b821b700cf052b27b8122f2473270cb3549f652f18715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T10:56:02.090 INFO:journalctl@ceph.osd.7.smithi144.stdout:Sep 10 10:56:01 smithi144 podman[68817]: 2024-09-10 10:56:01.923218644 +0000 UTC m=+0.585124489 container died b83d0e4adab964ca65b821b700cf052b27b8122f2473270cb3549f652f18715d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:56:02.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:01.823+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:02.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:01.823+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:02.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:01.823+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:02.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:01 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:01.823+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:02.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:02 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:02.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:02.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:02 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:02.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:02.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:02 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:02.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:02.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:02 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:02.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:02.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:02 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:02.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:02.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:02 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:02.088+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:02.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:02 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:02.088+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:02.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:02 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:02.088+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:02.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:02 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:02.088+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:02.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:02 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:02.088+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:02.600 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T10:56:02.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:02.638 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T10:56:02.638 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T10:56:02.638 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.8 2024-09-10T10:56:02.860 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.547+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:02.860 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.547+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:02.860 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.547+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:02.860 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.547+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:02.860 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 systemd[1]: Stopping Ceph osd.8 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:03.174 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.986+0000 7fa89a3d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:03.175 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.986+0000 7fa89a3d9640 -1 osd.8 93 *** Got signal Terminated *** 2024-09-10T10:56:03.175 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:02.986+0000 7fa89a3d9640 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:56:03.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:02.858+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:03.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:02.858+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:03.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:02.858+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:03.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:02 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:02.858+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:03.300 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:03.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:03.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:03.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:03.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.006+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:03.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:03.119+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:03.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:03.119+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:03.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:03.119+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:03.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:03.119+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:03.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:03.119+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:03.894 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:03.516+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:03.894 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:03.516+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:03.894 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:03.516+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:03.894 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:03.516+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:04.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:03.893+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:04.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:03.893+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:04.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:03.893+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:04.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:03 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:03.893+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:04.300 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.987+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:04.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.987+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:04.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.987+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:04.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.987+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:04.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:03 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:03.987+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:04.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:04 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:04.083+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:04.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:04 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:04.083+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:04.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:04 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:04.083+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:04.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:04 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:04.083+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:04.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:04 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:04.083+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:04.873 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:04.485+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:04.874 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:04.485+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:04.874 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:04.485+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:04.874 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:04.485+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:05.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:04.872+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:05.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:04.872+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:05.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:04.872+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:05.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:04 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:04.872+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:05.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:04 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:04.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:05.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:04 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:04.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:05.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:04 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:04.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:05.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:04 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:04.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:05.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:04 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:04.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:05.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:05 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:05.036+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:05.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:05 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:05.036+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:05.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:05 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:05.036+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:05.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:05 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:05.036+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:05.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:05 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:05.036+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:05.885 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:05.531+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:05.885 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:05.531+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:05.885 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:05.531+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:05.885 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:05.531+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:06.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:05.884+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:06.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:05.884+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:06.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:05.884+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:06.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:05.884+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:06.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:05 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:05.884+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:06.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:05 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:05.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:06.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:05 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:05.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:06.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:05 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:05.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:06.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:05 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:05.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:06.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:05 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:05.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:06.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:06 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:06.079+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:06.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:06 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:06.079+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:06.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:06 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:06.079+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:06.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:06 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:06.079+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:06.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:06 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:06.079+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:06.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:06 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:06.079+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:06.886 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:06.528+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:06.887 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:06.528+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:06.887 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:06.528+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:06.887 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:06.528+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:06.887 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:06.528+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:40.009709+0000 front 2024-09-10T10:55:40.009685+0000 (oldest deadline 2024-09-10T10:56:05.909690+0000) 2024-09-10T10:56:07.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:06.885+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:07.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:06.885+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:07.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:06.885+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:07.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:06.885+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:07.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:06 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:06.885+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:07.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:06 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:06.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:07.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:06 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:06.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:07.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:06 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:06.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:07.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:06 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:06.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:07.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:06 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:06.977+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:07.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:07.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:07.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:07.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:07.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:07.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.040+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:07.870 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:07.564+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.906003+0000 front 2024-09-10T10:55:11.905911+0000 (oldest deadline 2024-09-10T10:55:36.606062+0000) 2024-09-10T10:56:07.870 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:07.564+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:16.606381+0000 front 2024-09-10T10:55:16.606465+0000 (oldest deadline 2024-09-10T10:55:42.506452+0000) 2024-09-10T10:56:07.871 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:07.564+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:25.407609+0000 front 2024-09-10T10:55:25.407779+0000 (oldest deadline 2024-09-10T10:55:50.707589+0000) 2024-09-10T10:56:07.871 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:07.564+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.408666+0000 front 2024-09-10T10:55:35.408847+0000 (oldest deadline 2024-09-10T10:55:57.708675+0000) 2024-09-10T10:56:07.871 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8[53390]: 2024-09-10T10:56:07.564+0000 7fa8929e7640 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:40.009709+0000 front 2024-09-10T10:55:40.009685+0000 (oldest deadline 2024-09-10T10:56:05.909690+0000) 2024-09-10T10:56:08.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:07.869+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:08.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:07.869+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:08.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:07.869+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:08.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:07.869+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:08.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:07.869+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:08.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:07.939+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:08.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:07.939+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:08.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:07.939+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:08.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:07.939+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:08.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:07.939+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:08.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:07 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:07.939+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:08.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.996+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:08.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.996+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:08.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.996+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:08.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.996+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:08.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.996+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:08.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:07 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:07.996+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:08.656 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:08 smithi178 podman[69704]: 2024-09-10 10:56:08.364085846 +0000 UTC m=+5.517120478 container died 4cbaa114029eaf53adb287d6565fe2658e50b6574cd9b11b02c6f3471a54a08a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T10:56:08.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:08 smithi178 podman[69704]: 2024-09-10 10:56:08.745318832 +0000 UTC m=+5.898353454 container cleanup 4cbaa114029eaf53adb287d6565fe2658e50b6574cd9b11b02c6f3471a54a08a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T10:56:08.925 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:08 smithi178 bash[69704]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8 2024-09-10T10:56:08.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:08 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:08.876+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:08.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:08 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:08.876+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:08.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:08 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:08.876+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:08.926 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:08 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:08.876+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:08.926 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:08 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:08.876+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:09.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:08 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:08.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:09.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:08 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:08.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:09.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:08 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:08.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:09.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:08 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:08.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:09.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:08 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:08.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:09.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:08 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:08.954+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:09.378 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69720]: 2024-09-10 10:56:09.055473167 +0000 UTC m=+0.692860242 container remove 4cbaa114029eaf53adb287d6565fe2658e50b6574cd9b11b02c6f3471a54a08a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:56:09.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:08 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:08.954+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:09.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:08 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:08.954+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:09.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:08 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:08.954+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:09.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:08 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:08.954+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:09.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:08 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:08.954+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:09.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:08 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:08.954+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:09.675 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69806]: 2024-09-10 10:56:09.280675924 +0000 UTC m=+0.016014008 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:56:09.675 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69806]: 2024-09-10 10:56:09.381123814 +0000 UTC m=+0.116461885 container create e9002c01d38e48a8773209e8f6b10099c7c406a283d66443ad6db72da89cab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:56:09.675 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69806]: 2024-09-10 10:56:09.608977715 +0000 UTC m=+0.344315798 container init e9002c01d38e48a8773209e8f6b10099c7c406a283d66443ad6db72da89cab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T10:56:09.675 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69806]: 2024-09-10 10:56:09.612273767 +0000 UTC m=+0.347611839 container start e9002c01d38e48a8773209e8f6b10099c7c406a283d66443ad6db72da89cab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69806]: 2024-09-10 10:56:09.700509568 +0000 UTC m=+0.435847643 container attach e9002c01d38e48a8773209e8f6b10099c7c406a283d66443ad6db72da89cab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 10 10:56:09 smithi178 podman[69806]: 2024-09-10 10:56:09.806430079 +0000 UTC m=+0.541768164 container died e9002c01d38e48a8773209e8f6b10099c7c406a283d66443ad6db72da89cab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:09.843+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:09.843+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:09.843+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:09.843+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:10.163 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:09 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:09.843+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:10.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:09.956+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:10.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:09.956+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:10.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:09.956+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:10.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:09.956+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:10.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:09.956+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:10.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:09 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:09.956+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:10.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:09 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:09.979+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:10.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:09 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:09.979+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:10.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:09 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:09.979+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:10.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:09 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:09.979+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:10.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:09 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:09.979+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:10.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:09 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:09.979+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:10.489 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T10:56:10.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:10.530 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T10:56:10.530 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T10:56:10.531 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.9 2024-09-10T10:56:10.814 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 systemd[1]: Stopping Ceph osd.9 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:11.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:10 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:10.801+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:11.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:10 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:10.801+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:11.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:10 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:10.801+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:11.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:10 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:10.801+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:11.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:10 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:10.801+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.812+0000 7f88e88cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.812+0000 7f88e88cc640 -1 osd.9 93 *** Got signal Terminated *** 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.812+0000 7f88e88cc640 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.940+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.940+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.940+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.940+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.940+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:11.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:10 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:10.940+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:11.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:11 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:11.016+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:11.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:11 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:11.016+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:11.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:11 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:11.016+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:11.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:11 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:11.016+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:11.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:11 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:11.016+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:11.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:11 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:11.016+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:12.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:11.779+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:12.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:11.779+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:12.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:11.779+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:12.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:11.779+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:12.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:11.779+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:12.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:11.962+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:12.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:11.962+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:12.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:11.962+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:12.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:11.962+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:12.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:11.962+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:12.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:11 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:11.962+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:12.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:12.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:12.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:12.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:12.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:12.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.045+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:13.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:12 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:12.734+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:13.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:12 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:12.734+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:13.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:12 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:12.734+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:13.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:12 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:12.734+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:13.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:12 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:12.734+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:13.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:12 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:12.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:13.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:12 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:12.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:13.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:12 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:12.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:13.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:12 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:12.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:13.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:12 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:12.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:13.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:12 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:12.995+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:13.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.998+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:13.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.998+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:13.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.998+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:13.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.998+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:13.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.998+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:13.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:12 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:12.998+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:14.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:13 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:13.700+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:14.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:13 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:13.700+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:14.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:13 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:13.700+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:14.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:13 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:13.700+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:14.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:13 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:13.700+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:14.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:13 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:13.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:14.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:13 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:13.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:14.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:13 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:13.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:14.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:13 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:13.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:14.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:13 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:13.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:14.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:13 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:13.966+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:14.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:13 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:13.978+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:15.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:14 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:14.692+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:15.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:14 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:14.692+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:15.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:14 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:14.692+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:15.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:14 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:14.692+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:15.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:14 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:14.692+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:15.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:14.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:00.529102+0000 front 2024-09-10T10:55:00.529036+0000 (oldest deadline 2024-09-10T10:55:26.929744+0000) 2024-09-10T10:56:15.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:14.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:11.531350+0000 front 2024-09-10T10:55:11.531361+0000 (oldest deadline 2024-09-10T10:55:36.831123+0000) 2024-09-10T10:56:15.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:14.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:18.532370+0000 front 2024-09-10T10:55:18.532257+0000 (oldest deadline 2024-09-10T10:55:42.632219+0000) 2024-09-10T10:56:15.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:14.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:22.632565+0000 front 2024-09-10T10:55:22.632600+0000 (oldest deadline 2024-09-10T10:55:54.433197+0000) 2024-09-10T10:56:15.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:14.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:34.934124+0000 front 2024-09-10T10:55:34.934100+0000 (oldest deadline 2024-09-10T10:56:00.834068+0000) 2024-09-10T10:56:15.301 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:14 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9[65448]: 2024-09-10T10:56:14.993+0000 7f88e4ee2640 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:41.935147+0000 front 2024-09-10T10:55:41.935204+0000 (oldest deadline 2024-09-10T10:56:07.835196+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:15.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:14 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:14.985+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:15.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:15.665+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:15.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:15.665+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:15.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:15.665+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:15.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:15.665+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:15.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:15.665+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:15.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:15.665+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:16.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:15 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:15.946+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:16.472 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:16 smithi143 podman[158540]: 2024-09-10 10:56:16.188544378 +0000 UTC m=+5.465998955 container died 5645664645e529840d82f9e77e46cf3361efde4cd15b5a3087e7a2acfa09454a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:56:16.800 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:16 smithi143 podman[158540]: 2024-09-10 10:56:16.637791915 +0000 UTC m=+5.915246523 container cleanup 5645664645e529840d82f9e77e46cf3361efde4cd15b5a3087e7a2acfa09454a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T10:56:16.801 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:16 smithi143 bash[158540]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9 2024-09-10T10:56:17.174 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:16 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:16.678+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:17.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:16 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:16.678+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:17.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:16 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:16.678+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:17.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:16 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:16.678+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:17.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:16 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:16.678+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:17.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:16 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:16.678+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:17.257 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:16 smithi143 podman[158556]: 2024-09-10 10:56:16.922464667 +0000 UTC m=+0.733551997 container remove 5645664645e529840d82f9e77e46cf3361efde4cd15b5a3087e7a2acfa09454a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:17.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:16 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:16.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:17.551 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:17 smithi143 podman[158644]: 2024-09-10 10:56:17.158792924 +0000 UTC m=+0.015702295 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:56:17.551 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:17 smithi143 podman[158644]: 2024-09-10 10:56:17.256541705 +0000 UTC m=+0.113451067 container create a4ac36e74f7cf039e37e7aa0ff3ca176241fcd73ced6be45d4ce5ab737584ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T10:56:17.551 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:17 smithi143 podman[158644]: 2024-09-10 10:56:17.492516684 +0000 UTC m=+0.349426043 container init a4ac36e74f7cf039e37e7aa0ff3ca176241fcd73ced6be45d4ce5ab737584ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T10:56:17.551 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:17 smithi143 podman[158644]: 2024-09-10 10:56:17.495706344 +0000 UTC m=+0.352615704 container start a4ac36e74f7cf039e37e7aa0ff3ca176241fcd73ced6be45d4ce5ab737584ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T10:56:17.896 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:17 smithi143 podman[158644]: 2024-09-10 10:56:17.57241556 +0000 UTC m=+0.429324939 container attach a4ac36e74f7cf039e37e7aa0ff3ca176241fcd73ced6be45d4ce5ab737584ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T10:56:17.896 INFO:journalctl@ceph.osd.9.smithi143.stdout:Sep 10 10:56:17 smithi143 podman[158644]: 2024-09-10 10:56:17.680465911 +0000 UTC m=+0.537375280 container died a4ac36e74f7cf039e37e7aa0ff3ca176241fcd73ced6be45d4ce5ab737584ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:56:17.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:17 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:17.657+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:17.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:17 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:17.657+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:17.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:17 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:17.657+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:17.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:17 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:17.657+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:17.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:17 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:17.657+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:17.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:17 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:17.657+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:18.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T10:56:18.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:18.156 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T10:56:18.156 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T10:56:18.156 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.10 2024-09-10T10:56:18.201 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:18.201 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:18.201 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:18.201 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:18.202 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:18.202 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:18.202 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:17 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:17.935+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:18.471 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 systemd[1]: Stopping Ceph osd.10 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:18.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:18 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:18.652+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:18.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:18 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:18.652+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:18.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:18 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:18.652+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:18.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:18 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:18.652+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:18.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:18 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:18.652+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:18.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:18 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:18.652+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:18.936 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.470+0000 7fdcab7e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:18.936 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.470+0000 7fdcab7e2640 -1 osd.10 93 *** Got signal Terminated *** 2024-09-10T10:56:18.936 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.470+0000 7fdcab7e2640 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:19.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:18 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:18.977+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:20.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:19.674+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:20.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:19.674+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:20.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:19.674+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:20.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:19.674+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:20.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:19.674+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:20.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:19.674+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:20.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:20.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:20.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:20.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:20.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:20.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:20.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:19 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:19.984+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:20.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:20 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:20.627+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:20.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:20 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:20.627+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:20.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:20 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:20.627+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:20.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:20 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:20.627+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:20.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:20 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:20.627+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:20.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:20 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:20.627+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:21.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:21.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:21.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:21.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:21.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:21.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:21.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:20 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:20.970+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:21.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:21 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:21.602+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:21.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:21 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:21.602+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:21.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:21 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:21.602+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:21.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:21 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:21.602+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:21.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:21 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:21.602+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:21.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:21 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:21.602+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:22.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:22.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:22.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:22.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:22.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:22.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:22.437 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:21 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:21.953+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:22.924 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:22 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:22.557+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:22.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:22 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:22.557+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:22.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:22 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:22.557+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:22.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:22 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:22.557+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:22.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:22 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:22.557+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:22.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:22 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:22.557+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:01.958124+0000 front 2024-09-10T10:55:01.958174+0000 (oldest deadline 2024-09-10T10:55:26.058075+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-09-10T10:55:10.759160+0000 front 2024-09-10T10:55:10.759220+0000 (oldest deadline 2024-09-10T10:55:34.859368+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:17.560985+0000 front 2024-09-10T10:55:17.561023+0000 (oldest deadline 2024-09-10T10:55:41.060801+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:28.062250+0000 front 2024-09-10T10:55:28.062192+0000 (oldest deadline 2024-09-10T10:55:51.562272+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:31.562597+0000 front 2024-09-10T10:55:31.562580+0000 (oldest deadline 2024-09-10T10:55:56.862841+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:43.264404+0000 front 2024-09-10T10:55:43.264268+0000 (oldest deadline 2024-09-10T10:56:05.564206+0000) 2024-09-10T10:56:23.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:22 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10[58985]: 2024-09-10T10:56:22.962+0000 7fdca7df8640 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:50.765722+0000 front 2024-09-10T10:55:50.765637+0000 (oldest deadline 2024-09-10T10:56:13.665546+0000) 2024-09-10T10:56:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:23.527+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:23.527+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:23.527+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:23.527+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:23.527+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:23.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:23 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:23.527+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:24.320 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:24 smithi144 podman[68921]: 2024-09-10 10:56:24.008449705 +0000 UTC m=+5.661299575 container died b4e4235ed9b017bc4797412bffba0734f594243bd8aaa67fa49607a161634d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T10:56:24.320 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:24 smithi144 podman[68921]: 2024-09-10 10:56:24.319642519 +0000 UTC m=+5.972492395 container cleanup b4e4235ed9b017bc4797412bffba0734f594243bd8aaa67fa49607a161634d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T10:56:24.686 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:24 smithi144 bash[68921]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10 2024-09-10T10:56:24.686 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:24 smithi144 podman[68937]: 2024-09-10 10:56:24.529149614 +0000 UTC m=+0.513023705 container remove b4e4235ed9b017bc4797412bffba0734f594243bd8aaa67fa49607a161634d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T10:56:24.924 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:24 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:24.560+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:24.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:24 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:24.560+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:24.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:24 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:24.560+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:24.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:24 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:24.560+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:24.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:24 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:24.560+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:24.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:24 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:24.560+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:25.166 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:24 smithi144 podman[69027]: 2024-09-10 10:56:24.854787508 +0000 UTC m=+0.110854879 container create af3936a499121e25d80d44c08f3447af89b6f954abecc9a32a16cf01a99c4a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T10:56:25.166 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:24 smithi144 podman[69027]: 2024-09-10 10:56:24.76013085 +0000 UTC m=+0.016198188 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:56:25.166 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:25 smithi144 podman[69027]: 2024-09-10 10:56:25.082259273 +0000 UTC m=+0.338326598 container init af3936a499121e25d80d44c08f3447af89b6f954abecc9a32a16cf01a99c4a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T10:56:25.166 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:25 smithi144 podman[69027]: 2024-09-10 10:56:25.086287531 +0000 UTC m=+0.342354856 container start af3936a499121e25d80d44c08f3447af89b6f954abecc9a32a16cf01a99c4a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:56:25.166 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:25 smithi144 podman[69027]: 2024-09-10 10:56:25.16540971 +0000 UTC m=+0.421477038 container attach af3936a499121e25d80d44c08f3447af89b6f954abecc9a32a16cf01a99c4a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T10:56:25.436 INFO:journalctl@ceph.osd.10.smithi144.stdout:Sep 10 10:56:25 smithi144 podman[69027]: 2024-09-10 10:56:25.271162756 +0000 UTC m=+0.527230092 container died af3936a499121e25d80d44c08f3447af89b6f954abecc9a32a16cf01a99c4a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:56:25.829 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T10:56:25.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:25.868 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T10:56:25.868 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T10:56:25.868 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.11 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:25.897 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:25.561+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-09-10T10:55:55.451050+0000 front 2024-09-10T10:55:55.451033+0000 (oldest deadline 2024-09-10T10:56:25.451226+0000) 2024-09-10T10:56:26.174 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:25 smithi178 systemd[1]: Stopping Ceph osd.11 for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:26.555 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.181+0000 7f78f0dad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:26.555 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.181+0000 7f78f0dad640 -1 osd.11 93 *** Got signal Terminated *** 2024-09-10T10:56:26.555 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.181+0000 7f78f0dad640 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:56:26.555 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:26.555 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:26.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:26.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:26.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:26.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:26.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:26 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:26.553+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-09-10T10:55:55.451050+0000 front 2024-09-10T10:55:55.451033+0000 (oldest deadline 2024-09-10T10:56:25.451226+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:27.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:27 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:27.554+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-09-10T10:55:55.451050+0000 front 2024-09-10T10:55:55.451033+0000 (oldest deadline 2024-09-10T10:56:25.451226+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:28.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:28 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:28.507+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-09-10T10:55:55.451050+0000 front 2024-09-10T10:55:55.451033+0000 (oldest deadline 2024-09-10T10:56:25.451226+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-09-10T10:55:55.451050+0000 front 2024-09-10T10:55:55.451033+0000 (oldest deadline 2024-09-10T10:56:25.451226+0000) 2024-09-10T10:56:29.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:29 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:29.459+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-09-10T10:56:05.951988+0000 front 2024-09-10T10:56:05.952025+0000 (oldest deadline 2024-09-10T10:56:29.452026+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6806 osd.0 since back 2024-09-10T10:55:03.742522+0000 front 2024-09-10T10:55:03.742483+0000 (oldest deadline 2024-09-10T10:55:26.042458+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-10T10:55:19.444888+0000 front 2024-09-10T10:55:19.444848+0000 (oldest deadline 2024-09-10T10:55:41.145040+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6814 osd.3 since back 2024-09-10T10:55:27.046197+0000 front 2024-09-10T10:55:27.046041+0000 (oldest deadline 2024-09-10T10:55:54.046436+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-09-10T10:55:35.648079+0000 front 2024-09-10T10:55:35.648139+0000 (oldest deadline 2024-09-10T10:55:59.748137+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-10T10:55:42.648991+0000 front 2024-09-10T10:55:42.648952+0000 (oldest deadline 2024-09-10T10:56:05.549072+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.143:6822 osd.6 since back 2024-09-10T10:55:51.950467+0000 front 2024-09-10T10:55:51.950513+0000 (oldest deadline 2024-09-10T10:56:15.450460+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-09-10T10:55:55.451050+0000 front 2024-09-10T10:55:55.451033+0000 (oldest deadline 2024-09-10T10:56:25.451226+0000) 2024-09-10T10:56:30.925 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:30 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11[60023]: 2024-09-10T10:56:30.475+0000 7f78ed3c3640 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-09-10T10:56:05.951988+0000 front 2024-09-10T10:56:05.952025+0000 (oldest deadline 2024-09-10T10:56:29.452026+0000) 2024-09-10T10:56:31.866 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:31 smithi178 podman[69923]: 2024-09-10 10:56:31.575070905 +0000 UTC m=+5.515086076 container died 387c29afc3c5f8eef1ac6a56891f3da1ce1639ca479fd891cedfcb6b8993fc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:56:32.174 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:31 smithi178 podman[69923]: 2024-09-10 10:56:31.865231509 +0000 UTC m=+5.805246669 container cleanup 387c29afc3c5f8eef1ac6a56891f3da1ce1639ca479fd891cedfcb6b8993fc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T10:56:32.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:31 smithi178 bash[69923]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11 2024-09-10T10:56:32.175 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:32 smithi178 podman[69940]: 2024-09-10 10:56:32.099674655 +0000 UTC m=+0.516779019 container remove 387c29afc3c5f8eef1ac6a56891f3da1ce1639ca479fd891cedfcb6b8993fc87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T10:56:32.727 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:32 smithi178 podman[70026]: 2024-09-10 10:56:32.334878735 +0000 UTC m=+0.015827921 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:56:32.727 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:32 smithi178 podman[70026]: 2024-09-10 10:56:32.434009969 +0000 UTC m=+0.114959156 container create e7d2caca95337a3486706c8e3a1a443877a129e5bc5e4965e04f268b5445439e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T10:56:33.030 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:32 smithi178 podman[70026]: 2024-09-10 10:56:32.828477769 +0000 UTC m=+0.509426954 container init e7d2caca95337a3486706c8e3a1a443877a129e5bc5e4965e04f268b5445439e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:56:33.031 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:32 smithi178 podman[70026]: 2024-09-10 10:56:32.831790863 +0000 UTC m=+0.512740052 container start e7d2caca95337a3486706c8e3a1a443877a129e5bc5e4965e04f268b5445439e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T10:56:33.031 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:32 smithi178 podman[70026]: 2024-09-10 10:56:32.920232351 +0000 UTC m=+0.601181558 container attach e7d2caca95337a3486706c8e3a1a443877a129e5bc5e4965e04f268b5445439e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:56:33.306 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:33 smithi178 podman[70026]: 2024-09-10 10:56:33.029904867 +0000 UTC m=+0.710854060 container died e7d2caca95337a3486706c8e3a1a443877a129e5bc5e4965e04f268b5445439e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T10:56:33.557 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:33 smithi178 podman[70050]: 2024-09-10 10:56:33.540436807 +0000 UTC m=+0.502978695 container remove e7d2caca95337a3486706c8e3a1a443877a129e5bc5e4965e04f268b5445439e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:56:33.557 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:33 smithi178 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-10T10:56:33.557 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:33 smithi178 systemd[1]: Stopped Ceph osd.11 for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:56:33.557 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 10 10:56:33 smithi178 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.11.service: Consumed 58.088s CPU time. 2024-09-10T10:56:33.559 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T10:56:33.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:33.640 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T10:56:33.640 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T10:56:33.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.a 2024-09-10T10:56:33.990 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:56:33 smithi143 systemd[1]: Stopping Ceph mds.a for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:34.300 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:56:33 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a[72470]: 2024-09-10T10:56:33.989+0000 7fade1c95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:34.301 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:56:33 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a[72470]: 2024-09-10T10:56:33.989+0000 7fade1c95640 -1 mds.a *** got signal Terminated *** 2024-09-10T10:56:38.281 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:56:38 smithi143 podman[158746]: 2024-09-10 10:56:38.004993008 +0000 UTC m=+4.164293511 container died 0d481ce063579505fdb47f20c6a939cd50283611b856eee107522fe0c5c2c553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:56:38.551 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:56:38 smithi143 podman[158746]: 2024-09-10 10:56:38.35521982 +0000 UTC m=+4.514520314 container cleanup 0d481ce063579505fdb47f20c6a939cd50283611b856eee107522fe0c5c2c553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:56:38.551 INFO:journalctl@ceph.mds.a.smithi143.stdout:Sep 10 10:56:38 smithi143 bash[158746]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-a 2024-09-10T10:56:38.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T10:56:38.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:38.865 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T10:56:38.865 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T10:56:38.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.d 2024-09-10T10:56:39.301 INFO:journalctl@ceph.mds.d.smithi143.stdout:Sep 10 10:56:39 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-d[71264]: 2024-09-10T10:56:39.167+0000 7ff08c1e2640 -1 mds.d *** got signal Terminated *** 2024-09-10T10:56:41.301 INFO:journalctl@ceph.mds.d.smithi143.stdout:Sep 10 10:56:41 smithi143 podman[158868]: 2024-09-10 10:56:41.022539513 +0000 UTC m=+1.938491545 container died 171dd23dfd047d77768bdc56175db789a3fbfcd19d6e65417942a4e447027df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T10:56:41.745 INFO:journalctl@ceph.mds.d.smithi143.stdout:Sep 10 10:56:41 smithi143 podman[158868]: 2024-09-10 10:56:41.484623717 +0000 UTC m=+2.400575788 container cleanup 171dd23dfd047d77768bdc56175db789a3fbfcd19d6e65417942a4e447027df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T10:56:41.745 INFO:journalctl@ceph.mds.d.smithi143.stdout:Sep 10 10:56:41 smithi143 bash[158868]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-d 2024-09-10T10:56:41.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T10:56:41.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:41.955 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T10:56:41.955 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T10:56:41.955 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.g 2024-09-10T10:56:42.272 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:56:42 smithi143 systemd[1]: Stopping Ceph mds.g for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:42.550 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:56:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g[72138]: 2024-09-10T10:56:42.271+0000 7f9d66e57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:42.551 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:56:42 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g[72138]: 2024-09-10T10:56:42.271+0000 7f9d66e57640 -1 mds.g *** got signal Terminated *** 2024-09-10T10:56:46.298 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:56:46 smithi143 podman[158989]: 2024-09-10 10:56:46.033293763 +0000 UTC m=+3.865090992 container died b77e0014d299027f21889096d04319d05fff19d3d745ab9d76b3edd9c4df5621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:56:46.550 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:56:46 smithi143 podman[158989]: 2024-09-10 10:56:46.471630806 +0000 UTC m=+4.303428039 container cleanup b77e0014d299027f21889096d04319d05fff19d3d745ab9d76b3edd9c4df5621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T10:56:46.551 INFO:journalctl@ceph.mds.g.smithi143.stdout:Sep 10 10:56:46 smithi143 bash[158989]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-g 2024-09-10T10:56:46.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T10:56:46.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:46.874 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T10:56:46.874 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T10:56:46.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.j 2024-09-10T10:56:47.301 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:56:46 smithi143 systemd[1]: Stopping Ceph mds.j for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:47.301 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:56:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j[70865]: 2024-09-10T10:56:47.175+0000 7f533cbcf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:47.301 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:56:47 smithi143 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j[70865]: 2024-09-10T10:56:47.175+0000 7f533cbcf640 -1 mds.j *** got signal Terminated *** 2024-09-10T10:56:49.785 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:56:49 smithi143 podman[159112]: 2024-09-10 10:56:49.406236992 +0000 UTC m=+2.314399048 container died 74278ecfee44cc3740eb0f292b1d8cf9a0745018a9138081bc22a114905daaa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:56:50.036 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:56:49 smithi143 podman[159112]: 2024-09-10 10:56:49.784340669 +0000 UTC m=+2.692502722 container cleanup 74278ecfee44cc3740eb0f292b1d8cf9a0745018a9138081bc22a114905daaa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:56:50.036 INFO:journalctl@ceph.mds.j.smithi143.stdout:Sep 10 10:56:49 smithi143 bash[159112]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-j 2024-09-10T10:56:50.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T10:56:50.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:50.239 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T10:56:50.240 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T10:56:50.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.b 2024-09-10T10:56:50.611 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:56:50 smithi144 systemd[1]: Stopping Ceph mds.b for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:50.936 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:56:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b[64142]: 2024-09-10T10:56:50.609+0000 7fc6a60c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:50.936 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:56:50 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b[64142]: 2024-09-10T10:56:50.609+0000 7fc6a60c2640 -1 mds.b *** got signal Terminated *** 2024-09-10T10:56:54.311 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:56:54 smithi144 podman[69141]: 2024-09-10 10:56:54.042752252 +0000 UTC m=+3.604776564 container died e2d6791d1098e57d647f0fb295e04864742ad8af42f38a5c3391b1a21ef7b45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T10:56:54.686 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:56:54 smithi144 podman[69141]: 2024-09-10 10:56:54.418038062 +0000 UTC m=+3.980062378 container cleanup e2d6791d1098e57d647f0fb295e04864742ad8af42f38a5c3391b1a21ef7b45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:56:54.686 INFO:journalctl@ceph.mds.b.smithi144.stdout:Sep 10 10:56:54 smithi144 bash[69141]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-b 2024-09-10T10:56:54.842 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T10:56:54.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:54.877 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T10:56:54.877 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T10:56:54.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.e 2024-09-10T10:56:55.198 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:54 smithi144 systemd[1]: Stopping Ceph mds.e for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:55.686 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e[63526]: 2024-09-10T10:56:55.196+0000 7f62ba4b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:55.686 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:55 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e[63526]: 2024-09-10T10:56:55.196+0000 7f62ba4b3640 -1 mds.e *** got signal Terminated *** 2024-09-10T10:56:58.389 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 podman[69261]: 2024-09-10 10:56:58.055316951 +0000 UTC m=+2.957761158 container died a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T10:56:58.673 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 podman[69261]: 2024-09-10 10:56:58.480020469 +0000 UTC m=+3.382464668 container cleanup a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T10:56:58.673 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 bash[69261]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e 2024-09-10T10:56:58.910 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T10:56:58.931 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 podman[69274]: 2024-09-10 10:56:58.672985591 +0000 UTC m=+0.618720544 container remove a598d1eb540adf2371e4cc66b28f5ae5a58799245b5d85e17997f826bc456910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T10:56:58.931 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.e.service: Deactivated successfully. 2024-09-10T10:56:58.932 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 systemd[1]: Stopped Ceph mds.e for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:56:58.932 INFO:journalctl@ceph.mds.e.smithi144.stdout:Sep 10 10:56:58 smithi144 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.e.service: Consumed 2.698s CPU time. 2024-09-10T10:56:58.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:56:58.942 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T10:56:58.942 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T10:56:58.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.h 2024-09-10T10:56:59.186 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:56:59 smithi144 systemd[1]: Stopping Ceph mds.h for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:56:59.686 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:56:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h[63830]: 2024-09-10T10:56:59.275+0000 7f2e6c561640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:56:59.686 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:56:59 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h[63830]: 2024-09-10T10:56:59.275+0000 7f2e6c561640 -1 mds.h *** got signal Terminated *** 2024-09-10T10:57:02.436 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:57:02 smithi144 podman[69383]: 2024-09-10 10:57:02.042781473 +0000 UTC m=+2.886705191 container died 556e5bc8207e121e85ef0c8af8ceebef72e7b178f168406820bdab690d36b2dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:57:02.785 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:57:02 smithi144 podman[69383]: 2024-09-10 10:57:02.533274032 +0000 UTC m=+3.377197740 container cleanup 556e5bc8207e121e85ef0c8af8ceebef72e7b178f168406820bdab690d36b2dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T10:57:02.785 INFO:journalctl@ceph.mds.h.smithi144.stdout:Sep 10 10:57:02 smithi144 bash[69383]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-h 2024-09-10T10:57:02.966 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T10:57:02.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:57:02.998 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T10:57:02.998 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T10:57:02.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.k 2024-09-10T10:57:03.313 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:03 smithi144 systemd[1]: Stopping Ceph mds.k for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:57:03.686 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k[63222]: 2024-09-10T10:57:03.311+0000 7f1f2ebf1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:57:03.686 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:03 smithi144 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k[63222]: 2024-09-10T10:57:03.311+0000 7f1f2ebf1640 -1 mds.k *** got signal Terminated *** 2024-09-10T10:57:06.379 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 podman[69505]: 2024-09-10 10:57:06.038292072 +0000 UTC m=+2.829036008 container died 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:57:06.646 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 podman[69505]: 2024-09-10 10:57:06.461903672 +0000 UTC m=+3.252647606 container cleanup 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T10:57:06.647 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 bash[69505]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k 2024-09-10T10:57:06.878 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T10:57:06.899 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 podman[69516]: 2024-09-10 10:57:06.646211623 +0000 UTC m=+0.596912708 container remove 0ab487fa515617b398c74c142c73e32b7b4155626e0690bfcce23e5c60794457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T10:57:06.899 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-10T10:57:06.899 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 systemd[1]: Stopped Ceph mds.k for 34212b78-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:57:06.899 INFO:journalctl@ceph.mds.k.smithi144.stdout:Sep 10 10:57:06 smithi144 systemd[1]: ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.k.service: Consumed 1.977s CPU time. 2024-09-10T10:57:06.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:57:06.910 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T10:57:06.910 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T10:57:06.910 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.c 2024-09-10T10:57:07.215 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:57:06 smithi178 systemd[1]: Stopping Ceph mds.c for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:57:07.473 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:57:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c[64979]: 2024-09-10T10:57:07.213+0000 7fa72b251640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:57:07.473 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:57:07 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c[64979]: 2024-09-10T10:57:07.213+0000 7fa72b251640 -1 mds.c *** got signal Terminated *** 2024-09-10T10:57:10.424 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:57:10 smithi178 podman[70131]: 2024-09-10 10:57:10.043212361 +0000 UTC m=+2.935559989 container died 3bb6674ba0afd9399ed999756e3720f12667b34449692a5314af3c9e9c4fefd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T10:57:10.782 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:57:10 smithi178 podman[70131]: 2024-09-10 10:57:10.522515209 +0000 UTC m=+3.414862828 container cleanup 3bb6674ba0afd9399ed999756e3720f12667b34449692a5314af3c9e9c4fefd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T10:57:10.783 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 10 10:57:10 smithi178 bash[70131]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-c 2024-09-10T10:57:10.905 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T10:57:10.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:57:10.941 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T10:57:10.941 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T10:57:10.941 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.f 2024-09-10T10:57:11.174 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:57:11 smithi178 systemd[1]: Stopping Ceph mds.f for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:57:11.674 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:57:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: 2024-09-10T10:57:11.276+0000 7f73add31640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:57:11.675 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:57:11 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f[64680]: 2024-09-10T10:57:11.276+0000 7f73add31640 -1 mds.f *** got signal Terminated *** 2024-09-10T10:57:14.310 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:57:14 smithi178 podman[70264]: 2024-09-10 10:57:14.053224508 +0000 UTC m=+2.892759274 container died 2380f1afd3af9c440cfe028ec34fe438dd91a2959b87d33052bcf8fdd17e9bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T10:57:14.674 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:57:14 smithi178 podman[70264]: 2024-09-10 10:57:14.40994326 +0000 UTC m=+3.249478026 container cleanup 2380f1afd3af9c440cfe028ec34fe438dd91a2959b87d33052bcf8fdd17e9bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:57:14.675 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 10 10:57:14 smithi178 bash[70264]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-f 2024-09-10T10:57:14.844 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T10:57:14.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:57:14.878 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T10:57:14.878 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T10:57:14.878 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.i 2024-09-10T10:57:15.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:57:14 smithi178 systemd[1]: Stopping Ceph mds.i for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:57:15.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:57:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: 2024-09-10T10:57:15.205+0000 7f525c485640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:57:15.425 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:57:15 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i[65282]: 2024-09-10T10:57:15.205+0000 7f525c485640 -1 mds.i *** got signal Terminated *** 2024-09-10T10:57:18.400 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:57:18 smithi178 podman[70381]: 2024-09-10 10:57:18.044589319 +0000 UTC m=+2.946335658 container died ac24ad84cf7156482dfb5af9812ddfc09b599b83f0b1f26eafa9304b20d39bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:57:18.675 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:57:18 smithi178 podman[70381]: 2024-09-10 10:57:18.48058913 +0000 UTC m=+3.382335468 container cleanup ac24ad84cf7156482dfb5af9812ddfc09b599b83f0b1f26eafa9304b20d39bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:57:18.675 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 10 10:57:18 smithi178 bash[70381]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-i 2024-09-10T10:57:18.854 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T10:57:18.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:57:18.888 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T10:57:18.888 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T10:57:18.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.l 2024-09-10T10:57:19.174 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:57:18 smithi178 systemd[1]: Stopping Ceph mds.l for 34212b78-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:57:19.674 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:57:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l[64380]: 2024-09-10T10:57:19.212+0000 7f01e669f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:57:19.675 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:57:19 smithi178 ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l[64380]: 2024-09-10T10:57:19.212+0000 7f01e669f640 -1 mds.l *** got signal Terminated *** 2024-09-10T10:57:22.369 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:57:22 smithi178 podman[70501]: 2024-09-10 10:57:22.04723916 +0000 UTC m=+2.941968011 container died 8978b9fc4ab496c79cc59133bc25e16a894a4dd973925b2da3fd5f626fd06cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:57:22.675 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:57:22 smithi178 podman[70501]: 2024-09-10 10:57:22.459431003 +0000 UTC m=+3.354159846 container cleanup 8978b9fc4ab496c79cc59133bc25e16a894a4dd973925b2da3fd5f626fd06cc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:57:22.675 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 10 10:57:22 smithi178 bash[70501]: ceph-34212b78-6f60-11ef-bcea-c7b262605968-mds-l 2024-09-10T10:57:22.894 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-34212b78-6f60-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T10:57:22.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:57:22.928 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T10:57:22.928 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:57:23.132 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:57:25.502 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:57:25.702 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:57:28.176 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:57:28.377 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T10:57:30.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:57:30.813 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:57:30.840 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:57:30.867 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T10:57:30.869 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi143/crash 2024-09-10T10:57:30.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:57:30.896 INFO:teuthology.orchestra.run.smithi143.stderr:tar: /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:57:30.897 INFO:teuthology.orchestra.run.smithi143.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:57:30.899 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi144/crash 2024-09-10T10:57:30.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:57:30.931 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:57:30.931 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:57:30.933 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi178/crash 2024-09-10T10:57:30.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:57:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/34212b78-6f60-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:57:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:57:30.969 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T10:57:30.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T10:57:31.006 INFO:teuthology.orchestra.run.smithi143.stdout:2024-09-10T10:52:57.351575+0000 mds.f (mds.3) 20 : cluster [WRN] Scrub error on inode 0x400000015e1 (/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.f log and `damage ls` output for details 2024-09-10T10:57:31.006 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-10T10:57:31.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[SEC\]' /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T10:57:31.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]' /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T10:57:31.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[WRN\]' /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T10:57:31.228 INFO:teuthology.orchestra.run.smithi143.stdout:2024-09-10T10:52:57.351575+0000 mds.f (mds.3) 20 : cluster [WRN] Scrub error on inode 0x400000015e1 (/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.f log and `damage ls` output for details 2024-09-10T10:57:31.228 INFO:tasks.cephadm:Compressing logs... 2024-09-10T10:57:31.229 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:57:31.271 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:57:31.274 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:57:31.293 INFO:teuthology.orchestra.run.smithi143.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T10:57:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:: No such file or directory 2024-09-10T10:57:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T10:57:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 92.6% --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:57:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:57:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T10:57:31.301 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T10:57:31.301 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-09-10T10:57:31.301 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T10:57:31.301 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T10:57:31.301 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:57:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log: 86.5% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:57:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T10:57:31.303 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T10:57:31.304 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T10:57:31.304 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T10:57:31.304 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.k.log: /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:57:31.304 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:57:31.305 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:57:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.8% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T10:57:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T10:57:31.309 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:57:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T10:57:31.311 INFO:teuthology.orchestra.run.smithi144.stderr: 90.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:57:31.311 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T10:57:31.313 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T10:57:31.313 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.8% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:57:31.313 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:57:31.314 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:57:31.314 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:57:31.314 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T10:57:31.315 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T10:57:31.317 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T10:57:31.317 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T10:57:31.319 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T10:57:31.320 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:57:31.321 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.y.log: 92.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T10:57:31.321 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T10:57:31.323 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log: 86.4% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:57:31.323 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T10:57:31.328 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:57:31.341 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T10:57:31.350 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:57:31.362 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:57:31.371 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:57:31.393 INFO:teuthology.orchestra.run.smithi143.stderr: 94.3% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T10:57:31.400 INFO:teuthology.orchestra.run.smithi144.stderr: 94.6% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T10:57:31.426 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 86.4% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:57:31.429 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:57:31.493 INFO:teuthology.orchestra.run.smithi178.stderr: 82.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:57:31.516 INFO:teuthology.orchestra.run.smithi178.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:57:31.551 INFO:teuthology.orchestra.run.smithi178.stderr: 92.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T10:57:31.574 INFO:teuthology.orchestra.run.smithi143.stderr: 90.5% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T10:57:31.613 INFO:teuthology.orchestra.run.smithi178.stderr: 90.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:57:31.754 INFO:teuthology.orchestra.run.smithi144.stderr: 91.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T10:57:31.944 INFO:teuthology.orchestra.run.smithi144.stderr: 95.1% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T10:57:32.434 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T10:57:32.461 INFO:teuthology.orchestra.run.smithi143.stderr: 90.4% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T10:57:34.604 INFO:teuthology.orchestra.run.smithi144.stderr: 94.2% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T10:57:35.308 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T10:57:38.184 INFO:teuthology.orchestra.run.smithi144.stderr: 92.0% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T10:57:39.735 INFO:teuthology.orchestra.run.smithi178.stderr: 94.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T10:57:40.071 INFO:teuthology.orchestra.run.smithi144.stderr: 92.1% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T10:57:41.230 INFO:teuthology.orchestra.run.smithi144.stderr: 91.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T10:57:41.924 INFO:teuthology.orchestra.run.smithi143.stderr: 92.0% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T10:57:42.295 INFO:teuthology.orchestra.run.smithi143.stderr: 91.8% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T10:57:45.719 INFO:teuthology.orchestra.run.smithi143.stderr: 93.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T10:57:45.929 INFO:teuthology.orchestra.run.smithi143.stderr: 91.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T10:57:46.556 INFO:teuthology.orchestra.run.smithi144.stderr: 91.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T10:57:46.558 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-10T10:57:46.558 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m15.272s 2024-09-10T10:57:46.558 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m44.547s 2024-09-10T10:57:46.558 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m1.462s 2024-09-10T10:57:47.802 INFO:teuthology.orchestra.run.smithi143.stderr: 91.9% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T10:57:53.671 INFO:teuthology.orchestra.run.smithi178.stderr: 94.1% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T10:57:59.283 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T10:58:10.031 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T10:58:12.128 INFO:teuthology.orchestra.run.smithi178.stderr: 91.8% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T10:58:14.473 INFO:teuthology.orchestra.run.smithi178.stderr: 94.4% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T10:58:26.749 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T10:59:22.799 INFO:teuthology.orchestra.run.smithi143.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-10T10:59:22.801 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-09-10T10:59:22.801 INFO:teuthology.orchestra.run.smithi143.stderr:real 1m51.518s 2024-09-10T10:59:22.801 INFO:teuthology.orchestra.run.smithi143.stderr:user 2m54.297s 2024-09-10T10:59:22.802 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m5.453s 2024-09-10T11:01:30.563 INFO:teuthology.orchestra.run.smithi178.stderr: 94.1% -- replaced with /var/log/ceph/34212b78-6f60-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T11:01:30.565 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-10T11:01:30.565 INFO:teuthology.orchestra.run.smithi178.stderr:real 3m59.276s 2024-09-10T11:01:30.565 INFO:teuthology.orchestra.run.smithi178.stderr:user 4m3.712s 2024-09-10T11:01:30.565 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m8.723s 2024-09-10T11:01:30.565 INFO:tasks.cephadm:Archiving logs... 2024-09-10T11:01:30.566 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi143/log 2024-09-10T11:01:30.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:01:57.416 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi144/log 2024-09-10T11:01:57.417 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:02:04.331 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi178/log 2024-09-10T11:02:04.332 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:02:40.157 INFO:tasks.cephadm:Removing cluster... 2024-09-10T11:02:40.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --force 2024-09-10T11:02:40.359 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T11:02:40.766 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --force 2024-09-10T11:02:40.968 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T11:02:41.285 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 34212b78-6f60-11ef-bcea-c7b262605968 --force 2024-09-10T11:02:41.488 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 34212b78-6f60-11ef-bcea-c7b262605968 2024-09-10T11:02:41.994 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T11:02:41.994 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T11:02:42.014 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T11:02:42.031 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T11:02:42.048 INFO:tasks.cephadm:Teardown complete 2024-09-10T11:02:42.048 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T11:02:42.058 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T11:02:42.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:02:42.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:02:42.073 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:02:42.171 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T11:02:42.181 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T11:02:42.182 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:02:42.185 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:02:42.188 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:02:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:02:42.200 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:02:42.202 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:02:42.204 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:02:42.204 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-09-10T11:02:42.204 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -322us[ -308us] +/- 48ms 2024-09-10T11:02:42.204 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +105us[ +119us] +/- 75ms 2024-09-10T11:02:42.204 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +446us[ +459us] +/- 75ms 2024-09-10T11:02:42.204 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:02:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:02:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-09-10T11:02:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -320us[ -299us] +/- 48ms 2024-09-10T11:02:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +99us[ +112us] +/- 74ms 2024-09-10T11:02:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +476us[ +476us] +/- 76ms 2024-09-10T11:02:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:02:42.225 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:02:42.226 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-10T11:02:42.226 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 -333us[ -327us] +/- 48ms 2024-09-10T11:02:42.226 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 +86us[ +86us] +/- 75ms 2024-09-10T11:02:42.226 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +462us[ +464us] +/- 75ms 2024-09-10T11:02:42.226 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:02:42.226 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T11:02:42.249 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T11:02:42.249 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T11:02:42.338 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:02:42.427 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:02:42.495 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:02:42.610 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:02:42.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:42.642 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:02:42.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:42.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:02:42.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:42.699 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T11:02:42.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T11:02:42.733 INFO:teuthology.task.internal:Duration was 2046.145671 seconds 2024-09-10T11:02:42.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T11:02:42.776 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T11:02:42.776 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:02:42.779 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:02:42.781 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:02:42.818 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:02:42.819 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:02:42.853 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:02:43.251 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T11:02:43.252 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-09-10T11:02:43.252 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:02:43.277 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:02:43.277 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:02:43.300 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-09-10T11:02:43.301 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:02:43.331 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T11:02:43.331 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:02:43.334 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:02:43.350 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:02:43.406 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T11:02:43.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:02:43.414 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:02:43.432 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:02:44.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T11:02:44.652 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T11:02:44.652 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:02:44.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:02:44.708 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:02:44.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T11:02:44.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:02:44.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:02:44.770 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:02:44.788 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-09-10T11:02:44.795 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-09-10T11:02:44.816 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-09-10T11:02:44.866 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:02:44.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:44.881 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:02:44.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:44.898 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:02:44.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:44.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T11:02:44.933 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T11:02:44.935 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi143 2024-09-10T11:02:44.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:02:44.999 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi144 2024-09-10T11:02:44.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:02:45.037 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899033/remote/smithi178 2024-09-10T11:02:45.037 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:02:45.075 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T11:02:45.075 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:02:45.078 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:02:45.080 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:02:45.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T11:02:45.138 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T11:02:45.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T11:02:45.178 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T11:02:45.179 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:02:45.181 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:02:45.183 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:02:45.194 INFO:teuthology.orchestra.run.smithi143.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:02 /home/ubuntu/cephtest 2024-09-10T11:02:45.197 INFO:teuthology.orchestra.run.smithi144.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:02 /home/ubuntu/cephtest 2024-09-10T11:02:45.198 INFO:teuthology.orchestra.run.smithi178.stdout: 265972 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:02 /home/ubuntu/cephtest 2024-09-10T11:02:45.200 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T11:02:45.211 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T11:02:45.361 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2046.1456711292267 failure_reason: '"2024-09-10T10:52:57.351575+0000 mds.f (mds.3) 20 : cluster [WRN] Scrub error on inode 0x400000015e1 (/volumes/qa/sv_0/e30d7478-7fa9-47ad-a2b8-7faadc44ff73/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.f log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_rishabh@teuthology success: false 2024-09-10T11:02:45.361 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T11:02:45.452 INFO:teuthology.run:FAIL